-
Bug
-
Resolution: Obsolete
-
Major
-
None
-
1.9.0.Final, 1.9.5.Final
-
False
-
None
-
False
In order to make your issue reports as actionable as possible, please provide the following information, depending on the issue type.
Bug report
For bug reports, provide this information, please:
What Debezium connector do you use and what version?
debezium/server 1.9.5.Final with the following connector/sink type.
debezium.sink.type=pubsub
debezium.source.connector.class=io.debezium.connector.mysql.MySqlConnector
What is the connector configuration?
debezium.sink.pubsub.ordering.enabled=true
debezium.sink.pubsub.project.id=kenect-service
debezium.sink.type=pubsub
debezium.source.connector.class=io.debezium.connector.mysql.MySqlConnector
debezium.source.database.dbname=myDatabase
debezium.source.database.history.file.filename=/debezium/data/history.dat
debezium.source.database.history.skip.unparseable.ddl=true
debezium.source.database.history.store.only.captured.tables.ddl=true
debezium.source.database.history=io.debezium.relational.history.FileDatabaseHistory
debezium.source.database.hostname=127.0.0.1
debezium.source.database.port=3306
debezium.source.database.server.name=myDatabase
debezium.source.event.processing.failure.handling.mode=warn
debezium.source.include.schema.changes=false
debezium.source.inconsistent.schema.handling.mode=warn
debezium.source.key.converter.schemas.enable=false
debezium.source.max.queue.size=81290
debezium.source.offset.flush.interval.ms=0
debezium.source.offset.storage.file.filename=/debezium/data/offsets.dat
debezium.source.tombstones.on.delete=false
debezium.source.snapshot.locking.mode=none
debezium.source.snapshot.mode=schema_only
debezium.source.table.include.list=myDatabase.channel,myDatabase.contact_channel,myDatabase.conv,myDatabase.heartbeat,myDatabase.local_destination,myDatabase.message,myDatabase.message_resource,myDatabase.team,myDatabase.team_member,myDatabase.notification_subscription,myDatabase.legal_mat
debezium.source.value.converter.schemas.enable=false
debezium.transforms.Reroute.topic.regex=[^.]+(.*)
debezium.transforms.Reroute.topic.replacement=debezium$1
debezium.transforms.Reroute.type=io.debezium.transforms.ByLogicalTableRouter
What is the captured database version and mode of depoyment?
(E.g. on-premises, with a specific cloud provider, etc.)
Running debezium in a kubernetes pod using the debezium/server image 1.9.0.Final. The database is a Google Cloudsql mysql version 8 database. This issue exists with both mysql5 and mysql8. At least in Google CloudSql.
What behaviour do you expect?
Function normally.
What behaviour do you see?
Debezium stops processing any changes until we restart the debezium process.
Do you see the same behaviour using the latest relesead Debezium version?
This behavior exists in version 1.9.0.Final and also 1.9.5.Final
Do you have the connector logs, ideally from start till finish?
Logs provided before and near the incident. Incident started at time period 7:57 and a restart was issued at 8:03
2022-09-21 03:48:02.473 MDT 1 records sent during previous 02:00:01.592, last recorded offset: {transaction_id=null, ts_sec=1663753682, file=mysql-bin.021685, pos=94112103, gtids=06ea70dd-1f81-11ed-8866-4201ac1fd03a:1-35376948,0c51eaff-74b8-11e8-88c1-42010a800129:1-62523206,6b50bb0b-99fe-11e9-abaf-42010a800591:1-1032604843,8300061e-cb5b-11e8-8754-42010a80013e:1-34594475,ee829197-2638-11e8-b4d8-42010a80036b:1-12687247, row=1, server_id=2627354887, event=3}
2022-09-21 05:48:01.636 MDT 3 records sent during previous 01:59:59.163, last recorded offset: {transaction_id=null, ts_sec=1663760881, file=mysql-bin.021685, pos=101217646, gtids=06ea70dd-1f81-11ed-8866-4201ac1fd03a:1-35385785,0c51eaff-74b8-11e8-88c1-42010a800129:1-62523206,6b50bb0b-99fe-11e9-abaf-42010a800591:1-1032604843,8300061e-cb5b-11e8-8754-42010a80013e:1-34594475,ee829197-2638-11e8-b4d8-42010a80036b:1-12687247, row=1, server_id=2627354887, event=5}
2022-09-21 07:48:01.171 MDT 12 records sent during previous 01:59:59.535, last recorded offset: {transaction_id=null, ts_sec=1663768080, file=mysql-bin.021686, pos=79661307, gtids=06ea70dd-1f81-11ed-8866-4201ac1fd03a:1-35473297,0c51eaff-74b8-11e8-88c1-42010a800129:1-62523206,6b50bb0b-99fe-11e9-abaf-42010a800591:1-1032604843,8300061e-cb5b-11e8-8754-42010a80013e:1-34594475,ee829197-2638-11e8-b4d8-42010a80036b:1-12687247, row=1, server_id=2627354887, event=2}
2022-09-21 08:03:27.014 MDT Stopped reading binlog after 0 events, last recorded offset: {transaction_id=null, ts_sec=1663769006, file=mysql-bin.021686, pos=97118600, gtids=06ea70dd-1f81-11ed-8866-4201ac1fd03a:1-35491395,0c51eaff-74b8-11e8-88c1-42010a800129:1-62523206,6b50bb0b-99fe-11e9-abaf-42010a800591:1-1032604843,8300061e-cb5b-11e8-8754-42010a80013e:1-34594475,ee829197-2638-11e8-b4d8-42010a80036b:1-12687247, server_id=2627354887, event=1}
2022-09-21 08:03:27.047 MDT Received request to stop the engine
2022-09-21 08:03:27.047 MDT Stopping the embedded engine
2022-09-21 08:03:27.047 MDT Waiting for PT5M for connector to stop
How to reproduce the issue using our tutorial deployment?
Being able to reproduce on demand has not been identified. The issue happens between a few hours to a few days before presenting.
This behavior does exist on two separate instances of debezium, connecting to two different mysql 8 databases with similar configuration.