-
Bug
-
Resolution: Unresolved
-
Major
-
None
Architecture overview:
Debezium is configured to access Mysql cluster with master-slave replication mode through HAProxy, GTID enabled.
At first run, everything works as expected.
When Mysql instance where Debezium and underlying mysql-binlog-connector-java are connected crashes, mysql binlog client will try to reconnect, connection will be directed to new instance but instance will have different position inside binary log and task will be terminated.
Mysql replication error code 1236 was occurred, here’s the log.
When kafka connect task is restarted, only GTID info is set to binary log client, without binlog filename and position info and everything works
connect | 2018-03-26 13:44:02,384 INFO MySQL|dbserver1|binlog Stopped reading binlog after 12 events, last recorded offset:
Unknown macro: {ts_sec=1522071785, file=binlogfilename.000001, pos=12560, gtids=c197da1d-30f9-11e8-9aa3-0242ac1a0008}[io.debezium.connector.mysql.BinlogReader]
connect | 2018-03-26 13:44:40,637 INFO || WorkerSourceTaskUnknown macro: {id=ride-connector-0}Committing offsets [org.apache.kafka.connect.runtime.WorkerSourceTask]
connect | 2018-03-26 13:44:40,637 INFO || WorkerSourceTaskflushing 0 outstanding messages for offset commit [org.apache.kafka.connect.runtime.WorkerSourceTask]
connect | Mar 26, 2018 1:44:41 PM com.github.shyiko.mysql.binlog.BinaryLogClient$5 run
connect | INFO: Trying to restore lost connection to haproxy:3306
connect | 2018-03-26 13:44:41,887 INFO MySQL|dbserver1|binlog Creating thread debezium-mysqlconnector-dbserver1-binlog-client [io.debezium.util.Threads]
connect | Mar 26, 2018 1:44:41 PM com.github.shyiko.mysql.binlog.BinaryLogClient connect
connect | INFO: Connected to haproxy:3306 at c197da1d-30f9-11e8-9aa3-0242ac1a0008:1-37 (sid:184054, cid:361)
connect | 2018-03-26 13:44:41,987 INFO MySQL|dbserver1|binlog Connected to MySQL binlog at haproxy:3306, starting at GTIDs c197da1d-30f9-11e8-9aa3-0242ac1a0008:1-37 and binlog file 'binlogfilename.000001', pos=12867, skipping 1 events plus 0 rows [io.debezium.connector.mysql.BinlogReader]
connect | 2018-03-26 13:44:42,033 ERROR MySQL|dbserver1|binlog Error during binlog processing. Last offset stored =Unknown macro: {ts_sec=1522071785, file=binlogfilename.000001, pos=12560, gtids=c197da1d-30f9-11e8-9aa3-0242ac1a0008}, binlog reader near position = binlogfilename.000001/123 [io.debezium.connector.mysql.BinlogReader]
connect | 2018-03-26 13:44:42,035 INFO || WorkerSourceTaskUnknown macro: {id=ride-connector-0}Committing offsets [org.apache.kafka.connect.runtime.WorkerSourceTask]
connect | 2018-03-26 13:44:42,035 INFO MySQL|dbserver1|binlog Stopped reading binlog after 12 events, last recorded offset:Unknown macro: {ts_sec=1522071785, file=binlogfilename.000001, pos=12560, gtids=c197da1d-30f9-11e8-9aa3-0242ac1a0008}[io.debezium.connector.mysql.BinlogReader]