Details
-
Bug
-
Resolution: Not a Bug
-
Major
-
None
-
1.3.0.Final
-
None
-
False
-
False
-
Undefined
-
- create a memory table
- start debezium connector
- restart mysql database
Description
We're running debezium with MySQL 8 and the connector restricted to special tables (not the memory tables).
When the mysql database is restarted, it issues DELETE calls on the memory tables, as outlined here: https://dev.mysql.com/doc/refman/8.0/en/replication-features-memory.html
This causes the mysql-connector to crash and cannot be restarted.
2020-11-12 07:47:07,968 ERROR || WorkerSourceTask{id=debezium-ticketshop-producer-0} Task threw an uncaught and unrecoverable exception [org.apache.kafka.connect.runtime.WorkerTask]
org.apache.kafka.connect.errors.ConnectException: Received DML 'DELETE FROM `mydb`.`mymemorytable` /* generated by server, implicitly emptying in-memory table */' for processing, binlog probably contains events generated with statement or mixed based replication format
at io.debezium.connector.mysql.AbstractReader.wrap(AbstractReader.java:230)
at io.debezium.connector.mysql.AbstractReader.failed(AbstractReader.java:207)
at io.debezium.connector.mysql.BinlogReader.handleEvent(BinlogReader.java:604)
at com.github.shyiko.mysql.binlog.BinaryLogClient.notifyEventListeners(BinaryLogClient.java:1100)
at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:951)
at com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:594)
at com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:838)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: org.apache.kafka.connect.errors.ConnectException: Received DML 'DELETE FROM `cosmoshop`.`shoppreiscache` /* generated by server, implicitly emptying in-memory table */' for processing, binlog probably contains events generated with statement or mixed based replication format
at io.debezium.connector.mysql.BinlogReader.handleQueryEvent(BinlogReader.java:790)
at io.debezium.connector.mysql.BinlogReader.handleEvent(BinlogReader.java:587)
... 5 more
2020-11-12 07:47:07,969 ERROR || WorkerSourceTask{id=debezium-ticketshop-producer-0} Task is being killed and will not recover until manually restarted [org.apache.kafka.connect.runtime.WorkerTask]
Although mysql database outlines methods to circumvent the DELETE calls, it explicitely states, that this is not recommended if not running in NDB cluster.
