-
Bug
-
Resolution: Not a Bug
-
Major
-
None
-
2.7.0.Final
-
None
-
False
-
None
-
False
In order to make your issue reports as actionable as possible, please provide the following information, depending on the issue type.
Bug report
For bug reports, provide this information, please:
What Debezium connector do you use and what version?
<Your answer> 2.7.0
What is the connector configuration?
{
"name":"mysql-debezium-connector_oauth_login",
"config":{
"connector.class":"io.debezium.connector.mysql.MySqlConnector",
"database.hostname":"localhost",
"database.port":"3306",
"database.user":"mungmo",
"database.password":"1234",
"database.server.id":"1",
"database.server.name":"mungmoMember_server",
"database.include.list":"mungmoMember",
"table.include.list":"mungmoMember.oauth_login",
"database.history.kafka.bootstrap.servers":"localhost:9092",
"database.history.kafka.topic":"oauth_login_history",
"schema.history.internal":"io.debezium.storage.file.history.FileSchemaHistory",
"schema.history.internal.file.filename":"/Users/bighwang/Documents/workspace/MungMo/kafka_2.13-3.7.0/connect/storage/schemahistory.dat",
"topic.prefix":"test108",
"include.schema.changes":"true",
"primary.key.mode":"record_key",
"primary.key.fields":"member_id",
"auto.leader.rebalance.enable":"true",
"transforms":"unwrap",
"transforms.unwrap.type":"io.debezium.transforms.ExtractNewRecordState",
"topic.creation.default.replication.factor":1,
"topic.creation.default.partitions":1,
"snapshot.mode":"initial",
"key.converter":"io.confluent.connect.avro.AvroConverter",
"key.converter.schema.registry.url":"http://localhost:8081",
"value.converter":"io.confluent.connect.avro.AvroConverter",
"value.converter.schema.registry.url":"http://localhost:8081",
"key.converter.schemas.enable":"true",
"value.converter.schemas.enable":"true"
}
What is the captured database version and mode of deployment?
(E.g. on-premises, with a specific cloud provider, etc.)
<Your answer>mysql 8.3
What behavior do you expect?
<Your answer> error removal
What behavior do you see?
[2024-07-12 20:22:10,662] ERROR [mysql-debezium-connector_oauth_login|task-0] Encountered change event 'Event\{header=EventHeaderV4{timestamp=1720783330000, eventType=TABLE_MAP, serverId=1, headerLength=19, dataLength=89, nextPosition=4904, flags=0}, data=TableMapEventData\{tableId=479, database='mungmomember', table='oauth_login', columnTypes=3, 18, 8, 18, 8, 15, 15, 15, 15, 15, 15, 15, 15, 15, -2, -2, columnMetadata=0, 6, 0, 6, 0, 765, 765, 765, 765, 765, 765, 765, 765, 765, 63233, 63233, columnNullability={1, 3, 4, 6, 8, 9, 11, 12, 13, 14, 15}, eventMetadata=TableMapEventMetadata\{signedness={}, defaultCharset=33, charsetCollations=null, columnCharsets=null, columnNames=null, setStrValues=null, enumStrValues=null, geometryTypes=null, simplePrimaryKeys=null, primaryKeysWithPrefix=null, enumAndSetDefaultCharset=null, enumAndSetColumnCharsets=null,visibility=null}}}' at offset \{ts_sec=1720783313, file=mysql-bin.000003, pos=4702, server_id=1, event=1} for table mungmomember.oauth_login whose schema isn't known to this connector. One possible cause is an incomplete database schema history topic. Take a new snapshot in this case.
Use the mysqlbinlog tool to view the problematic event: mysqlbinlog --start-position=4796 --stop-position=4904 --verbose mysql-bin.000003 (io.debezium.connector.binlog.BinlogStreamingChangeEventSource:979)
[2024-07-12 20:22:10,662] ERROR [mysql-debezium-connector_oauth_login|task-0] Error during binlog processing. Last offset stored = \{ts_sec=1720783313, file=mysql-bin.000003, pos=4702, server_id=1, event=1}, binlog reader near position = mysql-bin.000003/4796 (io.debezium.connector.binlog.BinlogStreamingChangeEventSource:1145)
[2024-07-12 20:22:10,663] ERROR [mysql-debezium-connector_oauth_login|task-0] Producer failure (io.debezium.pipeline.ErrorHandler:52)
io.debezium.DebeziumException: Error processing binlog event
at io.debezium.connector.binlog.BinlogStreamingChangeEventSource.handleEvent(BinlogStreamingChangeEventSource.java:579)
at io.debezium.connector.binlog.BinlogStreamingChangeEventSource.lambda$execute$17(BinlogStreamingChangeEventSource.java:209)
at com.github.shyiko.mysql.binlog.BinaryLogClient.notifyEventListeners(BinaryLogClient.java:1268)
at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:1094)
at com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:653)
at com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:954)
at java.base/java.lang.Thread.run(Thread.java:840)
Caused by: io.debezium.DebeziumException: Encountered change event for table mungmomember.oauth_login whose schema isn't known to this connector
at io.debezium.connector.binlog.BinlogStreamingChangeEventSource.informAboutUnknownTableIfRequired(BinlogStreamingChangeEventSource.java:984)
at io.debezium.connector.binlog.BinlogStreamingChangeEventSource.informAboutUnknownTableIfRequired(BinlogStreamingChangeEventSource.java:1036)
at io.debezium.connector.binlog.BinlogStreamingChangeEventSource.handleUpdateTableMetadata(BinlogStreamingChangeEventSource.java:785)
at io.debezium.connector.binlog.BinlogStreamingChangeEventSource.lambda$execute$4(BinlogStreamingChangeEventSource.java:178)
at io.debezium.connector.binlog.BinlogStreamingChangeEventSource.handleEvent(BinlogStreamingChangeEventSource.java:559)
... 6 more
[2024-07-12 20:22:10,663] INFO [mysql-debezium-connector_oauth_login|task-0] Error processing binlog event, and propagating to Kafka Connect so it stops this connector. Future binlog events read before connector is shutdown will be ignored. (io.debezium.connector.binlog.BinlogStreamingChangeEventSource:584)
[2024-07-12 20:22:11,079] ERROR [mysql-debezium-connector_oauth_login|task-0] WorkerSourceTask\{id=mysql-debezium-connector_oauth_login-0} Task threw an uncaught and unrecoverable exception. Task is being killed and will not recover until manually restarted (org.apache.kafka.connect.runtime.WorkerTask:212)
org.apache.kafka.connect.errors.ConnectException: An exception occurred in the change event producer. This connector will be stopped.
at io.debezium.pipeline.ErrorHandler.setProducerThrowable(ErrorHandler.java:67)
at io.debezium.connector.binlog.BinlogStreamingChangeEventSource.handleEvent(BinlogStreamingChangeEventSource.java:579)
at io.debezium.connector.binlog.BinlogStreamingChangeEventSource.lambda$execute$17(BinlogStreamingChangeEventSource.java:209)
at com.github.shyiko.mysql.binlog.BinaryLogClient.notifyEventListeners(BinaryLogClient.java:1268)
at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:1094)
at com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:653)
at com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:954)
at java.base/java.lang.Thread.run(Thread.java:840)
Caused by: io.debezium.DebeziumException: Error processing binlog event
... 7 more
Caused by: io.debezium.DebeziumException: Encountered change event for table mungmomember.oauth_login whose schema isn't known to this connector
at io.debezium.connector.binlog.BinlogStreamingChangeEventSource.informAboutUnknownTableIfRequired(BinlogStreamingChangeEventSource.java:984)
at io.debezium.connector.binlog.BinlogStreamingChangeEventSource.informAboutUnknownTableIfRequired(BinlogStreamingChangeEventSource.java:1036)
at io.debezium.connector.binlog.BinlogStreamingChangeEventSource.handleUpdateTableMetadata(BinlogStreamingChangeEventSource.java:785)
at io.debezium.connector.binlog.BinlogStreamingChangeEventSource.lambda$execute$4(BinlogStreamingChangeEventSource.java:178)
at io.debezium.connector.binlog.BinlogStreamingChangeEventSource.handleEvent(BinlogStreamingChangeEventSource.java:559)
... 6 more
Do you see the same behaviour using the latest released Debezium version?
(Ideally, also verify with latest Alpha/Beta/CR version)
<Your answer> y
Do you have the connector logs, ideally from start till finish?
(You might be asked later to provide DEBUG/TRACE level log)
<Your answer>
How to reproduce the issue using our tutorial deployment?
<Your answer>
Feature request or enhancement
For feature requests or enhancements, provide this information, please:
Which use case/requirement will be addressed by the proposed feature?
<Your answer>
Implementation ideas (optional)
<Your answer>
