-
Bug
-
Resolution: Obsolete
-
Major
-
None
-
False
-
-
False
-
Important
Bug report
Using debezium source connector to mariadb for shiping binlogs to nats-jetstream
everything goes fine until I get
io.debezium.connector.binlog.BinlogStreamingChangeEventSource","level":"WARN","message":"Error during binlog processing.
What Debezium connector do you use and what version?
debezium.source.connector.class=io.debezium.connector.mariadb.MariaDbConnector
quay.io/debezium/server:3.2.1
What is the connector configuration?
#Debezium Source: MariaDB debezium.source.connector.class=io.debezium.connector.mariadb.MariaDbConnector debezium.source.database.hostname=hostname.dk5.tv debezium.source.database.port=3306 debezium.source.database.user=debezium_slip_log debezium.source.database.password=PASS debezium.source.database.server.id=1337 debezium.source.database.include.list=advbet_web_test debezium.source.table.include.list=advbet_web_test.slip_log #Debezium Topic/Format debezium.source.topic.prefix=debezium debezium.source.topic.capture.mode=change_streams debezium.format.value=json #Offset and Schema History Storage debezium.source.offset.storage.file.filename=/debezium/data/offsets.dat debezium.source.schema.history.internal=io.debezium.storage.file.history.FileSchemaHistory debezium.source.schema.history.internal.file.filename=/debezium/data/schema-history.dat debezium.source.include.schema.changes=true debezium.source.snapshot.mode=initial #Error Handling debezium.source.skip.messages.without.change=true debezium.source.event.processing.failure.handling.mode=warn debezium.source.event.converting.failure.handling.mode=warn debezium.source.skipped.operations=u,d,t #Logging debezium.log.level=TRACE #Debezium Sink: NATS JetStream debezium.sink.type=nats-jetstream debezium.sink.nats-jetstream.url=tls://hostname01.dk5.tv:4222,tls://hostname02.dk5.tv:4222,tls://hostname03.dk5.tv:4222 #TLS Authentication debezium.sink.nats-jetstream.auth.tls.keystore=/debezium/config/keystore.p12 debezium.sink.nats-jetstream.auth.tls.keystore.password=PASS debezium.sink.nats-jetstream.auth.tls.password=PASS #JetStream Stream Settings debezium.sink.nats-jetstream.create-stream=false debezium.sink.nats-jetstream.stream=db_changes_stream debezium.sink.nats-jetstream.subjects=debezium,debezium.> debezium.sink.nats-jetstream.auth.user=debezium debezium.sink.nats-jetstream.auth.password=PASS
What is the captured database version and mode of deployment?
on hetzner cloud hardware using kvm
MariaDB-server.x86_64 10.11.14-1.el9
What behavior do you expect?
<Your answer>
What behavior do you see?
Using debezium source connector to mariadb for shiping binlogs to nats-jetstream
everything goes fine until I get
{"timestamp":"2025-09-12T13:15:23.511281715Z","sequence":16415,"loggerClassName":"org.slf4j.impl.Slf4jLogger","loggerName":"io.debezium.connector.binlog.BinlogStreamingChangeEventSource","level":"WARN","message":"A deserialization failure event arrived","threadName":"blc-adv-prod-online-test-srv01.duok5.tv:3306","threadId":46,"mdc":{"dbz.connectorLogicalName":"debezium","dbz.taskId":"0","dbz.connectorType":"MariaDB","dbz.connectorContext":"binlog"},"ndc":"","hostName":"64a025d9b9c0","processName":"/usr/lib/jvm/java-21-openjdk-21.0.8.0.9-1.el8.x86_64/bin/java","processId":1,"exception":{"refId":1,"exceptionType":"com.github.shyiko.mysql.binlog.event.deserialization.EventDataDeserializationException","message":"Failed to deserialize data of EventHeaderV4{timestamp=1757682923000, eventType=WRITE_ROWS, serverId=2420921427, headerLength=19, dataLength=2005, nextPosition=1147945, flags=0}","frames":[{"class<span class="code-quote">":"com.github.shyiko.mysql.binlog.event.deserialization.EventDeserializer","method":"deserializeEventData","line":343},{"class<span class="code-quote">":"com.github.shyiko.mysql.binlog.event.deserialization.EventDeserializer","method":"nextEvent","line":246},{"class<span class="code-quote">":"io.debezium.connector.binlog.BinlogStreamingChangeEventSource$1","method":"nextEvent","line":425},{"class<span class="code-quote">":"com.github.shyiko.mysql.binlog.BinaryLogClient","method":"listenForEventPackets","line":1082},{"class<span class="code-quote">":"com.github.shyiko.mysql.binlog.BinaryLogClient","method":"connect","line":657},{"class<span class="code-quote">":"com.github.shyiko.mysql.binlog.BinaryLogClient$7","method":"run","line":959},{"class<span class="code-quote">":"java.lang.Thread","method":"run","line":1583}],"causedBy":{"exception":{"refId":2,"exceptionType":"com.github.shyiko.mysql.binlog.event.deserialization.MissingTableMapEventException","message":"No TableMapEventData has been found for table id:6276. Usually that means that you have started reading binary log 'within the logical event group' (e.g. from WRITE_ROWS and not proceeding TABLE_MAP","frames":[{"class<span class="code-quote">":"com.github.shyiko.mysql.binlog.event.deserialization.AbstractRowsEventDataDeserializer","method":"deserializeRow","line":109},{"class<span class="code-quote">":"com.github.shyiko.mysql.binlog.event.deserialization.WriteRowsEventDataDeserializer","method":"deserializeRows","line":64},{"class<span class="code-quote">":"com.github.shyiko.mysql.binlog.event.deserialization.WriteRowsEventDataDeserializer","method":"deserialize","line":56},{"class<span class="code-quote">":"com.github.shyiko.mysql.binlog.event.deserialization.WriteRowsEventDataDeserializer","method":"deserialize","line":32},{"class<span class="code-quote">":"com.github.shyiko.mysql.binlog.event.deserialization.EventDeserializer","method":"deserializeEventData","line":337},{"class<span class="code-quote">":"com.github.shyiko.mysql.binlog.event.deserialization.EventDeserializer","method":"nextEvent","line":246},{"class<span class="code-quote">":"io.debezium.connector.binlog.BinlogStreamingChangeEventSource$1","method":"nextEvent","line":425},{"class<span class="code-quote">":"com.github.shyiko.mysql.binlog.BinaryLogClient","method":"listenForEventPackets","line":1082},{"class<span class="code-quote">":"com.github.shyiko.mysql.binlog.BinaryLogClient","method":"connect","line":657},{"class<span class="code-quote">":"com.github.shyiko.mysql.binlog.BinaryLogClient$7","method":"run","line":959},{"class<span class="code-quote">":"java.lang.Thread","method":"run","line":1583}]}}}}
{"timestamp":"2025-09-12T13:15:23.511365563Z","sequence":16416,"loggerClassName":"org.slf4j.impl.Slf4jLogger","loggerName":"io.debezium.connector.binlog.BinlogStreamingChangeEventSource","level":"WARN","message":"Error during binlog processing. Last offset stored = {ts_sec=1757682730, file=mariadb-bin.219995, pos=1114640, gtids=0-2420921427-2869908247, server_id=2420921427, event=4}, binlog reader near position = mariadb-bin.219995/1114684","threadName":"blc-adv-prod-online-test-srv01.duok5.tv:3306","threadId":46,"mdc":{"dbz.connectorLogicalName":"debezium","dbz.taskId":"0","dbz.connectorType":"MariaDB","dbz.connectorContext":"binlog"},"ndc":"","hostName":"64a025d9b9c0","processName":"/usr/lib/jvm/java-21-openjdk-21.0.8.0.9-1.el8.x86_64/bin/java","processId":1}
Additional information related to database
binlog compression is disabled as well:
log_bin_compress | OFF binlog_format | ROW
table schema:
MariaDB ******** SLAVE ******** READ ONLY ******** [web_test]> describe slip_log;
--------{}{}{}{}{}-----------{}{}{}{}{}{}{}{}{}{}{}{}{}{}{}{}{}{}-------------{}{}{}{}{}{}{}{}-----------{+}
| Field | Type | Null | Key | Default | Extra |
--------{}{}{}{}{}-----------{}{}{}{}{}{}{}{}{}{}{}{}{}{}{}{}{}{}-------------{}{}{}{}{}{}{}{}-----------{+}
| slip_log_id | bigint(20) unsigned | NO | PRI | NULL | auto_increment |
| ts | timestamp(6) | NO | MUL | current_timestamp(6) | |
| slip_uuid | binary(16) | NO | MUL | NULL | |
| slip_id | bigint(20) unsigned | NO | NULL | ||
| website_id | bigint(20) unsigned | YES | MUL | NULL | |
| marshaled | mediumblob | YES | NULL |
--------{}{}{}{}{}-----------{}{}{}{}{}{}{}{}{}{}{}{}{}{}{}{}{}{}-------------{}{}{}{}{}{}{}{}-----------{+}