Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-9463

Error during binlog processing

XMLWordPrintable

    • False
    • Hide

      None

      Show
      None
    • False
    • Important

      Bug report

      Using debezium source connector to mariadb for shiping binlogs to nats-jetstream
      everything goes fine until I get 

      io.debezium.connector.binlog.BinlogStreamingChangeEventSource","level":"WARN","message":"Error during binlog processing.

       

      What Debezium connector do you use and what version?

      debezium.source.connector.class=io.debezium.connector.mariadb.MariaDbConnector
      quay.io/debezium/server:3.2.1

      What is the connector configuration?

       

      #Debezium Source: MariaDB
      debezium.source.connector.class=io.debezium.connector.mariadb.MariaDbConnector
      debezium.source.database.hostname=hostname.dk5.tv
      debezium.source.database.port=3306
      debezium.source.database.user=debezium_slip_log
      debezium.source.database.password=PASS
      debezium.source.database.server.id=1337
      debezium.source.database.include.list=advbet_web_test
      debezium.source.table.include.list=advbet_web_test.slip_log
      #Debezium Topic/Format
      debezium.source.topic.prefix=debezium
      debezium.source.topic.capture.mode=change_streams
      debezium.format.value=json
      #Offset and Schema History Storage
      debezium.source.offset.storage.file.filename=/debezium/data/offsets.dat
      debezium.source.schema.history.internal=io.debezium.storage.file.history.FileSchemaHistory
      debezium.source.schema.history.internal.file.filename=/debezium/data/schema-history.dat
      debezium.source.include.schema.changes=true
      debezium.source.snapshot.mode=initial
      #Error Handling
      debezium.source.skip.messages.without.change=true
      debezium.source.event.processing.failure.handling.mode=warn
      debezium.source.event.converting.failure.handling.mode=warn
      debezium.source.skipped.operations=u,d,t
      #Logging
      debezium.log.level=TRACE
      #Debezium Sink: NATS JetStream
      debezium.sink.type=nats-jetstream
      debezium.sink.nats-jetstream.url=tls://hostname01.dk5.tv:4222,tls://hostname02.dk5.tv:4222,tls://hostname03.dk5.tv:4222
      #TLS Authentication
      debezium.sink.nats-jetstream.auth.tls.keystore=/debezium/config/keystore.p12
      debezium.sink.nats-jetstream.auth.tls.keystore.password=PASS
      debezium.sink.nats-jetstream.auth.tls.password=PASS
      #JetStream Stream Settings
      debezium.sink.nats-jetstream.create-stream=false
      debezium.sink.nats-jetstream.stream=db_changes_stream
      debezium.sink.nats-jetstream.subjects=debezium,debezium.>
      debezium.sink.nats-jetstream.auth.user=debezium
      debezium.sink.nats-jetstream.auth.password=PASS 
      

      What is the captured database version and mode of deployment?

      on hetzner cloud hardware using  kvm
      MariaDB-server.x86_64               10.11.14-1.el9

      What behavior do you expect?

      <Your answer>

      What behavior do you see?

      Using debezium source connector to mariadb for shiping binlogs to nats-jetstream
      everything goes fine until I get 

       

      {"timestamp":"2025-09-12T13:15:23.511281715Z","sequence":16415,"loggerClassName":"org.slf4j.impl.Slf4jLogger","loggerName":"io.debezium.connector.binlog.BinlogStreamingChangeEventSource","level":"WARN","message":"A deserialization failure event arrived","threadName":"blc-adv-prod-online-test-srv01.duok5.tv:3306","threadId":46,"mdc":{"dbz.connectorLogicalName":"debezium","dbz.taskId":"0","dbz.connectorType":"MariaDB","dbz.connectorContext":"binlog"},"ndc":"","hostName":"64a025d9b9c0","processName":"/usr/lib/jvm/java-21-openjdk-21.0.8.0.9-1.el8.x86_64/bin/java","processId":1,"exception":{"refId":1,"exceptionType":"com.github.shyiko.mysql.binlog.event.deserialization.EventDataDeserializationException","message":"Failed to deserialize data of EventHeaderV4{timestamp=1757682923000, eventType=WRITE_ROWS, serverId=2420921427, headerLength=19, dataLength=2005, nextPosition=1147945, flags=0}","frames":[{"class<span class="code-quote">":"com.github.shyiko.mysql.binlog.event.deserialization.EventDeserializer","method":"deserializeEventData","line":343},{"class<span class="code-quote">":"com.github.shyiko.mysql.binlog.event.deserialization.EventDeserializer","method":"nextEvent","line":246},{"class<span class="code-quote">":"io.debezium.connector.binlog.BinlogStreamingChangeEventSource$1","method":"nextEvent","line":425},{"class<span class="code-quote">":"com.github.shyiko.mysql.binlog.BinaryLogClient","method":"listenForEventPackets","line":1082},{"class<span class="code-quote">":"com.github.shyiko.mysql.binlog.BinaryLogClient","method":"connect","line":657},{"class<span class="code-quote">":"com.github.shyiko.mysql.binlog.BinaryLogClient$7","method":"run","line":959},{"class<span class="code-quote">":"java.lang.Thread","method":"run","line":1583}],"causedBy":{"exception":{"refId":2,"exceptionType":"com.github.shyiko.mysql.binlog.event.deserialization.MissingTableMapEventException","message":"No TableMapEventData has been found for table id:6276. Usually that means that you have started reading binary log 'within the logical event group' (e.g. from WRITE_ROWS and not proceeding TABLE_MAP","frames":[{"class<span class="code-quote">":"com.github.shyiko.mysql.binlog.event.deserialization.AbstractRowsEventDataDeserializer","method":"deserializeRow","line":109},{"class<span class="code-quote">":"com.github.shyiko.mysql.binlog.event.deserialization.WriteRowsEventDataDeserializer","method":"deserializeRows","line":64},{"class<span class="code-quote">":"com.github.shyiko.mysql.binlog.event.deserialization.WriteRowsEventDataDeserializer","method":"deserialize","line":56},{"class<span class="code-quote">":"com.github.shyiko.mysql.binlog.event.deserialization.WriteRowsEventDataDeserializer","method":"deserialize","line":32},{"class<span class="code-quote">":"com.github.shyiko.mysql.binlog.event.deserialization.EventDeserializer","method":"deserializeEventData","line":337},{"class<span class="code-quote">":"com.github.shyiko.mysql.binlog.event.deserialization.EventDeserializer","method":"nextEvent","line":246},{"class<span class="code-quote">":"io.debezium.connector.binlog.BinlogStreamingChangeEventSource$1","method":"nextEvent","line":425},{"class<span class="code-quote">":"com.github.shyiko.mysql.binlog.BinaryLogClient","method":"listenForEventPackets","line":1082},{"class<span class="code-quote">":"com.github.shyiko.mysql.binlog.BinaryLogClient","method":"connect","line":657},{"class<span class="code-quote">":"com.github.shyiko.mysql.binlog.BinaryLogClient$7","method":"run","line":959},{"class<span class="code-quote">":"java.lang.Thread","method":"run","line":1583}]}}}}
      {"timestamp":"2025-09-12T13:15:23.511365563Z","sequence":16416,"loggerClassName":"org.slf4j.impl.Slf4jLogger","loggerName":"io.debezium.connector.binlog.BinlogStreamingChangeEventSource","level":"WARN","message":"Error during binlog processing. Last offset stored = {ts_sec=1757682730, file=mariadb-bin.219995, pos=1114640, gtids=0-2420921427-2869908247, server_id=2420921427, event=4}, binlog reader near position = mariadb-bin.219995/1114684","threadName":"blc-adv-prod-online-test-srv01.duok5.tv:3306","threadId":46,"mdc":{"dbz.connectorLogicalName":"debezium","dbz.taskId":"0","dbz.connectorType":"MariaDB","dbz.connectorContext":"binlog"},"ndc":"","hostName":"64a025d9b9c0","processName":"/usr/lib/jvm/java-21-openjdk-21.0.8.0.9-1.el8.x86_64/bin/java","processId":1}
       

      Additional information related to database

      binlog compression is disabled as well:

      log_bin_compress | OFF
      binlog_format | ROW
      

      table schema:
      MariaDB ******** SLAVE ******** READ ONLY ******** [web_test]> describe slip_log;
      --------{}{}{}{}{}-----------{}{}{}{}{}{}{}{}{}{}{}{}{}{}{}{}{}{}-------------{}{}{}{}{}{}{}{}-----------{+}

      Field       Type                 Null Key Default               Extra          

      --------{}{}{}{}{}-----------{}{}{}{}{}{}{}{}{}{}{}{}{}{}{}{}{}{}-------------{}{}{}{}{}{}{}{}-----------{+}

      slip_log_id bigint(20) unsigned NO   PRI NULL                 auto_increment
      ts           timestamp(6)         NO   MUL current_timestamp(6)                
      slip_uuid   binary(16)           NO   MUL NULL                                
      slip_id     bigint(20) unsigned NO       NULL                                
      website_id   bigint(20) unsigned YES   MUL NULL                                
      marshaled   mediumblob           YES       NULL                                

      --------{}{}{}{}{}-----------{}{}{}{}{}{}{}{}{}{}{}{}{}{}{}{}{}{}-------------{}{}{}{}{}{}{}{}-----------{+}

              Unassigned Unassigned
              eliteaz Darius Mukas (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Created:
                Updated:
                Resolved: