Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-7928

EventDataDeserializationException: eventType=TABLE_MAP

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Critical Critical
    • under-triaging
    • 2.4.2.Final, 2.5.0.CR1
    • mysql-connector
    • None
    • True
    • None
    • False
    • Important

      Bug report

      For bug reports, provide this information, please:

      What Debezium connector do you use and what version?

      Confluent Kafka Connect Version 2.4.2

      What is the connector configuration?

       

      {
  "name": "mysql-connector",
  "config":
    {
      "connector.class": "io.debezium.connector.mysql.MySqlConnector",
      "connector.adapter": "mariadb",
      "database.protocol": "jdbc:mariadb",
      "database.jdbc.driver": "org.mariadb.jdbc.Driver",
      "database.hostname": "",
      "database.port": "3306",
      "database.user": "",
      "database.password": "",
      "database.server.id": "1234",
      "database.ssl.mode": "required",
      "database.include.list": "budget",
      "table.include.list": "budget.kafka_test",
      "schema.history.internal.store.only.captured.tables.ddl": "true",
      "schema.history.internal.store.only.captured.databases.ddl": "true",
      "topic.prefix": "mysql-budget",
      "snapshot.mode": "when_needed",
      "errors.max.retries": "3",
      "incremental.snapshot.allow.schema.changes": "true",
      "snapshot.include.collection.list": "budget.kafka_test",
      "event.processing.failure.handling.mode": "warn",
      "event.deserialization.failure.handling.mode": "warn",
      "schema.history.internal.kafka.bootstrap.servers": "kafka:29092",
      "schema.history.internal.kafka.topic": "schema-changes.budget"
    }
}

       

      What is the captured database version and mode of deployment?

      Cloud Hosted via AWS - MySQL Aurora Version: 8.0.32

      What behaviour do you expect?

      Capture changes without the error.

      What behaviour do you see?

       

      Do you see the same behaviour using the latest released Debezium version?

      Yes

      Do you have the connector logs, ideally from start till finish?

      2024-06-06 21:43:24    
{before=[3, 1562757231451, 0, 2024-05-21T07:28:33Z], after=[3, 1562757231451, 0, 2024-06-06T13:15:42Z]}
2024-06-06 21:43:24 ]}} (io.debezium.connector.mysql.MySqlStreamingChangeEventSource)
2024-06-06 21:43:24 [2024-06-06 13:43:24,877] TRACE Current milliseconds behind source: 1662877 ms (io.debezium.connector.mysql.MySqlStreamingChangeEventSource)
2024-06-06 21:43:24 [2024-06-06 13:43:24,877] TRACE Received event: Event{header=EventHeaderV4
{timestamp=1717679742000, eventType=XID, serverId=1777642305, headerLength=19, dataLength=12, nextPosition=21216228, flags=0}
, data=XidEventData{xid=5351972625306}} (io.debezium.connector.mysql.MySqlStreamingChangeEventSource)
2024-06-06 21:43:24 [2024-06-06 13:43:24,877] TRACE Ignoring event due to missing handler: Event{header=EventHeaderV4
{timestamp=1717679741000, eventType=ANONYMOUS_GTID, serverId=1777642305, headerLength=19, dataLength=66, nextPosition=21216313, flags=0}, data=null} (io.debezium.connector.mysql.MySqlStreamingChangeEventSource)
2024-06-06 21:43:24 [2024-06-06 13:43:24,877] TRACE Current milliseconds behind source: 1663877 ms (io.debezium.connector.mysql.MySqlStreamingChangeEventSource)
2024-06-06 21:43:24 [2024-06-06 13:43:24,877] TRACE Received event: Event{header=EventHeaderV4{timestamp=1717679741000, eventType=ANONYMOUS_GTID, serverId=1777642305, headerLength=19, dataLength=66, nextPosition=21216313, flags=0}
, data=null} (io.debezium.connector.mysql.MySqlStreamingChangeEventSource)
2024-06-06 21:43:24 [2024-06-06 13:43:24,877] DEBUG Received query command: Event{header=EventHeaderV4
{timestamp=1717679741000, eventType=QUERY, serverId=1777642305, headerLength=19, dataLength=74, nextPosition=21216406, flags=8}, data=QueryEventData{threadId=4859333, executionTime=0, errorCode=0, database='awsdms_control', sql='BEGIN'}} (io.debezium.connector.mysql.MySqlStreamingChangeEventSource)
2024-06-06 21:43:24 [2024-06-06 13:43:24,877] TRACE Current milliseconds behind source: 1663877 ms (io.debezium.connector.mysql.MySqlStreamingChangeEventSource)
2024-06-06 21:43:24 [2024-06-06 13:43:24,877] TRACE Received event: Event{header=EventHeaderV4{timestamp=1717679741000, eventType=QUERY, serverId=1777642305, headerLength=19, dataLength=74, nextPosition=21216406, flags=8}
, data=QueryEventData{threadId=4859333, executionTime=0, errorCode=0, database='awsdms_control', sql='BEGIN'}} (io.debezium.connector.mysql.MySqlStreamingChangeEventSource)
2024-06-06 21:43:24 [2024-06-06 13:43:24,877] DEBUG A communication failure event arrived (io.debezium.connector.mysql.MySqlStreamingChangeEventSource)
2024-06-06 21:43:24 com.github.shyiko.mysql.binlog.event.deserialization.EventDataDeserializationException: Failed to deserialize data of EventHeaderV4{timestamp=1717679741000, eventType=TABLE_MAP, serverId=1777642305, headerLength=19, dataLength=272, nextPosition=21216697, flags=0}
2024-06-06 21:43:24     at com.github.shyiko.mysql.binlog.event.deserialization.EventDeserializer.deserializeEventData(EventDeserializer.java:343)
2024-06-06 21:43:24     at com.github.shyiko.mysql.binlog.event.deserialization.EventDeserializer.deserializeTableMapEventData(EventDeserializer.java:315)
2024-06-06 21:43:24     at com.github.shyiko.mysql.binlog.event.deserialization.EventDeserializer.nextEvent(EventDeserializer.java:239)
2024-06-06 21:43:24     at io.debezium.connector.mysql.MySqlStreamingChangeEventSource$1.nextEvent(MySqlStreamingChangeEventSource.java:233)
2024-06-06 21:43:24     at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:1051)
2024-06-06 21:43:24     at com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:631)
2024-06-06 21:43:24     at com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:932)
2024-06-06 21:43:24     at java.base/java.lang.Thread.run(Thread.java:829)
2024-06-06 21:43:24 Caused by: java.io.EOFException: Failed to read remaining 2 of 2 bytes from position 157. Block length: 0. Initial block length: 2.
2024-06-06 21:43:24     at com.github.shyiko.mysql.binlog.io.ByteArrayInputStream.fill(ByteArrayInputStream.java:115)
2024-06-06 21:43:24     at com.github.shyiko.mysql.binlog.io.ByteArrayInputStream.read(ByteArrayInputStream.java:105)
2024-06-06 21:43:24     at com.github.shyiko.mysql.binlog.event.deserialization.TableMapEventMetadataDeserializer.readBooleanList(TableMapEventMetadataDeserializer.java:114)
2024-06-06 21:43:24     at com.github.shyiko.mysql.binlog.event.deserialization.TableMapEventMetadataDeserializer.deserialize(TableMapEventMetadataDeserializer.java:100)
2024-06-06 21:43:24     at com.github.shyiko.mysql.binlog.event.deserialization.TableMapEventDataDeserializer.deserialize(TableMapEventDataDeserializer.java:47)
2024-06-06 21:43:24     at com.github.shyiko.mysql.binlog.event.deserialization.TableMapEventDataDeserializer.deserialize(TableMapEventDataDeserializer.java:27)
2024-06-06 21:43:24     at com.github.shyiko.mysql.binlog.event.deserialization.EventDeserializer.deserializeEventData(EventDeserializer.java:337)
2024-06-06 21:43:24     ... 7 more
2024-06-06 21:43:24 [2024-06-06 13:43:24,877] ERROR Error during binlog processing. Last offset stored = {transaction_id=null, file=mysql-bin-changelog.171432, pos=21216313, server_id=1777642305, event=1}, binlog reader near position = mysql-bin-changelog.171432/21216406 (io.debezium.connector.mysql.MySqlStreamingChangeEventSource)
2024-06-06 21:43:24 [2024-06-06 13:43:24,879] ERROR Producer failure (io.debezium.pipeline.ErrorHandler)
2024-06-06 21:43:24 io.debezium.DebeziumException: Failed to deserialize data of EventHeaderV4{timestamp=1717679741000, eventType=TABLE_MAP, serverId=1777642305, headerLength=19, dataLength=272, nextPosition=21216697, flags=0}
2024-06-06 21:43:24     at io.debezium.connector.mysql.MySqlStreamingChangeEventSource.wrap(MySqlStreamingChangeEventSource.java:1254)
2024-06-06 21:43:24     at io.debezium.connector.mysql.MySqlStreamingChangeEventSource$ReaderThreadLifecycleListener.onCommunicationFailure(MySqlStreamingChangeEventSource.java:1299)
2024-06-06 21:43:24     at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:1079)
2024-06-06 21:43:24     at com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:631)
2024-06-06 21:43:24     at com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:932)
2024-06-06 21:43:24     at java.base/java.lang.Thread.run(Thread.java:829)
2024-06-06 21:43:24 Caused by: com.github.shyiko.mysql.binlog.event.deserialization.EventDataDeserializationException: Failed to deserialize data of EventHeaderV4{timestamp=1717679741000, eventType=TABLE_MAP, serverId=1777642305, headerLength=19, dataLength=272, nextPosition=21216697, flags=0}
2024-06-06 21:43:24     at com.github.shyiko.mysql.binlog.event.deserialization.EventDeserializer.deserializeEventData(EventDeserializer.java:343)
2024-06-06 21:43:24     at com.github.shyiko.mysql.binlog.event.deserialization.EventDeserializer.deserializeTableMapEventData(EventDeserializer.java:315)
2024-06-06 21:43:24     at com.github.shyiko.mysql.binlog.event.deserialization.EventDeserializer.nextEvent(EventDeserializer.java:239)
2024-06-06 21:43:24     at io.debezium.connector.mysql.MySqlStreamingChangeEventSource$1.nextEvent(MySqlStreamingChangeEventSource.java:233)
2024-06-06 21:43:24     at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:1051)
2024-06-06 21:43:24     ... 3 more
2024-06-06 21:43:24 Caused by: java.io.EOFException: Failed to read remaining 2 of 2 bytes from position 157. Block length: 0. Initial block length: 2.
2024-06-06 21:43:24     at com.github.shyiko.mysql.binlog.io.ByteArrayInputStream.fill(ByteArrayInputStream.java:115)
2024-06-06 21:43:24     at com.github.shyiko.mysql.binlog.io.ByteArrayInputStream.read(ByteArrayInputStream.java:105)
2024-06-06 21:43:24     at com.github.shyiko.mysql.binlog.event.deserialization.TableMapEventMetadataDeserializer.readBooleanList(TableMapEventMetadataDeserializer.java:114)
2024-06-06 21:43:24     at com.github.shyiko.mysql.binlog.event.deserialization.TableMapEventMetadataDeserializer.deserialize(TableMapEventMetadataDeserializer.java:100)
2024-06-06 21:43:24     at com.github.shyiko.mysql.binlog.event.deserialization.TableMapEventDataDeserializer.deserialize(TableMapEventDataDeserializer.java:47)
2024-06-06 21:43:24     at com.github.shyiko.mysql.binlog.event.deserialization.TableMapEventDataDeserializer.deserialize(TableMapEventDataDeserializer.java:27)
2024-06-06 21:43:24     at com.github.shyiko.mysql.binlog.event.deserialization.EventDeserializer.deserializeEventData(EventDeserializer.java:337)
2024-06-06 21:43:24     ... 7 more
2024-06-06 21:43:24 [2024-06-06 13:43:24,879] WARN Retry 33 of unlimited retries will be attempted (io.debezium.pipeline.ErrorHandler)
2024-06-06 21:43:24 [2024-06-06 13:43:24,879] INFO Stopped reading binlog after 0 events, last recorded offset: {transaction_id=null, file=mysql-bin-changelog.171432, pos=21216313, server_id=1777642305, event=1} (io.debezium.connector.mysql.MySqlStreamingChangeEventSource)
2024-06-06 21:43:24 Jun 06, 2024 1:43:24 PM com.github.shyiko.mysql.binlog.BinaryLogClient$5 run
2024-06-06 21:43:24 INFO: threadExecutor is shut down, terminating keepalive thread
2024-06-06 21:43:25 [2024-06-06 13:43:25,221] WARN Going to restart connector after 10 sec. after a retriable exception (io.debezium.connector.common.BaseSourceTask)
2024-06-06 21:43:25 [2024-06-06 13:43:25,282] INFO Finished streaming (io.debezium.pipeline.ChangeEventSourceCoordinator)
2024-06-06 21:43:25 [2024-06-06 13:43:25,282] INFO SignalProcessor stopped (io.debezium.pipeline.signal.SignalProcessor)
2024-06-06 21:43:25 [2024-06-06 13:43:25,532] INFO Connection gracefully closed (io.debezium.jdbc.JdbcConnection)
2024-06-06 21:43:25 [2024-06-06 13:43:25,533] INFO [Producer clientId=mysql-budget-schemahistory] Closing the Kafka producer with timeoutMillis = 30000 ms. (org.apache.kafka.clients.producer.KafkaProducer)
2024-06-06 21:43:25 [2024-06-06 13:43:25,533] INFO Metrics scheduler closed (org.apache.kafka.common.metrics.Metrics)
2024-06-06 21:43:25 [2024-06-06 13:43:25,533] INFO Closing reporter org.apache.kafka.common.metrics.JmxReporter (org.apache.kafka.common.metrics.Metrics)
2024-06-06 21:43:25 [2024-06-06 13:43:25,533] INFO Metrics reporters closed (org.apache.kafka.common.metrics.Metrics)
2024-06-06 21:43:25 [2024-06-06 13:43:25,533] INFO App info kafka.producer for mysql-budget-schemahistory unregistered (org.apache.kafka.common.utils.AppInfoParser)
2024-06-06 21:43:25 [2024-06-06 13:43:25,533] WARN WorkerSourceTask{id=mysql-connector-0} failed to poll records from SourceTask. Will retry operation. (org.apache.kafka.connect.runtime.AbstractWorkerSourceTask)
2024-06-06 21:43:25 org.apache.kafka.connect.errors.RetriableException: An exception occurred in the change event producer. This connector will be restarted.
2024-06-06 21:43:25     at io.debezium.pipeline.ErrorHandler.setProducerThrowable(ErrorHandler.java:63)
2024-06-06 21:43:25     at io.debezium.connector.mysql.MySqlStreamingChangeEventSource$ReaderThreadLifecycleListener.onCommunicationFailure(MySqlStreamingChangeEventSource.java:1299)
2024-06-06 21:43:25     at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:1079)
2024-06-06 21:43:25     at com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:631)
2024-06-06 21:43:25     at com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:932)
2024-06-06 21:43:25     at java.base/java.lang.Thread.run(Thread.java:829)
2024-06-06 21:43:25 Caused by: io.debezium.DebeziumException: Failed to deserialize data of EventHeaderV4{timestamp=1717679741000, eventType=TABLE_MAP, serverId=1777642305, headerLength=19, dataLength=272, nextPosition=21216697, flags=0}
2024-06-06 21:43:25     at io.debezium.connector.mysql.MySqlStreamingChangeEventSource.wrap(MySqlStreamingChangeEventSource.java:1254)
2024-06-06 21:43:25     ... 5 more
2024-06-06 21:43:25 Caused by: com.github.shyiko.mysql.binlog.event.deserialization.EventDataDeserializationException: Failed to deserialize data of EventHeaderV4{timestamp=1717679741000, eventType=TABLE_MAP, serverId=1777642305, headerLength=19, dataLength=272, nextPosition=21216697, flags=0}
2024-06-06 21:43:25     at com.github.shyiko.mysql.binlog.event.deserialization.EventDeserializer.deserializeEventData(EventDeserializer.java:343)
2024-06-06 21:43:25     at com.github.shyiko.mysql.binlog.event.deserialization.EventDeserializer.deserializeTableMapEventData(EventDeserializer.java:315)
2024-06-06 21:43:25     at com.github.shyiko.mysql.binlog.event.deserialization.EventDeserializer.nextEvent(EventDeserializer.java:239)
2024-06-06 21:43:25     at io.debezium.connector.mysql.MySqlStreamingChangeEventSource$1.nextEvent(MySqlStreamingChangeEventSource.java:233)
2024-06-06 21:43:25     at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:1051)
2024-06-06 21:43:25     ... 3 more
2024-06-06 21:43:25 Caused by: java.io.EOFException: Failed to read remaining 2 of 2 bytes from position 157. Block length: 0. Initial block length: 2.
2024-06-06 21:43:25     at com.github.shyiko.mysql.binlog.io.ByteArrayInputStream.fill(ByteArrayInputStream.java:115)
2024-06-06 21:43:25     at com.github.shyiko.mysql.binlog.io.ByteArrayInputStream.read(ByteArrayInputStream.java:105)
2024-06-06 21:43:25     at com.github.shyiko.mysql.binlog.event.deserialization.TableMapEventMetadataDeserializer.readBooleanList(TableMapEventMetadataDeserializer.java:114)
2024-06-06 21:43:25     at com.github.shyiko.mysql.binlog.event.deserialization.TableMapEventMetadataDeserializer.deserialize(TableMapEventMetadataDeserializer.java:100)
2024-06-06 21:43:25     at com.github.shyiko.mysql.binlog.event.deserialization.TableMapEventDataDeserializer.deserialize(TableMapEventDataDeserializer.java:47)
2024-06-06 21:43:25     at com.github.shyiko.mysql.binlog.event.deserialization.TableMapEventDataDeserializer.deserialize(TableMapEventDataDeserializer.java:27)
2024-06-06 21:43:25     at com.github.shyiko.mysql.binlog.event.deserialization.EventDeserializer.deserializeEventData(EventDeserializer.java:337)
2024-06-06 21:43:25     ... 7 more
2024-06-06 21:43:25 [2024-06-06 13:43:25,534] INFO Awaiting end of restart backoff period after a retriable error (io.debezium.connector.common.BaseSourceTask)
2024-06-06 21:43:26 [2024-06-06 13:43:26,883] INFO 172.22.0.6 - - [06/Jun/2024:13:43:26 +0000] "GET /connectors HTTP/1.1" 200 19 "-" "curl/7.61.1" 1 (org.apache.kafka.connect.runtime.rest.RestServer)
2024-06-06 21:43:31 [2024-06-06 13:43:31,908] INFO 172.22.0.6 - - [06/Jun/2024:13:43:31 +0000] "GET /connectors HTTP/1.1" 200 19 "-" "curl/7.61.1" 2 (org.apache.kafka.connect.runtime.rest.RestServer)
2024-06-06 21:43:35 [2024-06-06 13:43:35,533] INFO Attempting to restart task. (io.debezium.connector.common.BaseSourceTask)
2024-06-06 21:43:35 [2024-06-06 13:43:35,533] INFO Loading the custom source info struct maker plugin: io.debezium.connector.mysql.MySqlSourceInfoStructMaker (io.debezium.config.CommonConnectorConfig)
2024-06-06 21:43:35 [2024-06-06 13:43:35,533] INFO Loading the custom topic naming strategy plugin: io.debezium.schema.DefaultTopicNamingStrategy (io.debezium.config.CommonConnectorConfig)
2024-06-06 21:43:36 [2024-06-06 13:43:36,932] INFO 172.22.0.6 - - [06/Jun/2024:13:43:36 +0000] "GET /connectors HTTP/1.1" 200 19 "-" "curl/7.61.1" 1 (org.apache.kafka.connect.runtime.rest.RestServer)
2024-06-06 21:43:37 [2024-06-06 13:43:37,792] DEBUG binlog_format=ROW (io.debezium.connector.mysql.MySqlConnection)
2024-06-06 21:43:38 [2024-06-06 13:43:38,041] DEBUG binlog_row_image=FULL (io.debezium.connector.mysql.MySqlConnection)

      How to reproduce the issue using our tutorial deployment?

      <Your answer>

            Unassigned Unassigned
            justpaulpadayao Justine Paul Padayao (Inactive)
            Votes:
            1 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated: