Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-217

Connector fails and stops when coming across corrupt event

    XMLWordPrintable

    Details

      Description

      After doing a snapshot and running for a while, an exception occurs and the connector stops. Restarting Kafka Connect results in the same message

      [2017-04-07 23:08:33,579] ERROR Task pushdb-connector-0 threw an uncaught and unrecoverable exception (org.apache.kafka.connect.runtime.WorkerTask:141)
      org.apache.kafka.connect.errors.ConnectException: Failed to deserialize data of EventHeaderV4{timestamp=1491606429000, eventType=WRITE_ROWS, serverId=1632581618, headerLength=19, dataLength=49, nextPosition=36405, flags=0}
              at io.debezium.connector.mysql.AbstractReader.wrap(AbstractReader.java:164)
              at io.debezium.connector.mysql.AbstractReader.failed(AbstractReader.java:131)
              at io.debezium.connector.mysql.BinlogReader$ReaderThreadLifecycleListener.onCommunicationFailure(BinlogReader.java:702)
              at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:768)
              at com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:472)
              at com.github.shyiko.mysql.binlog.BinaryLogClient$5.run(BinaryLogClient.java:657)
              at java.lang.Thread.run(Thread.java:745)
      Caused by: com.github.shyiko.mysql.binlog.event.deserialization.EventDataDeserializationException: Failed to deserialize data of EventHeaderV4{timestamp=1491606429000, eventType=WRITE_ROWS, serverId=1632581618, headerLength=19, dataLength=49, nextPosition=36405, flags=0}
              at com.github.shyiko.mysql.binlog.event.deserialization.EventDeserializer.deserializeEventData(EventDeserializer.java:212)
              at com.github.shyiko.mysql.binlog.event.deserialization.EventDeserializer.nextEvent(EventDeserializer.java:180)
              at io.debezium.connector.mysql.BinlogReader$1.nextEvent(BinlogReader.java:116)
              at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:741)
              ... 3 more
      Caused by: java.io.EOFException
              at com.github.shyiko.mysql.binlog.io.ByteArrayInputStream.read(ByteArrayInputStream.java:190)
              at com.github.shyiko.mysql.binlog.io.ByteArrayInputStream.readLong(ByteArrayInputStream.java:57)
              at com.github.shyiko.mysql.binlog.event.deserialization.AbstractRowsEventDataDeserializer.deserializeLongLong(AbstractRowsEventDataDeserializer.java:216)
              at com.github.shyiko.mysql.binlog.event.deserialization.AbstractRowsEventDataDeserializer.deserializeCell(AbstractRowsEventDataDeserializer.java:152)
              at com.github.shyiko.mysql.binlog.event.deserialization.AbstractRowsEventDataDeserializer.deserializeRow(AbstractRowsEventDataDeserializer.java:132)
              at com.github.shyiko.mysql.binlog.event.deserialization.WriteRowsEventDataDeserializer.deserializeRows(WriteRowsEventDataDeserializer.java:64)
              at com.github.shyiko.mysql.binlog.event.deserialization.WriteRowsEventDataDeserializer.deserialize(WriteRowsEventDataDeserializer.java:56)
              at com.github.shyiko.mysql.binlog.event.deserialization.WriteRowsEventDataDeserializer.deserialize(WriteRowsEventDataDeserializer.java:32)
              at com.github.shyiko.mysql.binlog.event.deserialization.EventDeserializer.deserializeEventData(EventDeserializer.java:206)
              ... 6 more
      

      Connector setup:

      {"config":{"connector.class":"io.debezium.connector.mysql.MySqlConnector","tasks.max":"1","database.hostname":"pushv2-mysql.xxx.xxx.rds.amazonaws.com","database.port":"3306","database.user":"debezium","database.password":"dbz","database.server.id":"2000","database.server.name":"pushdb","database.whitelist":"pushv2","database.history.kafka.bootstrap.servers":"localhost:9092","database.history.kafka.topic":"pushdb"},"name":"pushdb-connector"}
      

        Attachments

          Activity

            People

            Assignee:
            gunnar.morling Gunnar Morling
            Reporter:
            captain.magnus Magnus Boman (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved: