Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-341

Sanitize readings from database history topic

    XMLWordPrintable

Details

    • Bug
    • Resolution: Done
    • Critical
    • 0.6
    • None
    • mysql-connector
    • None

    Description

      When an incorrect event is recorded in database history topic the connector will fail to start like in this case:

      2017-08-24 20:48:39,129] ERROR Failed to start the connector (see other exception), but got this error while cleaning up (io.debezium.connector.mysql.MySqlConnectorTask:183)
      java.lang.NullPointerException
      at io.debezium.connector.mysql.MySqlConnectorTask.stop(MySqlConnectorTask.java:220)
      at io.debezium.connector.mysql.MySqlConnectorTask.start(MySqlConnectorTask.java:180)
      at org.apache.kafka.connect.runtime.WorkerSourceTask.execute(WorkerSourceTask.java:137)
      at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:140)
      at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:175)
      at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
      at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      at java.lang.Thread.run(Thread.java:745)
      [2017-08-24 20:48:39,140] ERROR Task mysql-connector-0 threw an uncaught and unrecoverable exception (org.apache.kafka.connect.runtime.WorkerTask:142)
      org.apache.kafka.connect.errors.ConnectException: java.lang.NullPointerException
      at io.debezium.connector.mysql.MySqlConnectorTask.start(MySqlConnectorTask.java:192)
      at org.apache.kafka.connect.runtime.WorkerSourceTask.execute(WorkerSourceTask.java:137)
      at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:140)
      at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:175)
      at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
      at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      at java.lang.Thread.run(Thread.java:745)
      Caused by: java.lang.NullPointerException
      at io.debezium.relational.history.HistoryRecordComparator.isSameSource(HistoryRecordComparator.java:60)
      at io.debezium.relational.history.HistoryRecordComparator.isAtOrBefore(HistoryRecordComparator.java:52)
      at io.debezium.relational.history.AbstractDatabaseHistory.lambda$recover$0(AbstractDatabaseHistory.java:53)
      at io.debezium.relational.history.AbstractDatabaseHistory$$Lambda$104/1659651213.accept(Unknown Source)
      at io.debezium.relational.history.KafkaDatabaseHistory.recoverRecords(KafkaDatabaseHistory.java:202)
      at io.debezium.relational.history.AbstractDatabaseHistory.recover(AbstractDatabaseHistory.java:52)
      at io.debezium.connector.mysql.MySqlSchema.loadHistory(MySqlSchema.java:312)
      at io.debezium.connector.mysql.MySqlTaskContext.loadHistory(MySqlTaskContext.java:116)
      at io.debezium.connector.mysql.MySqlConnectorTask.start(MySqlConnectorTask.java:80)
      ... 8 more
      [2017-08-24 20:48:39,142] ERROR Task is being killed and will not recover until manually restarted (org.apache.kafka.connect.runtime.WorkerTask:143)
      

      We should define invariants for message coming from the topic and validate it before processing.

      See https://gitter.im/debezium/user?at=59a02a5ea7b406262dc739b5 for context.

      Attachments

        Activity

          People

            jpechane Jiri Pechanec
            jpechane Jiri Pechanec
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: