Details
-
Bug
-
Resolution: Done
-
Critical
-
None
-
None
Description
When an incorrect event is recorded in database history topic the connector will fail to start like in this case:
2017-08-24 20:48:39,129] ERROR Failed to start the connector (see other exception), but got this error while cleaning up (io.debezium.connector.mysql.MySqlConnectorTask:183) java.lang.NullPointerException at io.debezium.connector.mysql.MySqlConnectorTask.stop(MySqlConnectorTask.java:220) at io.debezium.connector.mysql.MySqlConnectorTask.start(MySqlConnectorTask.java:180) at org.apache.kafka.connect.runtime.WorkerSourceTask.execute(WorkerSourceTask.java:137) at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:140) at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:175) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) [2017-08-24 20:48:39,140] ERROR Task mysql-connector-0 threw an uncaught and unrecoverable exception (org.apache.kafka.connect.runtime.WorkerTask:142) org.apache.kafka.connect.errors.ConnectException: java.lang.NullPointerException at io.debezium.connector.mysql.MySqlConnectorTask.start(MySqlConnectorTask.java:192) at org.apache.kafka.connect.runtime.WorkerSourceTask.execute(WorkerSourceTask.java:137) at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:140) at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:175) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.NullPointerException at io.debezium.relational.history.HistoryRecordComparator.isSameSource(HistoryRecordComparator.java:60) at io.debezium.relational.history.HistoryRecordComparator.isAtOrBefore(HistoryRecordComparator.java:52) at io.debezium.relational.history.AbstractDatabaseHistory.lambda$recover$0(AbstractDatabaseHistory.java:53) at io.debezium.relational.history.AbstractDatabaseHistory$$Lambda$104/1659651213.accept(Unknown Source) at io.debezium.relational.history.KafkaDatabaseHistory.recoverRecords(KafkaDatabaseHistory.java:202) at io.debezium.relational.history.AbstractDatabaseHistory.recover(AbstractDatabaseHistory.java:52) at io.debezium.connector.mysql.MySqlSchema.loadHistory(MySqlSchema.java:312) at io.debezium.connector.mysql.MySqlTaskContext.loadHistory(MySqlTaskContext.java:116) at io.debezium.connector.mysql.MySqlConnectorTask.start(MySqlConnectorTask.java:80) ... 8 more [2017-08-24 20:48:39,142] ERROR Task is being killed and will not recover until manually restarted (org.apache.kafka.connect.runtime.WorkerTask:143)
We should define invariants for message coming from the topic and validate it before processing.
See https://gitter.im/debezium/user?at=59a02a5ea7b406262dc739b5 for context.