Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-4933

NPE during snapshotting MySQL database if custom converters present and column is null

    XMLWordPrintable

Details

    • False
    • None
    • False

    Description

      Our team faced an NPE when trying to start sourcing data from MySQL 8.0 database. After a deep debugging session, we pinpointed the problem to be introduced in this git commit

       

      Our setup contains a custom converter, but this converter was not used to convert the column that triggered the condition. The column is of type VARCHAR and the value of the row was `null`. This state resulted in `getString(columnIndex)` to return `null`, which then resulted in a NullPointerException when `getBytes()` was invoked on it.

       

      Logs:

      ```

      2022-03-31 10:09:00,786 ERROR WorkerSourceTask{id=private.wms-matflo-fcms.to.v5-0} Task threw an uncaught and unrecoverable exception. Task is being killed and will not recover until manually restarted (org.apache.kafka.connect.runtime.WorkerTask) [task-thread-private.wms-matflo-fcms.to.v5-0]
      org.apache.kafka.connect.errors.ConnectException: An exception occurred in the change event producer. This connector will be stopped.
          at io.debezium.pipeline.ErrorHandler.setProducerThrowable(ErrorHandler.java:42)
          at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:115)
          at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
          at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
          at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
          at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
          at java.base/java.lang.Thread.run(Thread.java:829)
      Caused by: io.debezium.DebeziumException: java.lang.NullPointerException
          at io.debezium.pipeline.source.AbstractSnapshotChangeEventSource.execute(AbstractSnapshotChangeEventSource.java:85)
          at io.debezium.pipeline.ChangeEventSourceCoordinator.doSnapshot(ChangeEventSourceCoordinator.java:153)
          at io.debezium.pipeline.ChangeEventSourceCoordinator.executeChangeEventSources(ChangeEventSourceCoordinator.java:135)
          at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:108)
          ... 5 more
      Caused by: java.lang.NullPointerException
          at io.debezium.connector.mysql.AbstractMysqlFieldReader.readField(AbstractMysqlFieldReader.java:65)
          at io.debezium.connector.mysql.MySqlConnection.getColumnValue(MySqlConnection.java:595)
          at io.debezium.jdbc.JdbcConnection.rowToArray(JdbcConnection.java:1503)
          at io.debezium.relational.RelationalSnapshotChangeEventSource.createDataEventsForTable(RelationalSnapshotChangeEventSource.java:360)
          at io.debezium.relational.RelationalSnapshotChangeEventSource.createDataEvents(RelationalSnapshotChangeEventSource.java:309)
          at io.debezium.relational.RelationalSnapshotChangeEventSource.doExecute(RelationalSnapshotChangeEventSource.java:129)
          at io.debezium.pipeline.source.AbstractSnapshotChangeEventSource.execute(AbstractSnapshotChangeEventSource.java:76)
          ... 8 more

      ```

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              nenadstojanovikj Nenad Stojanovikj (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: