Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-4933

NPE during snapshotting MySQL database if custom converters present and column is null

    XMLWordPrintable

Details

    Description

      Our team faced an NPE when trying to start sourcing data from MySQL 8.0 database. After a deep debugging session, we pinpointed the problem to be introduced in this git commit

       

      Our setup contains a custom converter, but this converter was not used to convert the column that triggered the condition. The column is of type VARCHAR and the value of the row was `null`. This state resulted in `getString(columnIndex)` to return `null`, which then resulted in a NullPointerException when `getBytes()` was invoked on it.

       

      Logs:

      ```

      2022-03-31 10:09:00,786 ERROR WorkerSourceTask{id=private.wms-matflo-fcms.to.v5-0} Task threw an uncaught and unrecoverable exception. Task is being killed and will not recover until manually restarted (org.apache.kafka.connect.runtime.WorkerTask) [task-thread-private.wms-matflo-fcms.to.v5-0]
      org.apache.kafka.connect.errors.ConnectException: An exception occurred in the change event producer. This connector will be stopped.
          at io.debezium.pipeline.ErrorHandler.setProducerThrowable(ErrorHandler.java:42)
          at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:115)
          at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
          at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
          at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
          at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
          at java.base/java.lang.Thread.run(Thread.java:829)
      Caused by: io.debezium.DebeziumException: java.lang.NullPointerException
          at io.debezium.pipeline.source.AbstractSnapshotChangeEventSource.execute(AbstractSnapshotChangeEventSource.java:85)
          at io.debezium.pipeline.ChangeEventSourceCoordinator.doSnapshot(ChangeEventSourceCoordinator.java:153)
          at io.debezium.pipeline.ChangeEventSourceCoordinator.executeChangeEventSources(ChangeEventSourceCoordinator.java:135)
          at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:108)
          ... 5 more
      Caused by: java.lang.NullPointerException
          at io.debezium.connector.mysql.AbstractMysqlFieldReader.readField(AbstractMysqlFieldReader.java:65)
          at io.debezium.connector.mysql.MySqlConnection.getColumnValue(MySqlConnection.java:595)
          at io.debezium.jdbc.JdbcConnection.rowToArray(JdbcConnection.java:1503)
          at io.debezium.relational.RelationalSnapshotChangeEventSource.createDataEventsForTable(RelationalSnapshotChangeEventSource.java:360)
          at io.debezium.relational.RelationalSnapshotChangeEventSource.createDataEvents(RelationalSnapshotChangeEventSource.java:309)
          at io.debezium.relational.RelationalSnapshotChangeEventSource.doExecute(RelationalSnapshotChangeEventSource.java:129)
          at io.debezium.pipeline.source.AbstractSnapshotChangeEventSource.execute(AbstractSnapshotChangeEventSource.java:76)
          ... 8 more

      ```

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              nenadstojanovikj Nenad Stojanovikj (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: