Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-4979

NPE for non-table related DDLs

    XMLWordPrintable

Details

    Description

      Initial setup : debezium 1.9.0.Alpha2

      Multiple connectors deployed (both MySQL and Postgres) and working (initial snapshot OK, streaming OK - no incremental snapshot using signal API).

      Upgrade to debezium 1.9.0.Final + restart of Kafka Connect service.

      PG connectors are OK (streaming resumed).

      MySQL connectors fail each time a DDL is encountered (within include_list) :

      io.debezium.DebeziumException: Error processing binlog event
          at io.debezium.connector.mysql.MySqlStreamingChangeEventSource.handleEvent(MySqlStreamingChangeEventSource.java:369)
          at io.debezium.connector.mysql.MySqlStreamingChangeEventSource.lambda$execute$25(MySqlStreamingChangeEventSource.java:869)
          at com.github.shyiko.mysql.binlog.BinaryLogClient.notifyEventListeners(BinaryLogClient.java:1125)
          at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:973)
          at com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:599)
          at com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:857)
          at java.base/java.lang.Thread.run(Thread.java:829)
      Caused by: java.lang.NullPointerException
          at io.debezium.pipeline.source.snapshot.incremental.AbstractIncrementalSnapshotChangeEventSource.processSchemaChange(AbstractIncrementalSnapshotChangeEventSource.java:105)
          at io.debezium.pipeline.EventDispatcher.dispatchSchemaChangeEvent(EventDispatcher.java:312)
          at io.debezium.connector.mysql.MySqlStreamingChangeEventSource.handleQueryEvent(MySqlStreamingChangeEventSource.java:575)
          at io.debezium.connector.mysql.MySqlStreamingChangeEventSource.lambda$execute$14(MySqlStreamingChangeEventSource.java:841)
          at io.debezium.connector.mysql.MySqlStreamingChangeEventSource.handleEvent(MySqlStreamingChangeEventSource.java:349)
       

      Switched log level to DEBUG for  io.debezium.connector.mysql.MySqlStreamingChangeEventSource :

       

      DEBUG Received query command: Event{header=EventHeaderV4{timestamp=1649345227000, eventType=QUERY, serverId=11111111, headerLength=19, dataLength=213, nextPosition=9564126, flags=0}, data=QueryEventData{threadId=16, executionTime=0, errorCode=0, database='REDACTED', sql='create table if not exists `sql_schema_migrations` (`id` varchar(255) not null primary key, `applied_at` datetime)  engine=InnoDB charset=UTF8'}} (io.debezium.connector.mysql.MySqlStreamingChangeEventSource) 
      DEBUG Received query command: Event{header=EventHeaderV4{timestamp=1648543157000, eventType=QUERY, serverId=1111, headerLength=19, dataLength=112, nextPosition=31678, flags=0}, data=QueryEventData{threadId=83, executionTime=134, errorCode=0, database='', sql='DROP USER 'mysql_innodb_cluster_2'@'%''}} 

       

      I don't understand why https://github.com/debezium/debezium/blob/v1.9.0.Final/debezium-core/src/main/java/io/debezium/pipeline/EventDispatcher.java#L311

      incrementalSnapshotChangeEventSource should be null.

      Rollbacked to 1.9.0.Alpha2 and it works again without any issue.

      Attachments

        Issue Links

          Activity

            People

              jpechane Jiri Pechanec
              fuyar UYAR Farid
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: