Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-3952

"snapshot.include.collection.list" doesn't work with the new MySQL connector implementation

    XMLWordPrintable

Details

    Description

      The following error occurred when editing tables that are in table.include.list but not in snapshot.include.collection.list

      {"name":"mysql-source","connector":{"state":"RUNNING","worker_id":"kafka-connect:8083"},"tasks":[{"id":0,"state":"FAILED","worker_id":"kafka-connect:8083","trace":"org.apache.kafka.connect.errors.ConnectException: An exception occurred in the change event producer. This connector will be stopped.\n\tat io.debezium.pipeline.ErrorHandler.setProducerThrowable(ErrorHandler.java:42)\n\tat io.debezium.connector.mysql.MySqlStreamingChangeEventSource.handleEvent(MySqlStreamingChangeEventSource.java:366)\n\tat io.debezium.connector.mysql.MySqlStreamingChangeEventSource.lambda$execute$25(MySqlStreamingChangeEventSource.java:842)\n\tat com.github.shyiko.mysql.binlog.BinaryLogClient.notifyEventListeners(BinaryLogClient.java:1118)\n\tat com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:966)\n\tat com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:606)\n\tat com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:850)\n\tat java.base/java.lang.Thread.run(Thread.java:834)\nCaused by: io.debezium.DebeziumException: Error processing binlog event\n\t... 7 more\nCaused by: io.debezium.DebeziumException: Encountered change event for table db.table2 whose schema isn't known to this connector\n\tat io.debezium.connector.mysql.MySqlStreamingChangeEventSource.informAboutUnknownTableIfRequired(MySqlStreamingChangeEventSource.java:648)\n\tat io.debezium.connector.mysql.MySqlStreamingChangeEventSource.handleUpdateTableMetadata(MySqlStreamingChangeEventSource.java:628)\n\tat io.debezium.connector.mysql.MySqlStreamingChangeEventSource.lambda$execute$13(MySqlStreamingChangeEventSource.java:813)\n\tat io.debezium.connector.mysql.MySqlStreamingChangeEventSource.handleEvent(MySqlStreamingChangeEventSource.java:349)\n\t... 6 more\n"}],"type":"source"}
      

       

      Config:

      connector.class: MySqlConnector
      database.history.kafka.bootstrap.servers: kafka-service:29092
      database.history.kafka.topic: _kafka-connect-history
      database.history.skip.unparseable.ddl: true
      database.hostname: mysql
      database.include.list: db
      database.password: pass
      database.server.name: local_dev
      database.user: user
      name: mysql-source
      snapshot.include.collection.list: db.table1
      snapshot.locking.mode: none
      table.include.list: db.table1, db.table2
      tombstones.on.delete: false
      

      Same config and setup myql 8.0.13 works with debezium v1.4.0

       

      Workaround: set internal.implementation=legacy

      Attachments

        Activity

          People

            jpechane Jiri Pechanec
            mzhang8861 Meng Zhang (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: