Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-2303

Adding new table to cdc causes the sqlconnector to fail

XMLWordPrintable

    • Hide

      1. Using the attached zip, extract and follow the README.md steps.

      Show
      1. Using the attached zip, extract and follow the README.md steps.

      I have tables added to the connector using the table.whitelist config e.g

      {
          "connector.class" : "io.debezium.connector.sqlserver.SqlServerConnector",
          "tasks.max" : "1",
          "database.server.name" : "server1",
          "database.hostname" : "sqlserver",
          "database.port" : "1433",
          "database.user" : "sa",
          "database.password" : "****",
          "database.dbname" : "testDB",
          "database.history.kafka.bootstrap.servers" : "kafka:9092",
          "database.history.kafka.topic": "schema-changes.inventory",
          "table.whitelist": "dbo.orders,dbo.customers"
          
      }
      

      I now want to add a new table, so i update the config and update the connector e.g

      {
          "connector.class" : "io.debezium.connector.sqlserver.SqlServerConnector",
          "tasks.max" : "1",
          "database.server.name" : "server1",
          "database.hostname" : "sqlserver",
          "database.port" : "1433",
          "database.user" : "sa",
          "database.password" : "***",
          "database.dbname" : "testDB",
          "database.history.kafka.bootstrap.servers" : "kafka:9092",
          "database.history.kafka.topic": "schema-changes.inventory",
          "table.whitelist": "dbo.orders,dbo.customers,dbo.products"
          

      Now add the table to be tracked by CDC using: 

      USE testDB;
      EXEC sys.sp_cdc_enable_table @source_schema = 'dbo', @source_name = 'products', @role_name = NULL, @supports_net_changes = 0;

      The following error appears in the connect logs:

      [2020-06-30 07:14:36,508] ERROR [source-debezium-connector-sqlserver|task-0] Producer failure (io.debezium.pipeline.ErrorHandler:31)
      java.lang.NullPointerException
      at io.debezium.connector.AbstractSourceInfoStructMaker.commonStruct(AbstractSourceInfoStructMaker.java:53)
      at io.debezium.connector.sqlserver.SqlServerSourceInfoStructMaker.struct(SqlServerSourceInfoStructMaker.java:37)
      at io.debezium.connector.sqlserver.SqlServerSourceInfoStructMaker.struct(SqlServerSourceInfoStructMaker.java:14)
      at io.debezium.connector.AbstractSourceInfo.struct(AbstractSourceInfo.java:76)
      at io.debezium.connector.sqlserver.SqlServerOffsetContext.getSourceInfo(SqlServerOffsetContext.java:91)
      at io.debezium.connector.sqlserver.SqlServerSchemaChangeEventEmitter.emitSchemaChangeEvent(SqlServerSchemaChangeEventEmitter.java:42)
      at io.debezium.pipeline.EventDispatcher.dispatchSchemaChangeEvent(EventDispatcher.java:263)
      at io.debezium.connector.sqlserver.SqlServerStreamingChangeEventSource.getCdcTablesToQuery(SqlServerStreamingChangeEventSource.java:334)
      at io.debezium.connector.sqlserver.SqlServerStreamingChangeEventSource.execute(SqlServerStreamingChangeEventSource.java:108)
      at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:108)
      at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
      at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
      at java.lang.Thread.run(Thread.java:748)

            jpechane Jiri Pechanec
            rob3000 Robert Shepherd (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: