Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-2303

Adding new table to cdc causes the sqlconnector to fail

    XMLWordPrintable

Details

    • Hide

      1. Using the attached zip, extract and follow the README.md steps.

      Show
      1. Using the attached zip, extract and follow the README.md steps.

    Description

      I have tables added to the connector using the table.whitelist config e.g

      {
          "connector.class" : "io.debezium.connector.sqlserver.SqlServerConnector",
          "tasks.max" : "1",
          "database.server.name" : "server1",
          "database.hostname" : "sqlserver",
          "database.port" : "1433",
          "database.user" : "sa",
          "database.password" : "****",
          "database.dbname" : "testDB",
          "database.history.kafka.bootstrap.servers" : "kafka:9092",
          "database.history.kafka.topic": "schema-changes.inventory",
          "table.whitelist": "dbo.orders,dbo.customers"
          
      }
      

      I now want to add a new table, so i update the config and update the connector e.g

      {
          "connector.class" : "io.debezium.connector.sqlserver.SqlServerConnector",
          "tasks.max" : "1",
          "database.server.name" : "server1",
          "database.hostname" : "sqlserver",
          "database.port" : "1433",
          "database.user" : "sa",
          "database.password" : "***",
          "database.dbname" : "testDB",
          "database.history.kafka.bootstrap.servers" : "kafka:9092",
          "database.history.kafka.topic": "schema-changes.inventory",
          "table.whitelist": "dbo.orders,dbo.customers,dbo.products"
          

      Now add the table to be tracked by CDC using: 

      USE testDB;
      EXEC sys.sp_cdc_enable_table @source_schema = 'dbo', @source_name = 'products', @role_name = NULL, @supports_net_changes = 0;

      The following error appears in the connect logs:

      [2020-06-30 07:14:36,508] ERROR [source-debezium-connector-sqlserver|task-0] Producer failure (io.debezium.pipeline.ErrorHandler:31)
      java.lang.NullPointerException
      at io.debezium.connector.AbstractSourceInfoStructMaker.commonStruct(AbstractSourceInfoStructMaker.java:53)
      at io.debezium.connector.sqlserver.SqlServerSourceInfoStructMaker.struct(SqlServerSourceInfoStructMaker.java:37)
      at io.debezium.connector.sqlserver.SqlServerSourceInfoStructMaker.struct(SqlServerSourceInfoStructMaker.java:14)
      at io.debezium.connector.AbstractSourceInfo.struct(AbstractSourceInfo.java:76)
      at io.debezium.connector.sqlserver.SqlServerOffsetContext.getSourceInfo(SqlServerOffsetContext.java:91)
      at io.debezium.connector.sqlserver.SqlServerSchemaChangeEventEmitter.emitSchemaChangeEvent(SqlServerSchemaChangeEventEmitter.java:42)
      at io.debezium.pipeline.EventDispatcher.dispatchSchemaChangeEvent(EventDispatcher.java:263)
      at io.debezium.connector.sqlserver.SqlServerStreamingChangeEventSource.getCdcTablesToQuery(SqlServerStreamingChangeEventSource.java:334)
      at io.debezium.connector.sqlserver.SqlServerStreamingChangeEventSource.execute(SqlServerStreamingChangeEventSource.java:108)
      at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:108)
      at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
      at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
      at java.lang.Thread.run(Thread.java:748)

      Attachments

        Issue Links

          Activity

            Public project attachment banner

              context keys: [headless, issue, helper, isAsynchronousRequest, project, action, user]
              current Project key: DBZ

              People

                jpechane Jiri Pechanec
                rob3000 Robert Shepherd (Inactive)
                Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                  Created:
                  Updated:
                  Resolved: