Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-2364

dbz-sqlserver - always on replica - connector fails after loads all data

    XMLWordPrintable

Details

    • Bug
    • Resolution: Duplicate
    • Major
    • None
    • 1.2.0.Final
    • sqlserver-connector
    • None
    • Hide
      1. run connector 
      2. load data from sql server always on replica
      3. time varies for when connector will error out with stacktrace from above.
      4. connector config 
        1. {
  "connector.class": "io.debezium.connector.sqlserver.SqlServerConnector",
  "errors.log.include.messages": "true",
  "errors.retry.timeout": "-1",
  "database.dbname": "dbname",
  "database.user": "user",
  "database.history.kafka.bootstrap.servers": "kafka.bootstrap.servers:9092",
  "acks": "1",
  "database.history.kafka.topic": "dbhistory_kafka.topic",
  "database.server.name": "name",
  "offset.flush.timeout.ms": "60000",
  "database.applicationIntent" : "ReadOnly",
  "snapshot.isolation.mode": "snapshot",
  "buffer.memory": "10000",
  "database.port": "1433",
  "table.whitelist": "dbo.whitelistname",
  "offset.flush.interval.ms": "2000",
  "tombstones.on.delete": "false",
  "database.hostname": "ip-address",
  "database.password": "database.password",
  "name": "connector_name",
  "errors.log.enable": "true"
}

          Used https://debezium.io/documentation/reference/1.2/connectors/sqlserver.html#sqlserver-always-on-replica to configure.

       

      Background - we have been using sqlserver debezium connector for past 6 months with no issue - this issue is only arising when we connect to new sql server always on replica

      Show
      run connector  load data from sql server always on replica time varies for when connector will error out with stacktrace from above. connector config { "connector.class" : "io.debezium.connector.sqlserver.SqlServerConnector" , "errors.log.include.messages" : " true " , "errors.retry.timeout" : "-1" , "database.dbname" : "dbname" , "database.user" : "user" , "database.history.kafka.bootstrap.servers" : "kafka.bootstrap.servers:9092" , "acks" : "1" , "database.history.kafka.topic" : "dbhistory_kafka.topic" , "database.server.name" : "name" , "offset.flush.timeout.ms" : "60000" , "database.applicationIntent" : "ReadOnly" , "snapshot.isolation.mode" : "snapshot" , "buffer.memory" : "10000" , "database.port" : "1433" , "table.whitelist" : "dbo.whitelistname" , "offset.flush.interval.ms" : "2000" , "tombstones.on.delete" : " false " , "database.hostname" : "ip-address" , "database.password" : "database.password" , "name" : "connector_name" , "errors.log.enable" : " true " } Used  https://debezium.io/documentation/reference/1.2/connectors/sqlserver.html#sqlserver-always-on-replica  to configure.   Background - we have been using sqlserver debezium connector for past 6 months with no issue - this issue is only arising when we connect to new sql server always on replica

    Description

      ERROR Producer failure (io.debezium.pipeline.ErrorHandler) [debezium-sqlserverconnector-tos-change-event-source-coordinator]ERROR Producer failure (io.debezium.pipeline.ErrorHandler) [debezium-sqlserverconnector-tos-change-event-source-coordinator]java.lang.NullPointerException at io.debezium.connector.AbstractSourceInfoStructMaker.commonStruct(AbstractSourceInfoStructMaker.java:53) at io.debezium.connector.sqlserver.SqlServerSourceInfoStructMaker.struct(SqlServerSourceInfoStructMaker.java:37) at io.debezium.connector.sqlserver.SqlServerSourceInfoStructMaker.struct(SqlServerSourceInfoStructMaker.java:14) at io.debezium.connector.AbstractSourceInfo.struct(AbstractSourceInfo.java:76) at io.debezium.connector.sqlserver.SqlServerOffsetContext.getSourceInfo(SqlServerOffsetContext.java:91) at io.debezium.connector.sqlserver.SqlServerSchemaChangeEventEmitter.emitSchemaChangeEvent(SqlServerSchemaChangeEventEmitter.java:42) at io.debezium.pipeline.EventDispatcher.dispatchSchemaChangeEvent(EventDispatcher.java:263) at io.debezium.connector.sqlserver.SqlServerStreamingChangeEventSource.migrateTable(SqlServerStreamingChangeEventSource.java:276) at io.debezium.connector.sqlserver.SqlServerStreamingChangeEventSource.lambda$execute$1(SqlServerStreamingChangeEventSource.java:221) at io.debezium.jdbc.JdbcConnection.prepareQuery(JdbcConnection.java:524) at io.debezium.connector.sqlserver.SqlServerConnection.getChangesForTables(SqlServerConnection.java:190) at io.debezium.connector.sqlserver.SqlServerStreamingChangeEventSource.execute(SqlServerStreamingChangeEventSource.java:162) at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:108) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)

      Attachments

        Issue Links

          duplicates

          Bug - A problem that impairs or prevents the functions of the product. DBZ-2303 Adding new table to cdc causes the sqlconnector to fail

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Closed

          Activity

            People

              Unassigned Unassigned
              fmorminojr Frank Mormino (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: