Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-2522

ArrayIndexOutOfBoundsException with excluded column from CDC table

XMLWordPrintable

    • Hide

      Exclude a column from the CDC tables that is in the middle of the table via captured_column_list and the exclude list in the config.

       

      Show
      Exclude a column from the CDC tables that is in the middle of the table via captured_column_list and the exclude list in the config.  

      Follow up testing from DBZ-1068 reveals issue that only columns at the end of table can be excluded with captured_column_list as excluding columns in the middle of table throws an ArrayIndexOutOfBoundsException.

      With this table

      CREATE TABLE blacklist_column_table_a (id int, amount integer, name varchar(30), primary key(id))
      

      With this in the connector configuration

      column.exclude.list: dbo.blacklist_column_table_a.amount
      

      and this as a parameter to the cdc enable statement

      @captured_column_list = 'id,name'
      

      leads to 

      org.apache.kafka.connect.errors.ConnectException: An exception occurred in the change event producer. This connector will be stopped.org.apache.kafka.connect.errors.ConnectException: An exception occurred in the change event producer. This connector will be stopped. at io.debezium.pipeline.ErrorHandler.setProducerThrowable(ErrorHandler.java:42) at io.debezium.connector.sqlserver.SqlServerStreamingChangeEventSource.execute(SqlServerStreamingChangeEventSource.java:282) at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:108) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)Caused by: java.lang.ArrayIndexOutOfBoundsException: 2 at io.debezium.connector.sqlserver.SqlServerChangeTablePointer.lambda$createResultSetMapper$1(SqlServerChangeTablePointer.java:128) at io.debezium.connector.sqlserver.SqlServerChangeTablePointer.getData(SqlServerChangeTablePointer.java:90) at io.debezium.connector.sqlserver.SqlServerStreamingChangeEventSource.lambda$execute$1(SqlServerStreamingChangeEventSource.java:240) at io.debezium.jdbc.JdbcConnection.prepareQuery(JdbcConnection.java:539) at io.debezium.connector.sqlserver.SqlServerConnection.getChangesForTables(SqlServerConnection.java:194) at io.debezium.connector.sqlserver.SqlServerStreamingChangeEventSource.execute(SqlServerStreamingChangeEventSource.java:163) ... 6 more
      

              Unassigned Unassigned
              jgormley6 James Gormley (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Created:
                Updated:
                Resolved: