Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-5300

Snapshot fails when table's relational model is created using an abstract data type as unique index

    XMLWordPrintable

Details

    Description

      When explicitly not including table JTF_PF_REPOSITORY in the connector configuration, the Oracle connector still fails during snapshot due to the table using an ADT primary key.

      [2022-06-22 14:52:08,375] INFO WorkerSourceTask{id=kafka-connect-src-01-0} flushing 0 outstanding messages for offset commit (org.apache.kafka.connect.runtime.WorkerSourceTask)
      [2022-06-22 14:52:13,792] INFO Snapshot - Final stage (io.debezium.pipeline.source.AbstractSnapshotChangeEventSource)
      [2022-06-22 14:52:13,853] ERROR Producer failure (io.debezium.pipeline.ErrorHandler)
      io.debezium.DebeziumException: java.lang.IllegalArgumentException: The column ""PAGEOBJECT"."INFO"."RECID"" is referenced as PRIMARY KEY, but a matching column is not defined in table "myserver.JTF.JTF_PF_REPOSITORY"!
              at io.debezium.pipeline.source.AbstractSnapshotChangeEventSource.execute(AbstractSnapshotChangeEventSource.java:85)
              at io.debezium.pipeline.ChangeEventSourceCoordinator.doSnapshot(ChangeEventSourceCoordinator.java:155)
              at io.debezium.pipeline.ChangeEventSourceCoordinator.executeChangeEventSources(ChangeEventSourceCoordinator.java:137)
              at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:109)
              at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
              at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
              at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
              at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
              at java.base/java.lang.Thread.run(Thread.java:829)
      Caused by: java.lang.IllegalArgumentException: The column ""PAGEOBJECT"."INFO"."RECID"" is referenced as PRIMARY KEY, but a matching column is not defined in table "myserver.JTF.JTF_PF_REPOSITORY"!
              at io.debezium.relational.TableEditorImpl.lambda$updatePrimaryKeys$0(TableEditorImpl.java:106)
              at java.base/java.util.ArrayList.removeIf(ArrayList.java:1702)
              at java.base/java.util.ArrayList.removeIf(ArrayList.java:1690)
              at io.debezium.relational.TableEditorImpl.updatePrimaryKeys(TableEditorImpl.java:102)
              at io.debezium.relational.TableEditorImpl.create(TableEditorImpl.java:267)
              at io.debezium.relational.Tables.lambda$overwriteTable$2(Tables.java:192)
              at io.debezium.util.FunctionalReadWriteLock.write(FunctionalReadWriteLock.java:84)
              at io.debezium.relational.Tables.overwriteTable(Tables.java:186)
              at io.debezium.jdbc.JdbcConnection.readSchema(JdbcConnection.java:1209)
              at io.debezium.connector.oracle.OracleSnapshotChangeEventSource.readTableStructure(OracleSnapshotChangeEventSource.java:181)
              at io.debezium.connector.oracle.OracleSnapshotChangeEventSource.readTableStructure(OracleSnapshotChangeEventSource.java:35)
              at io.debezium.relational.RelationalSnapshotChangeEventSource.doExecute(RelationalSnapshotChangeEventSource.java:114)
              at io.debezium.pipeline.source.AbstractSnapshotChangeEventSource.execute(AbstractSnapshotChangeEventSource.java:76)
              ... 8 more
      [2022-06-22 14:52:13,855] INFO Connected metrics set to 'false' (io.debezium.pipeline.ChangeEventSourceCoordinator)
      [2022-06-22 14:52:14,112] INFO WorkerSourceTask{id=kafka-connect-src-01-0} flushing 0 outstanding messages for offset commit (org.apache.kafka.connect.runtime.WorkerSourceTask)
      [2022-06-22 14:52:14,112] ERROR WorkerSourceTask{id=kafka-connect-src-01-0} Task threw an uncaught and unrecoverable exception. Task is being killed and will not recover until manually restarted (org.apache.kafka.connect.runtime.WorkerTask)
      org.apache.kafka.connect.errors.ConnectException: An exception occurred in the change event producer. This connector will be stopped.
              at io.debezium.pipeline.ErrorHandler.setProducerThrowable(ErrorHandler.java:50)
              at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:116)
              at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
              at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
              at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
              at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
              at java.base/java.lang.Thread.run(Thread.java:829)
      Caused by: io.debezium.DebeziumException: java.lang.IllegalArgumentException: The column ""PAGEOBJECT"."INFO"."RECID"" is referenced as PRIMARY KEY, but a matching column is not defined in table "myserver.JTF.JTF_PF_REPOSITORY"!
              at io.debezium.pipeline.source.AbstractSnapshotChangeEventSource.execute(AbstractSnapshotChangeEventSource.java:85)
              at io.debezium.pipeline.ChangeEventSourceCoordinator.doSnapshot(ChangeEventSourceCoordinator.java:155)
              at io.debezium.pipeline.ChangeEventSourceCoordinator.executeChangeEventSources(ChangeEventSourceCoordinator.java:137)
              at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:109)
              ... 5 more
      Caused by: java.lang.IllegalArgumentException: The column ""PAGEOBJECT"."INFO"."RECID"" is referenced as PRIMARY KEY, but a matching column is not defined in table "myserver.JTF.JTF_PF_REPOSITORY"!
              at io.debezium.relational.TableEditorImpl.lambda$updatePrimaryKeys$0(TableEditorImpl.java:106)
              at java.base/java.util.ArrayList.removeIf(ArrayList.java:1702)
              at java.base/java.util.ArrayList.removeIf(ArrayList.java:1690)
              at io.debezium.relational.TableEditorImpl.updatePrimaryKeys(TableEditorImpl.java:102)
              at io.debezium.relational.TableEditorImpl.create(TableEditorImpl.java:267)
              at io.debezium.relational.Tables.lambda$overwriteTable$2(Tables.java:192)
              at io.debezium.util.FunctionalReadWriteLock.write(FunctionalReadWriteLock.java:84)
              at io.debezium.relational.Tables.overwriteTable(Tables.java:186)
              at io.debezium.jdbc.JdbcConnection.readSchema(JdbcConnection.java:1209)
              at io.debezium.connector.oracle.OracleSnapshotChangeEventSource.readTableStructure(OracleSnapshotChangeEventSource.java:181)
              at io.debezium.connector.oracle.OracleSnapshotChangeEventSource.readTableStructure(OracleSnapshotChangeEventSource.java:35)
              at io.debezium.relational.RelationalSnapshotChangeEventSource.doExecute(RelationalSnapshotChangeEventSource.java:114)
              at io.debezium.pipeline.source.AbstractSnapshotChangeEventSource.execute(AbstractSnapshotChangeEventSource.java:76)
              ... 8 more
      [2022-06-22 14:52:14,112] INFO Stopping down connector (io.debezium.connector.common.BaseSourceTask)
      [2022-06-22 14:52:14,123] INFO Connection gracefully closed (io.debezium.jdbc.JdbcConnection)
       

       

      CREATE TABLE "JTF"."JTF_PF_REPOSITORY" 
         (    "PAGEOBJECT" "APPS"."JTF_PF_PAGE_OBJECT" , 
          "OBJECT_VERSION_NUMBER" NUMBER NOT NULL ENABLE, 
          "CREATED_BY" NUMBER(15,0) NOT NULL ENABLE, 
          "CREATION_DATE" DATE NOT NULL ENABLE, 
          "LAST_UPDATED_BY" NUMBER(15,0) NOT NULL ENABLE, 
          "LAST_UPDATE_DATE" DATE NOT NULL ENABLE, 
          "LAST_UPDATE_LOGIN" NUMBER(15,0), 
          "SECURITY_GROUP_ID" NUMBER, 
          "TECH_STACK" VARCHAR2(20) DEFAULT 'JTF', 
          "TRACK_PURPOSE" NUMBER DEFAULT 1 NOT NULL ENABLE
         ) PCTFREE 10 PCTUSED 40 INITRANS 1 MAXTRANS 255 
        STORAGE(
        BUFFER_POOL DEFAULT FLASH_CACHE DEFAULT CELL_FLASH_CACHE DEFAULT)
        TABLESPACE "APPS_TS_TX_DATA" 
        PARTITION BY LIST ("TRACK_PURPOSE") 
       (PARTITION "NON_EBIZ"  VALUES (4) SEGMENT CREATION IMMEDIATE 
        PCTFREE 10 PCTUSED 40 INITRANS 1 MAXTRANS 255 
       NOCOMPRESS LOGGING 
        STORAGE(INITIAL 8388608 NEXT 1048576 MINEXTENTS 1 MAXEXTENTS 2147483645
        PCTINCREASE 0 FREELISTS 1 FREELIST GROUPS 1
        BUFFER_POOL DEFAULT FLASH_CACHE DEFAULT CELL_FLASH_CACHE DEFAULT)
        TABLESPACE "APPS_TS_TX_DATA" , 
       PARTITION "EBIZ"  VALUES (DEFAULT) SEGMENT CREATION IMMEDIATE 
        PCTFREE 10 PCTUSED 40 INITRANS 1 MAXTRANS 255 
       NOCOMPRESS LOGGING 
        STORAGE(INITIAL 8388608 NEXT 1048576 MINEXTENTS 1 MAXEXTENTS 2147483645
        PCTINCREASE 0 FREELISTS 1 FREELIST GROUPS 1
        BUFFER_POOL DEFAULT FLASH_CACHE DEFAULT CELL_FLASH_CACHE DEFAULT)
        TABLESPACE "APPS_TS_TX_DATA" ) 
      
      
      TYPE        "JTF_PF_PAGE_OBJECT" AS OBJECT (info JTF_PF_INFO_OBJECT,dpf JTF_PF_DPF_OBJECT,ses JTF_PF_SESSION_OBJECT,jsp JTF_PF_JSP_OBJECT,loc JTF_PF_LOCATION_OBJECT,lang JTF_PF_LANGUAGE_OBJECT,client JTF_PF_CLIENT_OBJECT,params JTF_PF_PARAMS_TABLE,cookies JTF_PF_COOKIES_TABLE,headers JTF_PF_PARAMS_TABLE)
      ALTER TYPE JTF_PF_PAGE_OBJECT ADD ATTRIBUTE(PRODINFO JTF_PF_PRODINFO_OBJECT) CASCADE
      TYPE        "JTF_PF_INFO_OBJECT" AS OBJECT (RecId INTEGER,Day DATE,Timestamp DATE) 
      
      

      Attachments

        Issue Links

          Activity

            People

              ccranfor@redhat.com Chris Cranford
              ccranfor@redhat.com Chris Cranford
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: