Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-5300

Snapshot fails when table's relational model is created using an abstract data type as unique index

    XMLWordPrintable

Details

    Description

      When explicitly not including table JTF_PF_REPOSITORY in the connector configuration, the Oracle connector still fails during snapshot due to the table using an ADT primary key.

      [2022-06-22 14:52:08,375] INFO WorkerSourceTask{id=kafka-connect-src-01-0} flushing 0 outstanding messages for offset commit (org.apache.kafka.connect.runtime.WorkerSourceTask)
[2022-06-22 14:52:13,792] INFO Snapshot - Final stage (io.debezium.pipeline.source.AbstractSnapshotChangeEventSource)
[2022-06-22 14:52:13,853] ERROR Producer failure (io.debezium.pipeline.ErrorHandler)
io.debezium.DebeziumException: java.lang.IllegalArgumentException: The column ""PAGEOBJECT"."INFO"."RECID"" is referenced as PRIMARY KEY, but a matching column is not defined in table "myserver.JTF.JTF_PF_REPOSITORY"!
        at io.debezium.pipeline.source.AbstractSnapshotChangeEventSource.execute(AbstractSnapshotChangeEventSource.java:85)
        at io.debezium.pipeline.ChangeEventSourceCoordinator.doSnapshot(ChangeEventSourceCoordinator.java:155)
        at io.debezium.pipeline.ChangeEventSourceCoordinator.executeChangeEventSources(ChangeEventSourceCoordinator.java:137)
        at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:109)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.lang.IllegalArgumentException: The column ""PAGEOBJECT"."INFO"."RECID"" is referenced as PRIMARY KEY, but a matching column is not defined in table "myserver.JTF.JTF_PF_REPOSITORY"!
        at io.debezium.relational.TableEditorImpl.lambda$updatePrimaryKeys$0(TableEditorImpl.java:106)
        at java.base/java.util.ArrayList.removeIf(ArrayList.java:1702)
        at java.base/java.util.ArrayList.removeIf(ArrayList.java:1690)
        at io.debezium.relational.TableEditorImpl.updatePrimaryKeys(TableEditorImpl.java:102)
        at io.debezium.relational.TableEditorImpl.create(TableEditorImpl.java:267)
        at io.debezium.relational.Tables.lambda$overwriteTable$2(Tables.java:192)
        at io.debezium.util.FunctionalReadWriteLock.write(FunctionalReadWriteLock.java:84)
        at io.debezium.relational.Tables.overwriteTable(Tables.java:186)
        at io.debezium.jdbc.JdbcConnection.readSchema(JdbcConnection.java:1209)
        at io.debezium.connector.oracle.OracleSnapshotChangeEventSource.readTableStructure(OracleSnapshotChangeEventSource.java:181)
        at io.debezium.connector.oracle.OracleSnapshotChangeEventSource.readTableStructure(OracleSnapshotChangeEventSource.java:35)
        at io.debezium.relational.RelationalSnapshotChangeEventSource.doExecute(RelationalSnapshotChangeEventSource.java:114)
        at io.debezium.pipeline.source.AbstractSnapshotChangeEventSource.execute(AbstractSnapshotChangeEventSource.java:76)
        ... 8 more
[2022-06-22 14:52:13,855] INFO Connected metrics set to 'false' (io.debezium.pipeline.ChangeEventSourceCoordinator)
[2022-06-22 14:52:14,112] INFO WorkerSourceTask{id=kafka-connect-src-01-0} flushing 0 outstanding messages for offset commit (org.apache.kafka.connect.runtime.WorkerSourceTask)
[2022-06-22 14:52:14,112] ERROR WorkerSourceTask{id=kafka-connect-src-01-0} Task threw an uncaught and unrecoverable exception. Task is being killed and will not recover until manually restarted (org.apache.kafka.connect.runtime.WorkerTask)
org.apache.kafka.connect.errors.ConnectException: An exception occurred in the change event producer. This connector will be stopped.
        at io.debezium.pipeline.ErrorHandler.setProducerThrowable(ErrorHandler.java:50)
        at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:116)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: io.debezium.DebeziumException: java.lang.IllegalArgumentException: The column ""PAGEOBJECT"."INFO"."RECID"" is referenced as PRIMARY KEY, but a matching column is not defined in table "myserver.JTF.JTF_PF_REPOSITORY"!
        at io.debezium.pipeline.source.AbstractSnapshotChangeEventSource.execute(AbstractSnapshotChangeEventSource.java:85)
        at io.debezium.pipeline.ChangeEventSourceCoordinator.doSnapshot(ChangeEventSourceCoordinator.java:155)
        at io.debezium.pipeline.ChangeEventSourceCoordinator.executeChangeEventSources(ChangeEventSourceCoordinator.java:137)
        at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:109)
        ... 5 more
Caused by: java.lang.IllegalArgumentException: The column ""PAGEOBJECT"."INFO"."RECID"" is referenced as PRIMARY KEY, but a matching column is not defined in table "myserver.JTF.JTF_PF_REPOSITORY"!
        at io.debezium.relational.TableEditorImpl.lambda$updatePrimaryKeys$0(TableEditorImpl.java:106)
        at java.base/java.util.ArrayList.removeIf(ArrayList.java:1702)
        at java.base/java.util.ArrayList.removeIf(ArrayList.java:1690)
        at io.debezium.relational.TableEditorImpl.updatePrimaryKeys(TableEditorImpl.java:102)
        at io.debezium.relational.TableEditorImpl.create(TableEditorImpl.java:267)
        at io.debezium.relational.Tables.lambda$overwriteTable$2(Tables.java:192)
        at io.debezium.util.FunctionalReadWriteLock.write(FunctionalReadWriteLock.java:84)
        at io.debezium.relational.Tables.overwriteTable(Tables.java:186)
        at io.debezium.jdbc.JdbcConnection.readSchema(JdbcConnection.java:1209)
        at io.debezium.connector.oracle.OracleSnapshotChangeEventSource.readTableStructure(OracleSnapshotChangeEventSource.java:181)
        at io.debezium.connector.oracle.OracleSnapshotChangeEventSource.readTableStructure(OracleSnapshotChangeEventSource.java:35)
        at io.debezium.relational.RelationalSnapshotChangeEventSource.doExecute(RelationalSnapshotChangeEventSource.java:114)
        at io.debezium.pipeline.source.AbstractSnapshotChangeEventSource.execute(AbstractSnapshotChangeEventSource.java:76)
        ... 8 more
[2022-06-22 14:52:14,112] INFO Stopping down connector (io.debezium.connector.common.BaseSourceTask)
[2022-06-22 14:52:14,123] INFO Connection gracefully closed (io.debezium.jdbc.JdbcConnection)

       

      CREATE TABLE "JTF"."JTF_PF_REPOSITORY" 
   (    "PAGEOBJECT" "APPS"."JTF_PF_PAGE_OBJECT" , 
    "OBJECT_VERSION_NUMBER" NUMBER NOT NULL ENABLE, 
    "CREATED_BY" NUMBER(15,0) NOT NULL ENABLE, 
    "CREATION_DATE" DATE NOT NULL ENABLE, 
    "LAST_UPDATED_BY" NUMBER(15,0) NOT NULL ENABLE, 
    "LAST_UPDATE_DATE" DATE NOT NULL ENABLE, 
    "LAST_UPDATE_LOGIN" NUMBER(15,0), 
    "SECURITY_GROUP_ID" NUMBER, 
    "TECH_STACK" VARCHAR2(20) DEFAULT 'JTF', 
    "TRACK_PURPOSE" NUMBER DEFAULT 1 NOT NULL ENABLE
   ) PCTFREE 10 PCTUSED 40 INITRANS 1 MAXTRANS 255 
  STORAGE(
  BUFFER_POOL DEFAULT FLASH_CACHE DEFAULT CELL_FLASH_CACHE DEFAULT)
  TABLESPACE "APPS_TS_TX_DATA" 
  PARTITION BY LIST ("TRACK_PURPOSE") 
 (PARTITION "NON_EBIZ"  VALUES (4) SEGMENT CREATION IMMEDIATE 
  PCTFREE 10 PCTUSED 40 INITRANS 1 MAXTRANS 255 
 NOCOMPRESS LOGGING 
  STORAGE(INITIAL 8388608 NEXT 1048576 MINEXTENTS 1 MAXEXTENTS 2147483645
  PCTINCREASE 0 FREELISTS 1 FREELIST GROUPS 1
  BUFFER_POOL DEFAULT FLASH_CACHE DEFAULT CELL_FLASH_CACHE DEFAULT)
  TABLESPACE "APPS_TS_TX_DATA" , 
 PARTITION "EBIZ"  VALUES (DEFAULT) SEGMENT CREATION IMMEDIATE 
  PCTFREE 10 PCTUSED 40 INITRANS 1 MAXTRANS 255 
 NOCOMPRESS LOGGING 
  STORAGE(INITIAL 8388608 NEXT 1048576 MINEXTENTS 1 MAXEXTENTS 2147483645
  PCTINCREASE 0 FREELISTS 1 FREELIST GROUPS 1
  BUFFER_POOL DEFAULT FLASH_CACHE DEFAULT CELL_FLASH_CACHE DEFAULT)
  TABLESPACE "APPS_TS_TX_DATA" ) 


TYPE        "JTF_PF_PAGE_OBJECT" AS OBJECT (info JTF_PF_INFO_OBJECT,dpf JTF_PF_DPF_OBJECT,ses JTF_PF_SESSION_OBJECT,jsp JTF_PF_JSP_OBJECT,loc JTF_PF_LOCATION_OBJECT,lang JTF_PF_LANGUAGE_OBJECT,client JTF_PF_CLIENT_OBJECT,params JTF_PF_PARAMS_TABLE,cookies JTF_PF_COOKIES_TABLE,headers JTF_PF_PARAMS_TABLE)
ALTER TYPE JTF_PF_PAGE_OBJECT ADD ATTRIBUTE(PRODINFO JTF_PF_PRODINFO_OBJECT) CASCADE
TYPE        "JTF_PF_INFO_OBJECT" AS OBJECT (RecId INTEGER,Day DATE,Timestamp DATE)

      Attachments

        Issue Links

          is duplicated by

          Bug - A problem that impairs or prevents the functions of the product. DBZ-5298 The column ""XMLDATA"."LINEITEMS"."LINEITEM"" is referenced as PRIMARY KEY, but a matching column is not defined in table "EDW.OE.PURCHASEORDER"!

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Closed
          is related to

          Feature Request - Feature requests from customers and/or users DBZ-5286 Support tables that use custom abstract data types (ADT) as primary keys

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Open

          Activity

            People

              ccranfor@redhat.com Chris Cranford
              ccranfor@redhat.com Chris Cranford
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: