Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-5337

Oracle XMLTYPE tables are not supported

    XMLWordPrintable

Details

    • Feature Request
    • Resolution: Unresolved
    • Major
    • 2.7.0.Alpha2
    • 1.9.4.Final, 2.0.0.Alpha2
    • oracle-connector
    • None

    Description

       

      <2022-06-23 17:55:00,437 INFO [io.deb.rel.RelationalSnapshotChangeEventSource] (debezium-oracleconnector-ora193_server01-change-event-source-coordinator) Snapshot step 5 - Reading structure of captured tables
      2022-06-23 17:55:00,439 INFO [io.deb.con.ora.OracleSnapshotChangeEventSource] (debezium-oracleconnector-ora193_server01-change-event-source-coordinator) All eligible tables schema should be captured, capturing:
      2022-06-23 17:55:05,944 INFO [io.deb.pip.sou.AbstractSnapshotChangeEventSource] (debezium-oracleconnector-ora193_server01-change-event-source-coordinator) Snapshot - Final stage
      2022-06-23 17:43:26,945 ERROR [io.deb.pip.ErrorHandler] (debezium-oracleconnector-ora193_server01-change-event-source-coordinator) Producer failure: io.debezium.DebeziumException: java.lang.IllegalArgumentException: The column ""XMLDATA"."LINEITEMS"."LINEITEM"" is referenced as PRIMARY KEY, but a matching column is not defined in table "EDW.OE.PURCHASEORDER"!
      at io.debezium.pipeline.source.AbstractSnapshotChangeEventSource.execute(AbstractSnapshotChangeEventSource.java:85)
      at io.debezium.pipeline.ChangeEventSourceCoordinator.doSnapshot(ChangeEventSourceCoordinator.java:155)
      at io.debezium.pipeline.ChangeEventSourceCoordinator.executeChangeEventSources(ChangeEventSourceCoordinator.java:137)
      at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:109)
      at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
      at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
      at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
      at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
      at java.base/java.lang.Thread.run(Thread.java:834)
      Caused by: java.lang.IllegalArgumentException: The column ""XMLDATA"."LINEITEMS"."LINEITEM"" is referenced as PRIMARY KEY, but a matching column is not defined in table "EDW.OE.PURCHASEORDER"!
      at io.debezium.relational.TableEditorImpl.lambda$updatePrimaryKeys$0(TableEditorImpl.java:106)
      at java.base/java.util.ArrayList.removeIf(ArrayList.java:1702)
      at java.base/java.util.ArrayList.removeIf(ArrayList.java:1690)
      at io.debezium.relational.TableEditorImpl.updatePrimaryKeys(TableEditorImpl.java:102)
      at io.debezium.relational.TableEditorImpl.create(TableEditorImpl.java:267)
      at io.debezium.relational.Tables.lambda$overwriteTable$2(Tables.java:192)
      at io.debezium.util.FunctionalReadWriteLock.write(FunctionalReadWriteLock.java:84)
      at io.debezium.relational.Tables.overwriteTable(Tables.java:186)
      at io.debezium.jdbc.JdbcConnection.readSchema(JdbcConnection.java:1209)
      at io.debezium.connector.oracle.OracleSnapshotChangeEventSource.readTableStructure(OracleSnapshotChangeEventSource.java:181)
      at io.debezium.connector.oracle.OracleSnapshotChangeEventSource.readTableStructure(OracleSnapshotChangeEventSource.java:35)
      at io.debezium.relational.RelationalSnapshotChangeEventSource.doExecute(RelationalSnapshotChangeEventSource.java:114)
      at io.debezium.pipeline.source.AbstractSnapshotChangeEventSource.execute(AbstractSnapshotChangeEventSource.java:76)
      ... 8 morer>

       

       

      my <application.properties>:
      debezium.sink.type=kafka
      debezium.source.database.server.name=ora193_server01
      debezium.sink.kafka.producer.bootstrap.servers=192.168.0.xxx:9092
      debezium.sink.kafka.producer.key.serializer=org.apache.kafka.common.serialization.StringSerializer
      debezium.sink.kafka.producer.value.serializer=org.apache.kafka.common.serialization.StringSerializer
      debezium.source.connector.class=io.debezium.connector.oracle.OracleConnector
      debezium.source.database.history=io.debezium.relational.history.FileDatabaseHistory
      debezium.source.offset.storage.file.filename=D:/JobUtil/Idea2020.2.3WorkSpace/geome_cdc/debezium/cdc_kafka/offsets.dat
      debezium.source.database.history.file.filename=D:/JobUtil/Idea2020.2.3WorkSpace/geome_cdc/debezium/cdc_kafka/history.dat
      debezium.source.offset.flush.interval.ms=0

      debezium.source.database.hostname=192.168.0.xxx
      debezium.source.database.port=1521
      debezium.source.database.user=cdc
      debezium.source.database.password=xxx
      debezium.source.database.dbname=edw
      debezium.source.schema.include.list=INMON
      debezium.source.table.include.list=INMON.A

      debezium.source.database.history.kafka.bootstrap.servers=192.168.0.xxx:9092
      debezium.source.database.history.kafka.topic=ora_history
      debezium.source.log.mining.strategy=online_catalog
      debezium.source.decimal.handling.mode=string

      debezium.source.tombstones.on.delete=false

       

       

       
      dbz-5300 is a bit similar but for unique index, whereas this is for primary key

      the other fixed a different method https://github.com/debezium/debezium/blob/v1.9.4.Final/debezium-core/src/main/java/io/debezium/jdbc/JdbcConnection.java#L1331 

      but https://github.com/debezium/debezium/blob/v1.9.4.Final/debezium-core/src/main/java/io/debezium/jdbc/JdbcConnection.java#L1312 is still affected

       
      cc ccranfor@redhat.com 

      Attachments

        Issue Links

          Activity

            People

              ccranfor@redhat.com Chris Cranford
              tooptoop toop toop (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated: