Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-6890

Dropping of primary key not reflected properly

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Major Major
    • 2.5-backlog
    • 2.3.0.Final
    • oracle-connector
    • None
    • False
    • None
    • False
    • Moderate

      What Debezium connector do you use and what version?

      Embedded Debezium with Oracle connector v 2.3.0.Final

      What is the connector configuration?

      connector.class = io.debezium.connector.oracle.OracleConnector
      log.mining.strategy = online_catalog
      schema.history.internal.store.only.captured.tables.ddl = true
      schema.history.internal.store.only.captured.databases.ddl = true
      schema.history.internal.file.filename = C:\Users\pv\data\pv_DebeziumPoC\debezium-interfaceId87549-schemahist
      topic.prefix = pv_DebeziumPoC
      decimal.handling.mode = STRING
      offset.storage.file.filename = C:\Users\pv\data\pv_DebeziumPoC\debezium-interfaceId87549-offset
      signal.data.collection = MWBIZINT.c##debezuser.DEBEZIUM_SIGNAL
      database.out.server.name = debezium_interfaceId87549
      errors.retry.delay.initial.ms = 300
      value.converter = org.apache.kafka.connect.json.JsonConverter
      key.converter = org.apache.kafka.connect.json.JsonConverter
      offset.commit.policy = io.debezium.engine.spi.OffsetCommitPolicy$PeriodicCommitOffsetPolicy
      database.user = c##debezuser
      database.dbname = C24MWB
      offset.storage = org.apache.kafka.connect.storage.FileOffsetBackingStore
      database.pdb.name = MWBIZINT
      database.url = jdbc:oracle:thin:@//exa02cl02-scan.onsemi.com:1521/svcMWBIZINT.onsemi.com
      offset.flush.timeout.ms = 5000
      errors.retry.delay.max.ms = 10000
      schema.history.internal.skip.unparseable.ddl = true
      offset.flush.interval.ms = 60000
      schema.history.internal = io.debezium.storage.file.history.FileSchemaHistory
      errors.max.retries = -1
      log.mining.query.filter.mode = none
      database.password = ********
      name = debezium-oracle:debezium_interfaceId_87549
      table.include.list = C##DEBEZUSER.DEBEZIUM_TEST_PAVEL,C##DEBEZUSER.DEBEZIUM_TEST_PAVEL2

      What is the captured database version and mode of depoyment?

      Oracle on-prem Database 19c EE Extreme Perf Release 19.0.0.0.0

      Bug description

       
      I'm trying to maintain an up-to-date collection of primary keys.
       
      To achieve this, I'm reading tableChanges -> table -> primaryKeyColumnNames from incoming connector events. It's working as expected except for one case: when a primary key is dropped, the primaryKeyColumnNames field value doesn't reflect that, even though ddl field has correct statement.
       
      Compare the following two messages:

      • First one comes when a primary key is dropped (primaryKeyColumnNames does not reflect the after state of ddl statement).
      • Second one when a new primary key is defined and there was no primary key on the table before this (in this case primaryKeyColumnNames correctly reflects the after state of ddl statement).

       
      Struct{source=Struct

      {version=2.3.0.Final,connector=oracle,name=pv_DebeziumPoC,ts_ms=1693532976000,db=MWBIZINT,schema=C##DEBEZUSER,table=DEBEZIUM_TEST_PAVEL,scn=11268870737191,commit_scn=11268870737191,rs_id=0x006eb1.00001c99.0090,ssn=0,redo_thread=2}

      ,ts_ms=1693558186410,databaseName=MWBIZINT,schemaName=C##DEBEZUSER,ddl=ALTER TABLE C##DEBEZUSER.DEBEZIUM_TEST_PAVEL DROP PRIMARY KEY;,tableChanges=[Struct{type=ALTER,id="MWBIZINT"."C##DEBEZUSER"."DEBEZIUM_TEST_PAVEL",table=Struct{primaryKeyColumnNames=[ID],columns=[Struct{name=ID,jdbcType=2,typeName=NUMBER,typeExpression=NUMBER,length=0,position=1,optional=false,autoIncremented=false,generated=false}, Struct{name=NAME,jdbcType=12,typeName=VARCHAR2,typeExpression=VARCHAR2,length=4000,position=2,optional=false,autoIncremented=false,generated=false}, Struct{name=ACT_VALUE,jdbcType=12,typeName=VARCHAR2,typeExpression=VARCHAR2,length=4000,position=3,optional=true,autoIncremented=false,generated=false}, Struct{name=NEW_COLUMN,jdbcType=12,typeName=VARCHAR2,typeExpression=VARCHAR2,length=20,position=4,optional=true,autoIncremented=false,generated=false}]}}]}

      Struct{source=Struct

      {version=2.3.0.Final,connector=oracle,name=pv_DebeziumPoC,ts_ms=1693533233000,db=MWBIZINT,schema=C##DEBEZUSER,table=DEBEZIUM_TEST_PAVEL,scn=11268870738964,commit_scn=11268870738964,rs_id=0x006eb2.00000a04.0060,ssn=0,redo_thread=2}

      ,ts_ms=1693558435668,databaseName=MWBIZINT,schemaName=C##DEBEZUSER,ddl=ALTER TABLE C##DEBEZUSER.DEBEZIUM_TEST_PAVEL ADD PRIMARY KEY ("ID", "NAME");,tableChanges=Struct{type=ALTER,id="MWBIZINT"."C##DEBEZUSER"."DEBEZIUM_TEST_PAVEL",table=Struct{{color:#008000}primaryKeyColumnNames=[ID, NAME],columns=[Struct{name=ID,jdbcType=2,typeName=NUMBER,typeExpression=NUMBER,length=0,position=1,optional=false,autoIncremented=false,generated=false}, Struct{name=NAME,jdbcType=12,typeName=VARCHAR2,typeExpression=VARCHAR2,length=4000,position=2,optional=false,autoIncremented=false,generated=false}, Struct{name=ACT_VALUE,jdbcType=12,typeName=VARCHAR2,typeExpression=VARCHAR2,length=4000,position=3,optional=true,autoIncremented=false,generated=false}, Struct{name=NEW_COLUMN,jdbcType=12,typeName=VARCHAR2,typeExpression=VARCHAR2,length=20,position=4,optional=true,autoIncremented=false,generated=false}]}}]}

      Expected behavior

      My expectation is that primaryKeyColumnNames behaves consistently, i.e., that with every DDL change it always shows the list of keys reflecting the after state, so the use case with dropping PK should be fixed.

            Unassigned Unassigned
            vrtal.pavel Pavel Vrtal
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated: