-
Bug
-
Resolution: Unresolved
-
Major
-
2.3.0.Final
-
None
-
False
-
None
-
False
-
Moderate
What Debezium connector do you use and what version?
Embedded Debezium with Oracle connector v 2.3.0.Final
What is the connector configuration?
connector.class = io.debezium.connector.oracle.OracleConnector
log.mining.strategy = online_catalog
schema.history.internal.store.only.captured.tables.ddl = true
schema.history.internal.store.only.captured.databases.ddl = true
schema.history.internal.file.filename = C:\Users\pv\data\pv_DebeziumPoC\debezium-interfaceId87549-schemahist
topic.prefix = pv_DebeziumPoC
decimal.handling.mode = STRING
offset.storage.file.filename = C:\Users\pv\data\pv_DebeziumPoC\debezium-interfaceId87549-offset
signal.data.collection = MWBIZINT.c##debezuser.DEBEZIUM_SIGNAL
database.out.server.name = debezium_interfaceId87549
errors.retry.delay.initial.ms = 300
value.converter = org.apache.kafka.connect.json.JsonConverter
key.converter = org.apache.kafka.connect.json.JsonConverter
offset.commit.policy = io.debezium.engine.spi.OffsetCommitPolicy$PeriodicCommitOffsetPolicy
database.user = c##debezuser
database.dbname = C24MWB
offset.storage = org.apache.kafka.connect.storage.FileOffsetBackingStore
database.pdb.name = MWBIZINT
database.url = jdbc:oracle:thin:@//exa02cl02-scan.onsemi.com:1521/svcMWBIZINT.onsemi.com
offset.flush.timeout.ms = 5000
errors.retry.delay.max.ms = 10000
schema.history.internal.skip.unparseable.ddl = true
offset.flush.interval.ms = 60000
schema.history.internal = io.debezium.storage.file.history.FileSchemaHistory
errors.max.retries = -1
log.mining.query.filter.mode = none
database.password = ********
name = debezium-oracle:debezium_interfaceId_87549
table.include.list = C##DEBEZUSER.DEBEZIUM_TEST_PAVEL,C##DEBEZUSER.DEBEZIUM_TEST_PAVEL2
What is the captured database version and mode of depoyment?
Oracle on-prem Database 19c EE Extreme Perf Release 19.0.0.0.0
Bug description
I'm trying to maintain an up-to-date collection of primary keys.
To achieve this, I'm reading tableChanges -> table -> primaryKeyColumnNames from incoming connector events. It's working as expected except for one case: when a primary key is dropped, the primaryKeyColumnNames field value doesn't reflect that, even though ddl field has correct statement.
Compare the following two messages:
- First one comes when a primary key is dropped (primaryKeyColumnNames does not reflect the after state of ddl statement).
- Second one when a new primary key is defined and there was no primary key on the table before this (in this case primaryKeyColumnNames correctly reflects the after state of ddl statement).
Struct{source=Struct
,ts_ms=1693558186410,databaseName=MWBIZINT,schemaName=C##DEBEZUSER,ddl=ALTER TABLE C##DEBEZUSER.DEBEZIUM_TEST_PAVEL DROP PRIMARY KEY;,tableChanges=[Struct{type=ALTER,id="MWBIZINT"."C##DEBEZUSER"."DEBEZIUM_TEST_PAVEL",table=Struct{primaryKeyColumnNames=[ID],columns=[Struct{name=ID,jdbcType=2,typeName=NUMBER,typeExpression=NUMBER,length=0,position=1,optional=false,autoIncremented=false,generated=false}, Struct{name=NAME,jdbcType=12,typeName=VARCHAR2,typeExpression=VARCHAR2,length=4000,position=2,optional=false,autoIncremented=false,generated=false}, Struct{name=ACT_VALUE,jdbcType=12,typeName=VARCHAR2,typeExpression=VARCHAR2,length=4000,position=3,optional=true,autoIncremented=false,generated=false}, Struct{name=NEW_COLUMN,jdbcType=12,typeName=VARCHAR2,typeExpression=VARCHAR2,length=20,position=4,optional=true,autoIncremented=false,generated=false}]}}]}
Struct{source=Struct
{version=2.3.0.Final,connector=oracle,name=pv_DebeziumPoC,ts_ms=1693533233000,db=MWBIZINT,schema=C##DEBEZUSER,table=DEBEZIUM_TEST_PAVEL,scn=11268870738964,commit_scn=11268870738964,rs_id=0x006eb2.00000a04.0060,ssn=0,redo_thread=2},ts_ms=1693558435668,databaseName=MWBIZINT,schemaName=C##DEBEZUSER,ddl=ALTER TABLE C##DEBEZUSER.DEBEZIUM_TEST_PAVEL ADD PRIMARY KEY ("ID", "NAME");,tableChanges=Struct{type=ALTER,id="MWBIZINT"."C##DEBEZUSER"."DEBEZIUM_TEST_PAVEL",table=Struct{{color:#008000}primaryKeyColumnNames=[ID, NAME],columns=[Struct{name=ID,jdbcType=2,typeName=NUMBER,typeExpression=NUMBER,length=0,position=1,optional=false,autoIncremented=false,generated=false}, Struct{name=NAME,jdbcType=12,typeName=VARCHAR2,typeExpression=VARCHAR2,length=4000,position=2,optional=false,autoIncremented=false,generated=false}, Struct{name=ACT_VALUE,jdbcType=12,typeName=VARCHAR2,typeExpression=VARCHAR2,length=4000,position=3,optional=true,autoIncremented=false,generated=false}, Struct{name=NEW_COLUMN,jdbcType=12,typeName=VARCHAR2,typeExpression=VARCHAR2,length=20,position=4,optional=true,autoIncremented=false,generated=false}]}}]}
Expected behavior
My expectation is that primaryKeyColumnNames behaves consistently, i.e., that with every DDL change it always shows the list of keys reflecting the after state, so the use case with dropping PK should be fixed.