Details
-
Bug
-
Resolution: Done
-
Critical
-
2.3.0.Final, 2.4.0.Final
-
None
Description
In order to make your issue reports as actionable as possible, please provide the following information, depending on the issue type.
Bug report
For bug reports, provide this information, please:
What Debezium connector do you use and what version?
Oracle Connector, tested with versions 2.3 and 2.4, running on Debezium Server.
What is the connector configuration?
debezium.source.connector.class=io.debezium.connector.oracle.OracleConnector debezium.source.database.hostname=$HOST debezium.source.database.port=1521 debezium.source.database.dbname=$SERVICE_NAME debezium.source.table.include.list=OWNER.TABLE1,OWNER.TABLE3,(...),OWNER.TABLE132 debezium.source.offset.storage.file.filename=data/offsets.dat debezium.source.schema.history.internal.store.only.captured.tables.ddl=true debezium.source.schema.history.internal=io.debezium.storage.file.history.FileSchemaHistory debezium.source.schema.history.internal.file.filename=data/schema_history.dat debezium.source.log.mining.strategy=online_catalog debezium.source.log.mining.batch.size.min=10000 debezium.source.log.mining.batch.size.max=500000 debezium.source.log.mining.sleep.time.default.ms=600 debezium.source.log.mining.transaction.retention.ms=79200000 debezium.source.log.mining.archive.log.only.mode=false ;(tested both true and false here) debezium.source.query.fetch.size=20000 debezium.source.tombstones.on.delete=false debezium.source.snapshot.mode=schema_only debezium.source.decimal.handling.mode=string debezium.source.key.converter.schemas.enable=false debezium.source.value.converter.schemas.enable=false debezium.source.topic.prefix=oracle-planck-ingestion debezium.transforms.Reroute.topic.replacement=oracle-planck-ingestion debezium.transforms.Reroute.type=io.debezium.transforms.ByLogicalTableRouter debezium.transforms.Reroute.topic.regex=.* debezium.transforms=Reroute quarkus.log.level=DEBUG quarkus.log.console.json=false
What is the captured database version and mode of depoyment?
Oracle 19c (19.19), deployed on OCI, EXACS
What behaviour do you expect?
The connector should process every operation occurred inside each transaction, including commit/rollback operations, detecting when the transaction is over and moving the offset SCN accordingly.
What behaviour do you see?
The connector seems to eventually miss the operations that end some transactions (even short-living ones), thus considering them to be active ad aeternum.
When this occurs, the start SCN ("scn" key) in the offset is never updated. This leads to missing data produced by such transactions and also, in the worst case scenario, causes loss of track of database archived logs upon connector restart.
In this particular case I could reproduce (logs attached) the offset got stuck with scn 10386088416326 (2023-09-18 23:17:07 UTC), from 2023-09-19 00:16:49 UTC until connector shutdown (2023-09-19 13:58:42).
We can see on the logs that this scn immediately precedes the transaction 22001200cbbc0400 (scn 10386088416327), which the connector considers to be active during the whole time. However, using the logminer to query the corresponding archived logs, we can see this was not a long transaction, rather it's been committed almost immediately:
XID SCN OPERATION ROLLBACK TS ---------------- -------------------- --------------- ---------- -------------------- 22001200CBBC0400 10386088416327 START 0 2023-09-18 23:17:09 22001200CBBC0400 10386088416327 UPDATE 0 2023-09-18 23:17:09 22001200CBBC0400 10386088416328 INSERT 0 2023-09-18 23:17:09 22001200CBBC0400 10386088416330 UPDATE 0 2023-09-18 23:17:09 22001200CBBC0400 10386088416330 INSERT 0 2023-09-18 23:17:09 22001200CBBC0400 10386088416331 UPDATE 0 2023-09-18 23:17:09 22001200CBBC0400 10386088416341 INSERT 0 2023-09-18 23:17:09 22001200CBBC0400 10386088416341 UPDATE 0 2023-09-18 23:17:09 22001200CBBC0400 10386088416342 UPDATE 0 2023-09-18 23:17:09 22001200CBBC0400 10386088416345 COMMIT 0 2023-09-18 23:17:09
Do you see the same behaviour using the latest relesead Debezium version?
Yes, tested with Debezium Server image version 2.4 (https://hub.docker.com/layers/debezium/server/2.4/images/sha256-5c14be289d968a7657d9cbe87bfd456d22ac0cb8be6a473cafe8614c6ea59ad5?context=explore)
Do you have the connector logs, ideally from start till finish?
(You might be asked later to provide DEBUG/TRACE level log)
DEBUG level logs attached, with some repetitive pattern filters applied to avoid excessive noise:
grep -v "checking for more records" logs.txt | grep -v "no records available" | grep -v "polling records" | grep -v "Embedded engine" | grep -v "Received no records" | grep -v "Applying topic name transformation" | grep -v "records to topic oracle-" | grep -v "pubsub.googleapis.com" > logs_clean.txt
If you need the unfiltered logs, please let me know.
Attachments
Issue Links
- incorporates
-
-
- Closed
-
-
-
- Closed
-
-
-
- Closed
-
- relates to
-
-
- Closed
-
- links to
-
RHBA-2024:126962
Red Hat build of Debezium 2.3.7 release
