Details
-
Bug
-
Resolution: Done
-
Major
-
1.9.3.Final
-
None
-
False
-
None
-
False
Description
What Debezium connector do you use and what version?
1.9.3 and Oracle
What is the connector configuration?
"log.mining.strategy": "online_catalog"
"database.dbname" and "database.pdb.name" have different values
What is the captured database version and mode of depoyment?
v19 oracle db, debzium running on k8s pods
What behaviour do you expect?
on deleting old k8s deployment/pods running debezium 1.9.1, deleting ALL kafka topics and then starting k8s deployment/pods running debezium 1.9.3 i expect a fresh snapshot to start
What behaviour do you see?
pod errors out:
[2022-06-04 04:03:41,632] INFO Schema locking was disabled in connector configuration (io.debezium.connector.oracle.OracleSnapshotChangeEventSource) [2022-06-04 04:03:41,632] INFO Snapshot step 4 - Determining snapshot offset (io.debezium.relational.RelationalSnapshotChangeEventSource) [2022-06-04 04:03:41,968] INFO Consulting V$TRANSACTION and transaction logs for resolving snapshot offset. (io.debezium.connector.oracle.logminer.LogMinerAdapter) [2022-06-04 04:03:44,062] INFO Connection gracefully closed (io.debezium.jdbc.JdbcConnection) [2022-06-04 04:03:44,066] INFO Snapshot - Final stage (io.debezium.pipeline.source.AbstractSnapshotChangeEventSource) [2022-06-04 04:03:44,078] ERROR Producer failure (io.debezium.pipeline.ErrorHandler) io.debezium.DebeziumException: io.debezium.DebeziumException: Failed to get transaction id for current SCN 524567374505 at io.debezium.pipeline.source.AbstractSnapshotChangeEventSource.execute(AbstractSnapshotChangeEventSource.java:85) at io.debezium.pipeline.ChangeEventSourceCoordinator.doSnapshot(ChangeEventSourceCoordinator.java:155) at io.debezium.pipeline.ChangeEventSourceCoordinator.executeChangeEventSources(ChangeEventSourceCoordinator.java:137) at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:109) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:829) Caused by: io.debezium.DebeziumException: Failed to get transaction id for current SCN 524567374505 at io.debezium.connector.oracle.logminer.LogMinerAdapter.determineSnapshotOffset(LogMinerAdapter.java:209) at io.debezium.connector.oracle.logminer.LogMinerAdapter.determineSnapshotOffset(LogMinerAdapter.java:124) at io.debezium.connector.oracle.OracleSnapshotChangeEventSource.determineSnapshotOffset(OracleSnapshotChangeEventSource.java:139) at io.debezium.connector.oracle.OracleSnapshotChangeEventSource.determineSnapshotOffset(OracleSnapshotChangeEventSource.java:34) at io.debezium.relational.RelationalSnapshotChangeEventSource.doExecute(RelationalSnapshotChangeEventSource.java:111) at io.debezium.pipeline.source.AbstractSnapshotChangeEventSource.execute(AbstractSnapshotChangeEventSource.java:76) ... 8 more [2022-06-04 04:03:44,080] INFO Connected metrics set to 'false' (io.debezium.pipeline.ChangeEventSourceCoordinator) [2022-06-04 04:03:44,096] INFO WorkerSourceTask{id=kafka-connect-src-01-0} flushing 0 outstanding messages for offset commit (org.apache.kafka.connect.runtime.WorkerSourceTask) [2022-06-04 04:03:44,096] ERROR WorkerSourceTask{id=kafka-connect-src-01-0} Task threw an uncaught and unrecoverable exception. Task is being killed and will not recover until manually restarted (org.apache.kafka.connect.runtime.WorkerTask) org.apache.kafka.connect.errors.ConnectException: An exception occurred in the change event producer. This connector will be stopped. at io.debezium.pipeline.ErrorHandler.setProducerThrowable(ErrorHandler.java:50) at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:116) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:829) Caused by: io.debezium.DebeziumException: io.debezium.DebeziumException: Failed to get transaction id for current SCN 524567374505 at io.debezium.pipeline.source.AbstractSnapshotChangeEventSource.execute(AbstractSnapshotChangeEventSource.java:85) at io.debezium.pipeline.ChangeEventSourceCoordinator.doSnapshot(ChangeEventSourceCoordinator.java:155) at io.debezium.pipeline.ChangeEventSourceCoordinator.executeChangeEventSources(ChangeEventSourceCoordinator.java:137) at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:109) ... 5 more Caused by: io.debezium.DebeziumException: Failed to get transaction id for current SCN 524567374505 at io.debezium.connector.oracle.logminer.LogMinerAdapter.determineSnapshotOffset(LogMinerAdapter.java:209) at io.debezium.connector.oracle.logminer.LogMinerAdapter.determineSnapshotOffset(LogMinerAdapter.java:124) at io.debezium.connector.oracle.OracleSnapshotChangeEventSource.determineSnapshotOffset(OracleSnapshotChangeEventSource.java:139) at io.debezium.connector.oracle.OracleSnapshotChangeEventSource.determineSnapshotOffset(OracleSnapshotChangeEventSource.java:34) at io.debezium.relational.RelationalSnapshotChangeEventSource.doExecute(RelationalSnapshotChangeEventSource.java:111) at io.debezium.pipeline.source.AbstractSnapshotChangeEventSource.execute(AbstractSnapshotChangeEventSource.java:76) ... 8 more [2022-06-04 04:03:44,098] INFO Stopping down connector (io.debezium.connector.common.BaseSourceTask)
then i tried to run an adhoc query against oracle:
i saw
select * from V$TRANSACTION
returned the SCN that the logs mention but
SELECT count(1) FROM V$LOGMNR_CONTENTS
gave error ORA-01306: dbms_logmnr.start_logmnr() must be invoked before selecting from v$logmnr_contents
now as a workaround i set "log.mining.query.logs.for.snapshot.offset": "false" and seemed to have past that error
i guess this error is introduced by DBZ-5085, my follow-up Q: does v1.9.3 with "log.mining.query.logs.for.snapshot.offset": "false" have more chance of missing records than just using v1.9.1 ?
