-
Bug
-
Resolution: Unresolved
-
Major
-
None
-
None
-
False
-
-
False
In order to make your issue reports as actionable as possible, please provide the following information, depending on the issue type.
Bug report
For bug reports, provide this information, please:
What Debezium connector do you use and what version?
Oracle connector running on Debezium 3.2.1
What is the connector configuration?
~140 tables in the include list, here's the LogMiner configuration. A heartbeat is running every minute on an externally scheduled Cron job:
"internal.log.mining.use.cte.query": "true", "query.fetch.size": "50000", "log.mining.batch.size.increment": "50000", "log.mining.batch.size.max": "100000",
What is the captured database version and mode of deployment?
(E.g. on-premises, with a specific cloud provider, etc.)
On premise Oracle 11g
What behavior do you expect?
LogMiner query times comparable to single-pass version of the query when there are 0 changes the connector is interested in in the batch
What behavior do you see?
LogMiner query time increase despite queries returning 0 rows.
This is during a period of heavy batch job activity, made up of large transactions potentially producing millions of rows. Other connectors are deployed against the same Oracle instance and the LogMiner query times are significantly lower. Here are some stats across this period that might be useful, taken from:
#community-oracle > Performance degradation after connector setup change @ 💬
Max logs mined
Orange is the CTE connector
LogMiner query performance
Purple is LastDurationOfFetchQueryInMilliseconds, light blue is LogSwitchCount, magenta is MaximumMinedLogCount
OffsetScn over time
Orange is the CTE connector, other lines are not using the CTE
LagFromSourceInMilliseconds
Orange is the CTE connector, converted to seconds
TotalProcessedRows
Orange is the CTE connector
Do you see the same behaviour using the latest released Debezium version?
(Ideally, also verify with latest Alpha/Beta/CR version)
We've only tested 3.2.1 for this one
Do you have the connector logs, ideally from start till finish?
(You might be asked later to provide DEBUG/TRACE level log)
Unfortunately not!
How to reproduce the issue using our tutorial deployment?
Potentially possible by enabling CTE queries in the Oracle connector, generating a lot of changes in tables outside of the include list and monitoring the impact on LastDurationOfFetchQueryInMilliseconds in two connectors - One using the CTE, one not using the CTE