Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-9447

Increased LogMiner query latency in CTE query despite returning 0 rows

XMLWordPrintable

    • False
    • Hide

      None

      Show
      None
    • False

      In order to make your issue reports as actionable as possible, please provide the following information, depending on the issue type.

      Bug report

      For bug reports, provide this information, please:

      What Debezium connector do you use and what version?

      Oracle connector running on Debezium 3.2.1

      What is the connector configuration?

      ~140 tables in the include list, here's the LogMiner configuration. A heartbeat is running every minute on an externally scheduled Cron job:

      "internal.log.mining.use.cte.query": "true",
      "query.fetch.size": "50000",
      "log.mining.batch.size.increment": "50000",
      "log.mining.batch.size.max": "100000"

      What is the captured database version and mode of deployment?

      (E.g. on-premises, with a specific cloud provider, etc.)

      On premise Oracle 11g

      What behavior do you expect?

      LogMiner query times comparable to single-pass version of the query when there are 0 changes the connector is interested in in the batch

      What behavior do you see?

      LogMiner query time increase despite queries returning 0 rows.

      This is during a period of heavy batch job activity, made up of large transactions potentially producing millions of rows. Other connectors are deployed against the same Oracle instance and the LogMiner query times are significantly lower. Here are some stats across this period that might be useful, taken from:

      #community-oracle > Performance degradation after connector setup change @ 💬

      Max logs mined

      Orange is the CTE connector

      LogMiner query performance

      Purple is LastDurationOfFetchQueryInMilliseconds, light blue is LogSwitchCount, magenta is MaximumMinedLogCount

      OffsetScn over time

      Orange is the CTE connector, other lines are not using the CTE

      LagFromSourceInMilliseconds

      Orange is the CTE connector, converted to seconds

      TotalProcessedRows

      Orange is the CTE connector

      Do you see the same behaviour using the latest released Debezium version?

      (Ideally, also verify with latest Alpha/Beta/CR version)

      We've only tested 3.2.1 for this one

      Do you have the connector logs, ideally from start till finish?

      (You might be asked later to provide DEBUG/TRACE level log)

      Unfortunately not!

      How to reproduce the issue using our tutorial deployment?

      Potentially possible by enabling CTE queries in the Oracle connector, generating a lot of changes in tables outside of the include list and monitoring the impact on LastDurationOfFetchQueryInMilliseconds in two connectors - One using the CTE, one not using the CTE

        1. image-2025-09-05-16-00-59-084.png
          57 kB
          Paul Roper
        2. image-2025-09-05-16-01-38-211.png
          92 kB
          Paul Roper
        3. image-2025-09-05-16-01-53-328.png
          51 kB
          Paul Roper
        4. image-2025-09-05-16-02-52-256.png
          53 kB
          Paul Roper
        5. image-2025-09-05-16-03-36-257.png
          237 kB
          Paul Roper

              Unassigned Unassigned
              proper2 Paul Roper
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Created:
                Updated: