Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-3089

Improve starting LogMiner session & result-set fetch time

    XMLWordPrintable

Details

    • Enhancement
    • Resolution: Unresolved
    • Major
    • None
    • None
    • oracle-connector
    • None
    • False
    • False
    • Undefined

    Description

      There were 3 areas of performance visible with the LogMiner implementation. The first was the time spent parsing the DML operations, captured in DBZ-3078. The latter two were the performance impact of starting the LogMiner session and fetching the results using the default configuration options.

      The LogMiner session takes upwards of 30 seconds on my local machine in order to initiate the session and this is because the data dictionary must be read by Oracle, iterating and building this on the fly based on all the assigned redo-logs to the session. Anytime the session must end and a new session started with a new set of redo logs, this performance impact is taken. The default strategy to use the redo logs as the source of this dictionary is what influences this behavior. There is a log mining strategy, online_catalog to use the online dictionary from the database which reduces this startup time to ~2-3 seconds, however, there is the trade-off that DML operations will be reconstructed based on the current structure of the table and not the structure the table had at the time the event occurred. For environments where DDL changes don't happen often or at all, using the online_catalog strategy is the right choice; however, for environments that can't use this option is there a way to expedite this initialization in Oracle?

      Secondly, fetching results from V$LOGMNR_CONTENTS is slow, this is because the table is populated as the redo logs are read sequentially, which means Disk I/O impacts the speed at which the database returns these results. Is there a way in which these results can be read faster, Oracle configurable option smaller SCN window, or using a real logical table and perhaps a backward thread that feeds the data from this view into the logical table to avoid any processing latency?

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              ccranfor@redhat.com Chris Cranford
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated: