Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-3752

Implement a LogMiner event buffer SPI

    XMLWordPrintable

Details

    Description

      It would be useful to introduce an SPI that allows events read from LogMiner to be managed in different ways such as:

      • An in-memory buffer like what the connector currently uses
      • A JCache backed buffer that can be persisted
      • A JDBC backed buffer that can be persisted in the local Oracle database or a remote database (followed up by DBZ-3798)

      The LogMiner event buffer SPI is meant to address a few concerns:

      • Allow advancing the SCN values in offsets more regularly
      • Handle long-running transactions without memory overhead when using persist backed buffer solutions
      • Allow resuming the connector with SCN offsets that are closer to real-time when events are persisted via the buffer
      • Allows using certain techniques to perform bulk operations more easily on data, i.e. JDBC backed buffers would allow bulk record filtering & processing

      Attachments

        Issue Links

          Activity

            People

              ccranfor@redhat.com Chris Cranford
              ccranfor@redhat.com Chris Cranford
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: