Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-8783

Multi-threaded single table snapshots for Oracle

XMLWordPrintable

    • Icon: Feature Request Feature Request
    • Resolution: Unresolved
    • Icon: Major Major
    • 3.2-plan
    • None
    • oracle-connector
    • None
    • False
    • Hide

      None

      Show
      None
    • False

      An Oracle initial snapshot is performed by applying a Flashback Query, which uses the AS OF SCN clause to reconstruct the table's data at a specific time. We can use this to our advantage to support multi-threaded captures of a single table.

      I propose we alter the behavior for Oracle to extend the createDataEvents methods inside the RelationalSnapshotChangeEventSource as follows:

      1. Execute a select to gather the row count for the table based on AS OF SCN
      2. Generate a number of "chunks" based on some snapshot.chunk.size property by wrapping the snapshot select with a chunk-based query
      3. Submit tasks using the chunk-based query rather than the raw snapshot select so that we can snapshot a single table across multiple threads.

              Unassigned Unassigned
              ccranfor@redhat.com Chris Cranford
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Created:
                Updated: