Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-4737

Oracle Logminer: streaming start offset is off by one

XMLWordPrintable

    • False
    • False
    • Hide
      • start long-running TX
      • drop all archive and redo logs
      • start Debezium
      • wait for snapshot phase to be complete

      At the first invocation of the LogminerStreamingChangeEventSource, the rewind logic will kick in, and rewind the startSCN to firstSCN-1. The actual mining logic will conclude that startSCN < firstSCN, and complain that you need to take a new snapshot.

      Show
      start long-running TX drop all archive and redo logs start Debezium wait for snapshot phase to be complete At the first invocation of the LogminerStreamingChangeEventSource, the rewind logic will kick in, and rewind the startSCN to firstSCN-1. The actual mining logic will conclude that startSCN < firstSCN, and complain that you need to take a new snapshot.

      DBZ-4367 introduced "rewind" functionality that changes the initial mining offset during the streaming phase when there were pending transactions during the taking of the initial snapshot. Debezium now starts mining from the start of the first such transaction, to capture changes with SCN < snapshotSCN that were only committed with commitSCN > snapshotSCN.

      If the start of the oldest pending transaction lies before the oldest SCN still available in the archive logs, mining is supposed to start from the oldest SCN still in the logs.

      Due to an off-by-one error in that logic, the Logminer streaming change event source attempts to start mining at (oldestSCN-1) instead of oldestSCN, which causes the connector the throw an exception.

              Unassigned Unassigned
              dominique.chanet@klarrio.com Dominique Chanet (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Created:
                Updated:
                Resolved: