Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-8973

LogMiner does not always abandon transactions deterministically

XMLWordPrintable

    • False
    • Hide

      None

      Show
      None
    • False

      The test case OracleConnectorIT#testBacklogTransactionShouldNotBeAbandon may fail periodically and indicates transactions may not be abandoned consistently.

      In this example

      Fetching results for SCN [9171554, 9171998]   
      SCN=9171978, TxId=0a001400211c0000: insert into "DEBEZIUM"."DBZ6355"("ID","NAME") values ('2','Minnie Mouse');
      Counters{rows=22, stuckCount=0, dmlCount=1, ddlCount=0, insertCount=1, updateCount=0, deleteCount=0, commitCount=17, rollbackCount=0, tableMetadataCount=0, partialRollbackCount=0}
      Processed in 3 ms. Lag: 15196. Offset SCN: 9171554, Offset Commit SCN: CommitScn [redoThreadCommitScns={1=RedoThreadCommitScn{thread=1, commitScn=9171976, txIds=[090011002d040000]}}], Active Transactions: 1
      

      Then there are multiple data fetches that look like this

      Fetching results for SCN [9171978, 9171998]
      Counters{rows=0, stuckCount=0, dmlCount=0, ddlCount=0, insertCount=0, updateCount=0, deleteCount=0, commitCount=0, rollbackCount=0, tableMetadataCount=0, partialRollbackCount=0}
      Processed in 0 ms. Lag: 15196. Offset SCN: 9171977, Offset Commit SCN: CommitScn [redoThreadCommitScns={1=RedoThreadCommitScn{thread=1, commitScn=9171976, txIds=[090011002d040000]}}], Active Transactions: 1
      

      then eventually

      Fetching results for SCN [9171978, 9172074]
      SCN=9172031, TxId=070005001c030000: insert into "DEBEZIUM"."DBZ6355"("ID","NAME") values ('3','Donald Duck')
      Committing transaction 070005001c030000 with 1 events (scn: 9172032, thread: 1, oldest buffer scn: 9171978)
      Dispatching event SCN 9172031
      Committing transaction 0a001400211c0000 with 1 events (scn: 9172035, thread: 1, oldest buffer scn: null)
      Dispatching event SCN 9171978
      

      Given that transactions are only abandoned after processing a full mining iteration and due to the way the data manifested in the iterations, the transaction 0a001400211c0000 is never abandoned and is instead emitted, despite having aged beyond the configured retention period.

      Beyond the fact that the transaction wasn't abandoned, it also highlights that if there is a transaction in the cache and there are long periods of no data fetched, the cache isn't being flushed, which means that the low watermark could eventually become stale.

              ccranfor@redhat.com Chris Cranford
              ccranfor@redhat.com Chris Cranford
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Created:
                Updated: