Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-9157

Request for mechanism to prevent data loss due to Informix'x log rollover during connector downtime

XMLWordPrintable

    • False
    • Hide

      None

      Show
      None
    • False
    • Important

      Which use case/requirement will be addressed by the proposed feature?

      We experience data loss when Debezium connectors are stopped for extended periods (e.g., 4–5 hours during DDL deployments), and the database’s logical logs roll over during this downtime. Upon restart, connectors fail with "LSN not found" exceptions because the last committed LSN no longer exists. Resetting the LSN causes missed changes and data loss.

      This feature request aims to implement a mechanism that can detect missing change events caused by log rollover and connector downtime, and automatically resynchronize or replay those changes to avoid data loss.

      Implementation ideas:

      • Introduce a delta detection mechanism that compares the last committed LSN with the current log position and identifies missing changes.
      • Provide automatic resynchronization or replay of missed events without manual LSN resets.
      • Possibly leverage a dedicated “recovery” mode or integration with database log retention metadata to safely handle log rollovers.
      • Provide configuration options to control the behavior during connector restarts after downtime.

              Unassigned Unassigned
              aahmad08@i2cinc.com Aqeel Ahmad
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Created:
                Updated: