Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-1136

Describe topic-level settings to ensure event consumption when log compaction is enabled

XMLWordPrintable

    • Icon: Task Task
    • Resolution: Done
    • Icon: Minor Minor
    • 0.9.2.Final
    • None
    • website
    • None

      In order to not lose data intermittently when using log compaction, Kafka topics should be configured with a non-zero min.compaction.lag.ms. The default of 0 appears, in some circumstances, to result in delete events being compacted by their tombstone before the consumer can process the delete.

      This isn't a bug in Debezium as much as it's a undocumented configuration land mine. There isn't another bug filed describing this problem, so I figured I'd put it in here.

              gunnar.morling Gunnar Morling
              rbranson_jira Rick Branson (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Created:
                Updated:
                Resolved: