Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-1136

Describe topic-level settings to ensure event consumption when log compaction is enabled

    Details

    • Type: Task
    • Status: Closed (View Workflow)
    • Priority: Minor
    • Resolution: Done
    • Affects Version/s: None
    • Fix Version/s: 0.9.2.Final
    • Component/s: website
    • Labels:
      None

      Description

      In order to not lose data intermittently when using log compaction, Kafka topics should be configured with a non-zero min.compaction.lag.ms. The default of 0 appears, in some circumstances, to result in delete events being compacted by their tombstone before the consumer can process the delete.

      This isn't a bug in Debezium as much as it's a undocumented configuration land mine. There isn't another bug filed describing this problem, so I figured I'd put it in here.

        Gliffy Diagrams

          Attachments

            Activity

              People

              • Assignee:
                gunnar.morling Gunnar Morling
                Reporter:
                rbranson Rick Branson
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: