Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-1136

Describe topic-level settings to ensure event consumption when log compaction is enabled

    XMLWordPrintable

Details

    • Task
    • Status: Closed (View Workflow)
    • Minor
    • Resolution: Done
    • None
    • 0.9.2.Final
    • website
    • None

    Description

      In order to not lose data intermittently when using log compaction, Kafka topics should be configured with a non-zero min.compaction.lag.ms. The default of 0 appears, in some circumstances, to result in delete events being compacted by their tombstone before the consumer can process the delete.

      This isn't a bug in Debezium as much as it's a undocumented configuration land mine. There isn't another bug filed describing this problem, so I figured I'd put it in here.

      Attachments

        Activity

          People

            gmorling@redhat.com Gunnar Morling
            rbranson_jira Rick Branson (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: