Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-6685

Partition duplication after rebalances with single leader task

    XMLWordPrintable

Details

    • False
    • None
    • False

    Description

      There can be potential partition duplication after rebalance events when there is only one leader task. in that scenario, the leader will not wait to receive rebalance answer messages before initializing the state to NEW_EPOCH_STARTED. In that event, it can potentially process its own stale rebalance answer message (with the same rebalance generation ID) after initializing the state to NEW_EPOCH_STARTED.

      In this event, we need to avoid processing both rebalance answers and new epoch messages in SyncEventHandler::process. We should also filter out messages produced by the same task Uid inside SyncEventMerger.merge.

      Attachments

        Activity

          People

            Unassigned Unassigned
            nancyxu567@gmail.com Nancy Xu
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: