Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-6685

Partition duplication after rebalances with single leader task

    XMLWordPrintable

Details

    • False
    • None
    • False

    Description

      There can be potential partition duplication after rebalance events when there is only one leader task. in that scenario, the leader will not wait to receive rebalance answer messages before initializing the state to NEW_EPOCH_STARTED. In that event, it can potentially process its own stale rebalance answer message (with the same rebalance generation ID) after initializing the state to NEW_EPOCH_STARTED.

      In this event, we need to avoid processing both rebalance answers and new epoch messages in SyncEventHandler::process. We should also filter out messages produced by the same task Uid inside SyncEventMerger.merge.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              nancyxu567@gmail.com Nancy Xu
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: