-
Feature Request
-
Resolution: Done
-
Major
-
None
-
None
-
False
-
None
-
False
Feature request or enhancement
This came up in Zulip discussion here. The motivation is that when kafka re-partition or other event happens where keys lose their ordering properties (i.e., ordering only guaranteed per partition), then getting consistent & correct data is both manual and error prone.
The possible solution is to have each message contain a collection of fields on the change event's ordering metadata. With these fields any system can establish for an arbitrary pair of changes for the same primary key which is more recent.
Which use case/requirement will be addressed by the proposed feature?
Allow for downstream systems to automatically handle kafka re-partition events.
Implementation ideas (optional)
In order to help make the discussion concrete, we created a PR in the debezium-vitess-connector repo that demonstrates how this will look for this connector. However, it can be generalized to all connectors (as long as they provide an implementation of the respective interfaces).