Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-7698

Provide option to encode ordering metadata in each record

XMLWordPrintable

    • False
    • None
    • False

      Feature request or enhancement

      This came up in Zulip discussion here. The motivation is that when kafka re-partition or other event happens where keys lose their ordering properties (i.e., ordering only guaranteed per partition), then getting consistent & correct data is both manual and error prone.

      The possible solution is to have each message contain a collection of fields on the change event's ordering metadata. With these fields any system can establish for an arbitrary pair of changes for the same primary key which is more recent.

      Which use case/requirement will be addressed by the proposed feature?

      Allow for downstream systems to automatically handle kafka re-partition events.

      Implementation ideas (optional)

      In order to help make the discussion concrete, we created a PR in the debezium-vitess-connector repo that demonstrates how this will look for this connector. However, it can be generalized to all connectors (as long as they provide an implementation of the respective interfaces).

              Unassigned Unassigned
              tthorn Thomas Thornton
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated:
                Resolved: