Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-69

Optionally partition events using foreign keys

    XMLWordPrintable

Details

    • Enhancement
    • Resolution: Won't Do
    • Major
    • None
    • 0.3
    • mysql-connector

    Description

      Debezium emits only row-level change events. When an entity persisted in multiple tables with foreign keys, any single (logical) change to an entity will result in one or more row-level events in the corresponding topics for the tables. Debezium currently uses no partitioning logic, and as such Kafka Connect's producer will use its built-in partitioner to partition based upon the event's key. This works great for tables that use the primary key of the entity, but the key for events on a table that has a foreign key to the entity will likely be partitioned differently, as the key is based upon the key of the table, not the foreign key.

      Debezium should look for such tables and, when configured, use a message key that is based upon the foreign key. Can this be enabled at the connector level to apply to all such tables, or is it required to specify which tables are emitted this way. And what about chained multiple tables, where C has a foreign key to B, and B has a foreign key to A? How should row-level events in C be keyed?

      Attachments

        Activity

          People

            Unassigned Unassigned
            rhauch Randall Hauch (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: