Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-1453

Provide means of controlling the topic partition where events sent to

    XMLWordPrintable

Details

    • Feature Request
    • Status: Open (View Workflow)
    • Major
    • Resolution: Unresolved
    • None
    • Backlog
    • core-library
    • None

    Description

      There should be a new connector option that takes the column(s) which should impact the partition, applying the same hashing strategy that's applied to the record key. E.g. given there are two tables:

      PURCHASE_ORDER(ID, DATE, STATUS)
      ORDER_LINE(ID, PURCHASE_ORDER_ID, ITEM, PRICE)
      

      If there's a configuration given message.partition.columns=inventory.order_line:purchase_order_id, then the same partition selection strategy should be applied for both tables. In particular, in a distributed Kafka Streams application, the corresponding messages from both change data topics should be handled by one and the same application node.

      A current work-around is to use a custom SMT, as described here.

      This feature should be beneficial for table-to-table joins on non message keys, as enabled in AK 2.4 by KIP-213 ("Support non-key joining in KTable").

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              gmorling@redhat.com Gunnar Morling
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated: