Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-3698

PoC for adding transformations / SMT steps to the Debezium UI


      Create a PoC that adds another Debezium UI wizard page/step (or multiple steps/pages) to configure Kafka Connect transformations, aka SMTs - Single Message Transforms.
      SMTs provide capabilities to apply transformations like adding, dropping or aggregating field data, renaming fields, masking data, filter out messages that should not go to Kafka, etc.

      Here's a writeup by Gunnar about SMTs on his blog: https://www.morling.dev/blog/single-message-transforms-swiss-army-knife-of-kafka-connect/

      Confluent gives a good overview of the most common SMTs that come shipped with every Kafka distribution:

      Some advanced SMTs by Debezium in the docs:

      Please decide on 2-3 transformations (choose basic ones like filtering, replace field, etc) that should be implemented as a PoC, sketch a UI for it and let's discuss that with UX and finally implement a PoC to add transformations to the Debezium UI. It should be possible to use a single transformation or multiple and it should be possible to order them (which one gets called first, which next, which last).
      The UI does not necessarily have to return the exact Kafka Connect config format. An abstract format that can be converted to the Kafka Connect config is sufficient (as you see fit).

        1. image-2021-07-22-15-59-47-759.png
          15 kB
          Indra Shukla
        2. image-2021-07-22-16-00-33-348.png
          28 kB
          Indra Shukla
        3. image-2021-07-26-16-20-42-409.png
          100 kB
          Indra Shukla
        4. recording.webm
          8.68 MB
          Indra Shukla
        5. Screenshot from 2021-08-20 14-28-03.png
          69 kB
          Indra Shukla

            ishukla Indra Shukla
            rk3rn3r René Kerner
            0 Vote for this issue
            5 Start watching this issue