Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-5283

Adding new option for "ExtractNewRecordState" SMT to exclude unchanged fields

    XMLWordPrintable

Details

    • Enhancement
    • Resolution: Done
    • Optional
    • 2.2.0.Alpha2
    • None
    • core-library
    • None

    Description

      Feature request or enhancement

      Which use case/requirement will be addressed by the proposed feature?

      Currently the built-in SMT "io.debezium.transforms.ExtractNewRecordState" provides "payload.after" content as flattened result for other Kafka connect plugins to use.

       

      However, sometimes it is useful to get only changed columns and optionally primary/unique key columns in a flattened record for further data handling.

       

      It might be great if "ExtractNewRecordState" can optionally compare "payload.before" / "payload.after" fields/values and only keeping changed after-state fields to compose the result flattened record.

       

      Implementation ideas (optional)

      Such feature may be implemented by providing the following properties for "ExtractNewRecordState".

      • "drop.unchanged.fields" (boolean) : to keep only changed fields in the resulting flattened record
      • "null.unchanged.fields" (boolean) : (optional) provides JSON NULL value for all unchanged fields in the resulting flattened record
      • "include.primary.keys" (boolean) : to optionally keep primary/unique key fields in the resulting record no matter the previous setting is present or not.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              chaiachun Chia-Chun Hsu
              Votes:
              2 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: