Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-4265

Support single header column for Outbox Event Router

    XMLWordPrintable

Details

    • Feature Request
    • Resolution: Unresolved
    • Major
    • 1.9-backlog
    • None
    • outbox
    • None
    • False
    • False
    • 0
    • 0% 0%

    Description

      Hi folks, my team is using Outbox Event Router to do transactional Kafka publishing from PostgreSQL dbs. Due to legacy reasons (headers not supported in a previous connector version), we are currently using a payload envelope to send event metadata, but now that they are supported, we would like to switch to using Kafka message headers for this metadata. I see that you can configure the connector via table.fields.additional.placement, however this strategy requires us to add one table column per header we would like to produce. This means that any time we'd like to make a change to event metadata (e.g. add a new field), we need to perform a db migration followed by a config update for the connector for each service that is producing transactional events.

      Has there been any thought about adding config for a single JSON "headers" column, which would automatically turn whatever contents in the JSON object into Kafka message headers? This way we can simply update producers to add/change/remove headers from that column, and consumers automatically see the changes propagate to their kafka message headers, without the need for any database or connector changes. I'm imagining optional configuration like table.field.event.headers which points to a VARCHAR or JSON typed column.

      Attachments

        Activity

          People

            Unassigned Unassigned
            chris-convoy Chris Ellis (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated: