Details
-
Feature Request
-
Resolution: Unresolved
-
Major
-
None
-
None
-
False
-
False
-
0
-
0%
Description
Hi folks, my team is using Outbox Event Router to do transactional Kafka publishing from PostgreSQL dbs. Due to legacy reasons (headers not supported in a previous connector version), we are currently using a payload envelope to send event metadata, but now that they are supported, we would like to switch to using Kafka message headers for this metadata. I see that you can configure the connector via table.fields.additional.placement, however this strategy requires us to add one table column per header we would like to produce. This means that any time we'd like to make a change to event metadata (e.g. add a new field), we need to perform a db migration followed by a config update for the connector for each service that is producing transactional events.
Has there been any thought about adding config for a single JSON "headers" column, which would automatically turn whatever contents in the JSON object into Kafka message headers? This way we can simply update producers to add/change/remove headers from that column, and consumers automatically see the changes propagate to their kafka message headers, without the need for any database or connector changes. I'm imagining optional configuration like table.field.event.headers which points to a VARCHAR or JSON typed column.