-
Feature Request
-
Resolution: Done
-
Minor
-
3.0.0.Alpha1
-
None
-
False
-
None
-
False
Feature request or enhancement
Debezium doesn't pass the record headers to the converter so that they can be manipulated or used as part of the converter
Source 1 - ConverterBuilder
Source 2 - Kafka Connect Converter
We can pass the headers from the record.headers but they need to be casted from connect.header.Headers -> common.header.Headers.
Which use case/requirement will be addressed by the proposed feature?
Internally we rely on these headers in the value converter to get around the max message size of Kafka.
Essentially we can modify the value structure and pass information in the Headers but they need to be passed through in order to then get merged into the payload by any header converter.
Implementation ideas (optional)
Example of what might need to be done to make this a reality: https://github.com/debezium/debezium/compare/main...ryanvanhuuksloot:debezium:rvh.example-record-headers
Note: it is ugly and untested but hopefully it is demonstrative
I was trying to think of a smart way to avoid a double for loop but I can't see any way because of the double cast from org.apache.kafka.connect.header.Headers -> org.apache.kafka.common.header.Headers -> List<io.debezium.engine.Header>. You can't just pass in empty headers into the key/value converter because it is possible users may want access to the headers for read purposes.