Details
-
Enhancement
-
Resolution: Unresolved
-
Major
-
None
-
False
-
False
-
Undefined
Description
Following up with jpechane suggestions in Debezium's chat, I'm creating this enhancement to see if the team could come up with a demo that shows how different changes in multiple tables could be aggregated by transaction id at the consumer side.
Today, Debezium sends transactional information to a transactions topic that can be used to detect when transactions start and end. Debezium also, can enhance events with transactional data. At the same time, Debezium by default matches changes done in database tables to individual topics. Information in those topics will be consumed by different Kafka consumers in different machines. This is a complex scenario but Debezium and Kafka offer enough building blocks for being able to group changes in those multiple tables in such a way that can be processed at the transactional level.
It would be great to have a demo that shows how this problem can be solved using Debezium and Kafka Streams or other different technology.