-
Enhancement
-
Resolution: Duplicate
-
Major
-
None
-
None
-
None
-
False
-
None
-
False
-
-
Current embedded engine architecture mimics Kafka Connect architecture/behaviour.
A batch of Struct messages are received from connector and then serially run through transformations, serialized and delivered.
The transfromation and serialization can be CPU consuming task.
The parallelized engine should have a pool to which the Struct is passed and transformation and serialization is done on many messages in parallel. The messages are then delivered serially to the sink in order.
The follow-up tasks will be
- delivering messages to sinks in parallel when supported
- extending connector architecture in a way the incoming data will be decoded and processed in parallel when possible
- pilot could be PostgreSQL connector - a single thread will read data from replication slot and chunk of bytes representing messages will be processed in parallel
- relates to
-
DBZ-7024 Create new implementation of DebeziumEngine
- Resolved