-
Feature Request
-
Resolution: Done
-
Major
-
None
-
None
-
False
-
None
-
False
I am using Debezium Server version 3.0.4 Final, connected to a PostgreSQL and producing events using Outbox Event Router to Pulsar. In my specific use case, I would like to use the KeyShared subscription with batching in my consumers.
According to the Pulsar documentation, key-based batching needs to be enabled on the producer side. The producer config should look like this:
Producer<byte[]> producer = client.newProducer() .topic("my-topic") .batcherBuilder(BatcherBuilder.KEY_BASED) .create();
The documentation also highlights the requirement for enabling KEY_BASED batching for compatibility with KeyShared subscriptions: https://pulsar.apache.org/docs/2.10.x/client-libraries-java/#key_shared
However, I couldn't find any way to configure the Debezium Pulsar producer to enable key-based batching. Looking at the Debezium codebase: https://github.com/debezium/debezium-server/blob/v3.0.6.Final/debezium-server-pulsar/src/main/java/io/debezium/server/pulsar/PulsarChangeConsumer.java#L111-L122
It seems that the default batcher builder is being used for the producer. Am I overlooking something? Is there currently a way to enable key-based batching for the Pulsar producer in Debezium server, or is this a limitation?