-
Bug
-
Resolution: Unresolved
-
Major
-
None
-
2.3.6.GA
-
None
-
False
-
False
-
Undefined
-
Using vertx-spring-boot-starter-kafka, when using seekToBeginning when partition is assigned, the consumer sometimes reads the same message multiple times.
In my situation I have single topic with 2 messages (message1, message2). Reading it when openshift pod starts up, using following code:
consumer.partitionsAssignedHandler(partitions -> { Flux.fromIterable(partitions) .flatMap(consumer::seekToBeginning) // Time out in case this takes too long .take(Duration.ofSeconds(2)) .subscribe(); }); consumer.subscribe(TOPIC_NAME).block(); disposableConsumer = consumer.flux() .subscribe(message -> { System.out.println("Received message: " + message); messages.add(message); });
This code should reset partition to the beginning and then read all messages in the topic.
Sometimes it reads message 2 and then message 1 and message 2 (getting 3 messages in total, although there are only 2 in total).
Attaching full log from the pod. But it seems the kafka consumer works like:
- Connect to topic
- read some messages
- seekToBeginning
- read all messages again
It should read the messages only once, and that's after it seekToBeginning is finished.