-
Bug
-
Resolution: Done
-
Major
-
1.9.5.Final
-
None
-
False
-
None
-
False
-
Setup Details
Debezium Version: 1.9.5.FINAL
Debezium Runtime: Debezium/Connect Docker container in standalone mode; Deployed on-prem on bare metal, talking to PG on the same machine. Kafka cluster in a managed cloud.
Postgres version: 12:12
Postgres Logical Decoder: pgoutput
Description
I encountered this Debezium exception that I need some help deciphering -
Message with LSN 'LSN{22E3/5E30CDD8}' larger than expected LSN 'LSN{22E3/5E30CAB8}'
The happens right after Debezium connector restarts after a connector failure, where the connector failed to talk to PG. I am attaching the log file that has all the log messages from the connector restart (after a failure), leading up to this exception.
According to the logs,
- the PG confirmed_flush_lsn was 22E3/29F39960 (line 2758 in log)
- the committed LSN in Kafka was 22E3/5E3118F8 (line 2759)
- the last processed LSN in Kafka was 22E3/5E30C0E0 (line 2759)
The error suggests that PG sends LSN 22E3/5E30CDD8 right after connecting (line 2935) and then (presumably) sends 22E3/5E30CAB8 right after that (line 2936). It establishes 22E3/5E30CAB8 as the restart point for streaming because it figures 22E3/5E30CAB8 is the first greater offset after 22E3/5E30C0E0 (the last processed offset according to the Kafka internal topic). However, it then complains that 22E3/5E30CAB is less than 22E3/5E30CDD8. I would appreciate help in understanding why PG would send a second message with an LSN that is lesser than the one it previously sent.
Thanks!
Note: This was filed on advice from the thread started here - https://debezium.zulipchat.com/#narrow/stream/302529-users/topic/Message.20with.20LSN.20foo.20larger.20than.20expected.20LSN.20bar
- relates to
-
DBZ-5567 Producer Failure - This is unexpected and can lead to an infinite loop or a data loss.
- Closed