Setup Details

Debezium Version: 1.9.5.FINAL

Debezium Runtime: Debezium/Connect Docker container in standalone mode; Deployed on-prem on bare metal, talking to PG on the same machine. Kafka cluster in a managed cloud.

Postgres version: 12:12

Postgres Logical Decoder: pgoutput

Description

I encountered this Debezium exception that I need some help deciphering -
 

Message with LSN 'LSN{22E3/5E30CDD8}' larger than expected LSN 'LSN{22E3/5E30CAB8}'

The happens right after Debezium connector restarts after a connector failure, where the connector failed to talk to PG. I am attaching the log file that has all the log messages from the connector restart (after a failure), leading up to this exception.

According to the logs,

• the PG confirmed_flush_lsn was 22E3/29F39960 (line 2758 in log)
• the committed LSN in Kafka was 22E3/5E3118F8 (line 2759)
• the last processed LSN in Kafka was 22E3/5E30C0E0 (line 2759)

The error suggests that PG sends LSN 22E3/5E30CDD8 right after connecting (line 2935) and then (presumably) sends 22E3/5E30CAB8 right after that (line 2936). It establishes 22E3/5E30CAB8 as the restart point for streaming because it figures 22E3/5E30CAB8 is the first greater offset after 22E3/5E30C0E0 (the last processed offset according to the Kafka internal topic). However, it then complains that 22E3/5E30CAB is less than 22E3/5E30CDD8. I would appreciate help in understanding why PG would send a second message with an LSN that is lesser than the one it previously sent.

Thanks!

Note: This was filed on advice from the thread started here - https://debezium.zulipchat.com/#narrow/stream/302529-users/topic/Message.20with.20LSN.20foo.20larger.20than.20expected.20LSN.20bar