-
Bug
-
Resolution: Done
-
Major
-
2.2.1.Final
-
None
Bug report
I have a PostgreSQL cluster, where some databases are being replicated by Debezium, I recently noticed that in some cases where Debezium is restarted for some reason and there is no lag in the replication slot, the heartbeat.action.query just starts to be triggered after some write happens in the database replicated by Debezium.
For the heartbeat to work again, I have to update the database or send a log signal to Debezium.
What Debezium connector do you use and what version?
2.2.1.Final
What is the connector configuration?
{
"name": "${connectName}",
"config": {
"plugin.name": "pgoutput",
"tasks.max": 1,
"connector.class": "io.debezium.connector.postgresql.PostgresConnector",
"database.hostname": "${databaseHost}",
"database.port": "${databasePort}",
"database.user": "${databaseUser}",
"database.password": "${databasePass}",
"database.dbname": "${databaseName}",
"topic.prefix": "nucleo",
"snapshot.mode": "${snapshotMode}",
"snapshot.custom.class": "...",
"heartbeat.interval.ms": 10000,
"heartbeat.action.query": "INSERT INTO debezium_heartbeat_table (id, time) VALUES (1, now()) ON CONFLICT (id) DO UPDATE SET time = now();",
"slot.name": "${slotName}",
"table.include.list": "${tableIncludeList}",
"decimal.handling.mode": "string",
"time.precision.mode": "connect",
"signal.data.collection": "public.debezium_signals"
}
What is the captured database version and mode of depoyment?
PostgreSQL 11/13
Do you have the connector logs, ideally from start till finish?
Log after detecting the problem, there is an alert in Grafana that notifies when the heartbeat stops working, in this case I just sent a debezium signal with the log `Log sent via nucleo-stream`
[2023-07-10 14:29:01,593] INFO [nucleo-a|task-0|offsets] WorkerSourceTask
{id=nucleo-nevesmd-0} flushing 0 outstanding messages for offset commit (org.apache.kafka.connect.runtime.WorkerSourceTask:510)
[2023-07-10 14:30:01,595] INFO [nucleo-a|task-0|offsets] WorkerSourceTask{id=nucleo-nevesmd-0}
flushing 0 outstanding messages for offset commit (org.apache.kafka.connect.runtime.WorkerSourceTask:510)
[2023-07-10 14:31:01,597] INFO [nucleo-a|task-0|offsets] WorkerSourceTask
{id=nucleo-nevesmd-0} flushing 0 outstanding messages for offset commit (org.apache.kafka.connect.runtime.WorkerSourceTask:510)
[2023-07-10 14:31:30,671] INFO [nucleo-a|task-0] First LSN 'LSN{5779/EE7FEEF0}' received (io.debezium.connector.postgresql.connection.WalPositionLocator:71)
[2023-07-10 14:31:30,836] INFO [nucleo-a|task-0] Received COMMIT LSN 'LSN{5779/EE801858}' larger than than last stored commit LSN 'LSN{5779/3BB60D28}' (io.debezium.connector.postgresql.connection.WalPositionLocator:108)
[2023-07-10 14:31:30,836] INFO [nucleo-a|task-0] Will restart from LSN 'LSN{5779/EE7FEEF0}' that is start of the first unprocessed transaction (io.debezium.connector.postgresql.connection.WalPositionLocator:120)
[2023-07-10 14:31:30,836] INFO [nucleo-a|task-0] WAL resume position 'LSN{5779/EE7FEEF0}' discovered (io.debezium.connector.postgresql.PostgresStreamingChangeEventSource:348)
[2023-07-10 14:31:30,837] INFO [nucleo-a|task-0] Connection gracefully closed (io.debezium.jdbc.JdbcConnection:947)
[2023-07-10 14:31:30,838] INFO [nucleo-a|task-0] Connection gracefully closed (io.debezium.jdbc.JdbcConnection:947)
[2023-07-10 14:31:30,843] INFO [nucleo-a|task-0] Initializing PgOutput logical decoder publication (io.debezium.connector.postgresql.connection.PostgresReplicationConnection:150)
[2023-07-10 14:31:30,847] INFO [nucleo-a|task-0] Seeking to LSN{5779/3BB60D28} on the replication slot with command SELECT pg_replication_slot_advance('debezium_nucleo_nevesmd', '5779/3BB60D28') (io.debezium.connector.postgresql.connection.PostgresReplicationConnection:366)
[2023-07-10 14:31:32,426] INFO [nucleo-a|task-0] Requested thread factory for connector PostgresConnector, id = zeroglosanucleo named = keep-alive (io.debezium.util.Threads:270)
[2023-07-10 14:31:32,427] INFO [nucleo-a|task-0] Creating thread debezium-postgresconnector-zeroglosanucleo-keep-alive (io.debezium.util.Threads:287)
[2023-07-10 14:31:32,427] INFO [nucleo-a|task-0] Processing messages (io.debezium.connector.postgresql.PostgresStreamingChangeEventSource:211)
[2023-07-10 14:31:32,777] INFO [nucleo-a|task-0] 762 records sent during previous 03:19:48.55, last recorded offset of {server=zeroglosanucleo} partition is {transaction_id=null, lsn_proc=96178204445992, messageType=UPDATE, lsn_commit=96178204445992, lsn=96178204445992, txId=2971633362, ts_usec=1688991349800695} (io.debezium.connector.common.BaseSourceTask:195)
[2023-07-10 14:31:48,624] INFO [nucleo-a|task-0] Message with LSN 'LSN{5779/EE7FEEF0}' arrived, switching off the filtering (io.debezium.connector.postgresql.connection.WalPositionLocator:152)
[2023-07-10 14:31:48,656] INFO [nucleo-a|task-0] Log enviado via nucleo-stream (io.debezium.pipeline.signal.Log:28)
[2023-07-10 14:32:01,598] INFO [nucleo-a|task-0|offsets] WorkerSourceTask{id=nucleo-nevesmd-0}
flushing 0 outstanding messages for offset commit (org.apache.kafka.connect.runtime.WorkerSourceTask:510)
- links to
-
RHEA-2024:129636
Red Hat build of Debezium 2.5.4 release
