Details
-
Bug
-
Resolution: Unresolved
-
Major
-
None
-
None
-
False
-
None
-
False
-
Critical
Description
In order to make your issue reports as actionable as possible, please provide the following information, depending on the issue type.
Bug report
For bug reports, provide this information, please:
What Debezium connector do you use and what version?
Debezium connector version : 1.9.5_Final , Postgres connector
What is the connector configuration?
config:
database.dbname: main
database.hostname: HOST
database.password: PASSWORD
database.port: 5432
database.server.name: server-0
database.user: main
decimal.handling.mode: double
heartbeat.action.query: INSERT INTO kafka_connect_heartbeat (id, last_heartbeat_time)
VALUES (1, NOW() at time zone 'utc') ON CONFLICT(id) DO UPDATE SET last_heartbeat_time=EXCLUDED.last_heartbeat_time;
heartbeat.interval.ms: "10000"
heartbeat.topics.prefix: debezium-heartbeat
heartbeat.writeback.enabled: "true"
heartbeat.writeback.table: public.kafka_connect_heartbeat
plugin.name: pgoutput
publication.autocreate.mode: filtered
publication.name: uno_main_0
slot.name: uno_main_0
table.include.list: public.outbox_wal
table.whitelist: public.outbox_wal
tasksMax: 1
transforms: sample
transforms.sample.route.topic.replacement: ${routedByValue}
transforms.sample.table.expand.json.payload: "true"
transforms.sample.table.fields.additional.placement: type:header:eventType,id:header:messageId
transforms.sample.type: io.debezium.transforms.outbox.EventRouter
What is the captured database version and mode of depoyment?
DB : AWS Postgres RDS 12.11 (Managed Service)
What behavior do you expect?
There should not be exponential growth in replication slot disk usage upon connector restart and message publishing should resume.
What behaviour do you see?
- In case of stuck transaction(long running queries) , transaction disk usage increases exponentially.
- Upon killing queries and restart of kafka connector , debezium catchup takes lot of time and database runs out of disk space.
- Upon increasing additional disk space and iops approximately after 12-14 hrs connectors starts publishing messages
- We observed restart_lsn doesn't get updated and it takes 12-14 hrs to get it matched with confirmed_flush_lsn.
- We have implemented hearbeat as well.
Do you see the same behaviour using the latest relesead Debezium version?
Yes
Do you have the connector logs, ideally from start till finish?
No(You might be asked later to provide DEBUG/TRACE level log)
How to reproduce the issue using our tutorial deployment?
Feature request or enhancement
For feature requests or enhancements, provide this information, please:
Which use case/requirement will be addressed by the proposed feature?
<Your answer>
Implementation ideas (optional)
<Your answer>