-
Bug
-
Resolution: Unresolved
-
Major
-
2.1.2.Final
-
None
-
False
-
None
-
False
-
Critical
What Debezium connector do you use and what version?
debezium-connector-postgres 2.1.2.Final
What is the connector configuration?
{
"connector.class": "io.debezium.connector.postgresql.PostgresConnector",
"incremental.snapshot.chunk.size": "5120",
"slot.name": "slot_name",
"tasks.max": "1",
"publication.name": "pub_name",
"value.converter.enhanced.avro.schema.support": "true",
"value.converter.basic.auth.credentials.source": "USER_INFO",
"database.sslmode": "require",
"topic.prefix": "registry_insert_only",
"signal.data.collection": "public.signals",
"value.converter": "io.confluent.connect.avro.AvroConverter",
"key.converter": "io.confluent.connect.avro.AvroConverter",
"publication.autocreate.mode": "disabled",
"database.dbname": "snykregistry",
"database.user": "${dir:/etc/debezium:username}",
"database.server.name": "insert_only",
"key.converter.enhanced.avro.schema.support": "true",
"database.port": "5432",
"plugin.name": "pgoutput",
"key.converter.basic.auth.user.info": "${dir:/etc/schema-registry-credentials:user-info}",
"value.converter.schema.registry.url": "${dir:/etc/schema-registry-credentials:url}",
"column.exclude.list": "......",
"value.converter.basic.auth.user.info": "${dir:/etc/schema-registry-credentials:user-info}",
"database.hostname": "hostname",
"database.password": "${dir:/etc/debezium:password}",
"name": "debezium-registry-insert-only-cdc",
"skipped.operations": "t,u,d",
"table.include.list": "tables",
"key.converter.schema.registry.url": "${dir:/etc/schema-registry-credentials:url}",
"key.converter.basic.auth.credentials.source": "USER_INFO",
"snapshot.mode": "never"
}
What is the captured database version and mode of depoyment?
GCP PostgreSQL 13
What behaviour do you expect?
Debezium is able to stream non-stop under normal circumstances
What behaviour do you see?
Debezium stops streaming regularly with "ERROR: cannot advance replication slot to XXXX/XXXXX, minimum is YYYY/YYYY" (LSN obviously change every time).
I have to change the offsets manually each time to:
{
"transaction_id": null, <- leaving unchanged
"lsn_proc": 123123123, <- minimum from error log
"lsn_commit": null, <- set to null
"lsn": 123123123, <- minimum from error log
"txId": 234234, <- leaving unchanged
"ts_usec": 234234 <- leaving unchanged
}
This behaviour seems to be related to heavy load in the order of 10,000 events/second.
Do you see the same behaviour using the latest relesead Debezium version?
I can not test pre-release version in PROD
Do you have the connector logs, ideally from start till finish?
Trying to gather logs, need to cleanup any sensitive information first.
- is duplicated by
-
-
- Open
-
