-
Bug
-
Resolution: Done
-
Major
-
0.8.0.Final
-
None
If a PostgreSQL table is ignored via the table.whitelist or table.blacklist connector option, Debezium ignores WAL events on that table. However, it does not commit the LSNs. This causes WAL files to build up on the database until write activity happens on the whitelisted tables. This may cause problems for tables with infrequent writes.
From the code, it seems that LSN commits are triggered whenever the Kafka offsets are committed:
https://github.com/debezium/debezium/blob/v0.8.0.Final/debezium-connector-postgres/src/main/java/io/debezium/connector/postgresql/PostgresConnectorTask.java#L136
However, table filtering bypasses that code:
https://github.com/debezium/debezium/blob/v0.8.0.Final/debezium-connector-postgres/src/main/java/io/debezium/connector/postgresql/RecordsStreamProducer.java#L229
This bug might also be occurring when the table operation is unknown/ignored:
https://github.com/debezium/debezium/blob/v0.8.0.Final/debezium-connector-postgres/src/main/java/io/debezium/connector/postgresql/RecordsStreamProducer.java#L254