-
Enhancement
-
Resolution: Obsolete
-
Major
-
None
-
2
-
False
-
-
False
Feature request or enhancement
For feature requests or enhancements, provide this information, please:
Which use case/requirement will be addressed by the proposed feature?
When connecting Debezium to the writer endpoint of an RDS Aurora cluster using Postgres, in case of failover of the primary, the connector stops working because of the following error:
{{2024-12-30 14:59:09,488 ERROR || WorkerSourceTask
{id=core-connector-0}
Task threw an uncaught and unrecoverable exception. Task is being killed and will not recover until manually restarted [org.apache.kafka.connect.runtime.WorkerTask]org.apache.kafka.connect.errors.ConnectException: Unable to update filtered publication dbz_publication for "Common"."OutboxEvent"at io.debezium.connector.postgresql.connection.PostgresReplicationConnection.createOrUpdatePublicationModeFilterted(PostgresReplicationConnection.java:212)at io.debezium.connector.postgresql.connection.PostgresReplicationConnection.initPublication(PostgresReplicationConnection.java:177)at io.debezium.connector.postgresql.connection.PostgresReplicationConnection.createReplicationSlot(PostgresReplicationConnection.java:435)at io.debezium.connector.postgresql.PostgresConnectorTask.start(PostgresConnectorTask.java:136)at io.debezium.connector.common.BaseSourceTask.start(BaseSourceTask.java:136)at io.debezium.connector.common.BaseSourceTask.startIfNeededAndPossible(BaseSourceTask.java:234)at io.debezium.connector.common.BaseSourceTask.poll(BaseSourceTask.java:158)at org.apache.kafka.connect.runtime.AbstractWorkerSourceTask.poll(AbstractWorkerSourceTask.java:452)at org.apache.kafka.connect.runtime.AbstractWorkerSourceTask.execute(AbstractWorkerSourceTask.java:346)at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:189)at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:244)at org.apache.kafka.connect.runtime.AbstractWorkerSourceTask.run(AbstractWorkerSourceTask.java:72)at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)at java.base/java.lang.Thread.run(Thread.java:829)Caused by: org.postgresql.util.PSQLException: ERROR: cannot execute ALTER PUBLICATION in a read-only transactionat org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2676)at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2366)at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:356)at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:496)at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:413)at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:333)at org.postgresql.jdbc.PgStatement.executeCachedSql(PgStatement.java:319)at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:295)at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:290)at io.debezium.connector.postgresql.connection.PostgresReplicationConnection.createOrUpdatePublicationModeFilterted(PostgresReplicationConnection.java:209)... 16 more2024-12-30 14:59:09,494 INFO || Stopping down connector [io.debezium.connector.common.BaseSourceTask]}}
We discussed this case here.
Implementation ideas (optional)
This is expected in such a setup since the writer endpoint might be pointing to a reader instance for a while until it's promoted to primary. From my POV such errors should be retried by default.
