-
Bug
-
Resolution: Done
-
Major
-
3.1.3.Final
-
None
-
False
-
-
False
Connector : Postgres connector
Version : 3.1.3.Final
The retry configuration :
props.setProperty("errors.max.retries", "35");
props.setProperty("errors.retry.delay.initial.ms", "300");
props.setProperty("errors.retry.delay.max.ms", "10000");
After stoping the database container, I am expecting my connector to retry until the number of retries are completed.
With the deprecated Debezium engine (on version 3.1.3.Final), I have the expected outcome (the connector makes the retry attemps) :
10:47:52.669 [pool-8-thread-1] ERROR FT: i.d.e.EmbeddedEngine - Can't start the connector, will retry later... io.debezium.DebeziumException: Couldn't obtain encoding for database db at io.debezium.connector.postgresql.connection.PostgresConnection.getDatabaseCharset(PostgresConnection.java:599) ~[debezium-connector-postgres-3.1.3.Final.jar:3.1.3.Final] 10:47:57.470 [pool-8-thread-1] INFO FT: i.d.e.EmbeddedEngine - Starting connector, attempt 6 10:47:57.470 [pool-8-thread-1] INFO FT: i.d.c.c.BaseSourceTask - Stopping down connector 10:47:57.470 [pool-8-thread-1] DEBUG FT: i.d.c.p.c.PostgresReplicationConnection - Closing message decoder 10:47:57.470 [pool-8-thread-1] DEBUG FT: i.d.c.p.c.PostgresReplicationConnection - Closing replication connection 10:47:57.470 [pool-8-thread-1] DEBUG FT: i.d.c.c.BaseSourceTask - Setting task state to 'STOPPED', previous state was 'INITIAL' 10:47:57.471 [pool-8-thread-1] DEBUG FT: i.d.c.c.BaseSourceTask - Setting task state to 'INITIAL', previous state was 'STOPPED' 10:47:57.471 [pool-8-thread-1] INFO FT: i.d.c.c.BaseSourceTask - Starting PostgresConnectorTask with configuration:
But on the AsyncEmbeddedEngine, my connector stops after the first retry :
10:06:02.290 [pool-8-thread-1] INFO FT: i.d.c.c.BaseSourceTask - Attempting to restart task. 10:06:22.575 [pool-10-thread-1] ERROR FT: i.d.e.a.AsyncEmbeddedEngine - Engine has failed with java.util.concurrent.ExecutionException: io.debezium.DebeziumException: Couldn't obtain encoding for database db Caused by: io.debezium.DebeziumException: Couldn't obtain encoding for database db 10:06:22.577 [pool-10-thread-1] DEBUG FT: i.d.e.a.AsyncEmbeddedEngine - Stopping io.debezium.embedded.async.AsyncEmbeddedEngine 10:06:22.577 [pool-10-thread-1] INFO FT: i.d.e.a.AsyncEmbeddedEngine - Engine state has changed from 'POLLING_TASKS' to 'STOPPING' 10:06:22.577 [pool-10-thread-1] DEBUG FT: i.d.e.a.AsyncEmbeddedEngine - Tasks were already started, stopping record service and tasks. 10:06:22.577 [pool-10-thread-1] DEBUG FT: i.d.e.a.AsyncEmbeddedEngine - Stopping records service. 10:06:22.578 [pool-10-thread-1] DEBUG FT: i.d.e.a.AsyncEmbeddedEngine - Stopping source connector tasks. 10:06:22.579 [pool-10-thread-1] DEBUG FT: i.d.e.a.AsyncEmbeddedEngine - Waiting max. for 180000 ms for individual source tasks to stop. 10:06:22.581 [pool-8-thread-1] DEBUG FT: i.d.e.a.AsyncEmbeddedEngine - Committing task's offset. 10:06:22.581 [pool-8-thread-1] TRACE FT: i.d.e.a.AsyncEmbeddedEngine - No offset to be committed. 10:06:22.582 [pool-8-thread-1] DEBUG FT: i.d.e.a.AsyncEmbeddedEngine - Stopping Connect task. 10:06:22.582 [pool-8-thread-1] INFO FT: i.d.c.c.BaseSourceTask - Stopping down connector 10:06:22.583 [pool-8-thread-1] DEBUG FT: i.d.c.p.c.PostgresReplicationConnection - Closing message decoder 10:06:22.584 [pool-8-thread-1] DEBUG FT: i.d.c.p.c.PostgresReplicationConnection - Closing replication connection 10:06:22.584 [pool-8-thread-1] DEBUG FT: i.d.c.c.BaseSourceTask - Setting task state to 'STOPPED', previous state was 'RESTARTING' 10:06:22.585 [pool-10-thread-1] INFO FT: i.d.e.a.AsyncEmbeddedEngine - Stopped task #1 out of 1 tasks (it took 5 ms to stop the task). 10:06:22.585 [pool-10-thread-1] DEBUG FT: i.d.e.a.AsyncEmbeddedEngine - Calling connector callback after task is stopped. 10:06:22.587 [pool-10-thread-1] DEBUG FT: i.d.e.a.AsyncEmbeddedEngine - Stopping all remaining tasks if there are any. 10:06:22.587 [pool-10-thread-1] DEBUG FT: i.d.e.a.AsyncEmbeddedEngine - Stopping offset backing store. 10:06:23.079 [pool-10-thread-1] DEBUG FT: i.d.e.a.AsyncEmbeddedEngine - Stopping the connector. 10:06:23.079 [pool-10-thread-1] DEBUG FT: i.d.e.a.AsyncEmbeddedEngine - Calling connector callback after connector stop 10:06:23.080 [pool-10-thread-1] INFO FT: i.d.e.a.AsyncEmbeddedEngine - Engine is stopped. 10:06:23.080 [pool-10-thread-1] INFO FT: i.d.e.a.AsyncEmbeddedEngine - Engine state has changed from 'STOPPING' to 'STOPPED' 10:06:23.080 [pool-10-thread-1] DEBUG FT: i.d.e.a.AsyncEmbeddedEngine - Calling completion handler.
With that behaviour, my connector crashes if I do have a small network outage.
