-
Feature Request
-
Resolution: Done
-
Minor
-
None
-
None
-
False
-
None
-
False
It is observed that immediately after completing the snapshot, the Debezium connectors occasionally encounter a transient exception (e.g. a network failure) when starting the streaming process. In some cases, if the offset for the last snapshot record is not flushed and the connector is restarted, it leads to re-snapshotting of the tables.
To avoid this issue, this ticket proposes introducing a configurable delay before starting the streaming process after the snapshot is completed.
A new configuration parameter (`streaming.delay.ms`) can be added to control this delay. This is symmetrical to `snapshot.delay.ms` configuration that is already available to add delays before starting snapshotting.
The default value would be set to `0`, which means there would be no delay by default. Users can configure a value slightly greater than the `offset.flush.interval.ms` configuration, ensuring that offsets are properly flushed before streaming begins.
This adjustment should help prevent unnecessary re-snapshotting and improve the overall stability of the connector during the transition from snapshot to streaming.
- links to
-
RHEA-2024:139598 Red Hat build of Debezium 2.5.4 release