-
Enhancement
-
Resolution: Obsolete
-
Major
-
None
-
1.7.0.CR1
-
None
Based on experience and looking into the source code, Debezium's MongoDB Connector's behavior is to restart the initial sync process whenever there is any interruption in the process - e.g. Kafka Connect service restart, task rebalancing, exception, ... This is also documented in previous Issues:
https://issues.jboss.org/projects/DBZ/issues/DBZ-182?filter=allissues
https://issues.jboss.org/browse/DBZ-712
https://issues.jboss.org/browse/DBZ-838
This poses a problem in cases where the initial sync process takes a long time to finish (even several days). The chance of some interruption event occuring is growing with the length of initial sync operation, making it very difficult and that much more time-consuming to get to the oplog-reading state.
We propose a modification to the behavior that will allow Debezium to continue and finish already started initial sync process when recovering from any interruption event (given all other constraints remain valid, such oplog not rolling over in the meantime, data consistency).
- relates to
-
DBZ-3342 Implement incremental snapshotting for MongoDB
- Closed