Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-3342

Implement incremental snapshotting for MongoDB

XMLWordPrintable

    • Icon: Feature Request Feature Request
    • Resolution: Done
    • Icon: Major Major
    • 1.8.0.CR1
    • 1.7.0.CR1
    • mongodb-connector
    • None
    • False
    • False
    • Undefined

      Overview:

      I raised DBZ-3331 because we have a very large MongoDB collection that isn't being snapshotted in time of the oplog window. A solution to this problem could be providing a feature to optionally stream data from the oplog concurrently while performing the snapshot. 

      I think this feature would be useful for more than just MongoDB. With PostgreSQL it is possible that the disks get filled with WAL files if a snapshot takes too long on a high traffic server. With MySQL there is the same problem as MongoDB that if the snapshot takes longer than the binlog retention period the connector will fail.

              jpechane Jiri Pechanec
              ryan.tomczik Ryan Tomczik (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Created:
                Updated:
                Resolved: