Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-3342

Implement incremental snapshotting for MongoDB

    XMLWordPrintable

Details

    • Feature Request
    • Resolution: Done
    • Major
    • 1.8.0.CR1
    • 1.7.0.CR1
    • mongodb-connector
    • None
    • False
    • False
    • 0
    • 0% 0%
    • Undefined

    Description

      Overview:

      I raised DBZ-3331 because we have a very large MongoDB collection that isn't being snapshotted in time of the oplog window. A solution to this problem could be providing a feature to optionally stream data from the oplog concurrently while performing the snapshot. 

      I think this feature would be useful for more than just MongoDB. With PostgreSQL it is possible that the disks get filled with WAL files if a snapshot takes too long on a high traffic server. With MySQL there is the same problem as MongoDB that if the snapshot takes longer than the binlog retention period the connector will fail.

      Attachments

        Issue Links

          is related to

          Feature Request - Feature requests from customers and/or users DBZ-3692 Implement incremental snapshotting for Oracle

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Closed

          Task - Represents a small unit of work that is not end-user facing. DBZ-4427 Complete MongoDB incremental snapshotting implementation

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Closed

          Enhancement - An enhancement to or refactoring of existing functionality that is not configurable by an end user (typically a change made by an internal team that affects users) DBZ-998 MongoDB initial sync process should be more robust

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Closed

          Activity

            People

              jpechane Jiri Pechanec
              ryan.tomczik Ryan Tomczik (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: