Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-3788

Initiating MongoDB connector causes oplog table scan

    XMLWordPrintable

Details

    Description

      The fix for DBZ-3331 implemented in these lines creates a table scan of the oplog because there are no (and can be no) indices. We have an oplog of ~50GB and we had to kill several queries (one per connector) that already ran for 4 hours, not a single one succeeded till then. 

      One of our engineers suggested this:

      the oplog is ordered, you could just query the first item and see if the timestamp is less than oplogStart

      Attachments

        Activity

          Public project attachment banner

            context keys: [headless, issue, helper, isAsynchronousRequest, project, action, user]
            current Project key: DBZ

            People

              ccranfor@redhat.com Chris Cranford
              frankkoornstra Frank Koornstra (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: