Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-3788

Initiating MongoDB connector causes oplog table scan

    XMLWordPrintable

Details

    Description

      The fix for DBZ-3331 implemented in these lines creates a table scan of the oplog because there are no (and can be no) indices. We have an oplog of ~50GB and we had to kill several queries (one per connector) that already ran for 4 hours, not a single one succeeded till then. 

      One of our engineers suggested this:

      the oplog is ordered, you could just query the first item and see if the timestamp is less than oplogStart

      Attachments

        Activity

          People

            ccranfor@redhat.com Chris Cranford
            frankkoornstra Frank Koornstra (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: