Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-5677

MongoDB Connector with DocumentDB errors with "{$natural: -1} is not supported"

XMLWordPrintable

      What Debezium connector do you use and what version?

      1.9.6.Final

      What is the connector configuration?

      name: my-connector
      connector.class: io.debezium.connector.mongodb.MongoDbConnector
      mongodb.name: my-connector
      mongodb.hosts: <redacted DocumentDB hosts>
      mongodb.authsource: admin
      mongodb.user: <redacted username>
      mongodb.password: <redacted password>
      tasks.max: 1
      snapshot.mode: never
      capture.mode: change_streams_update_full 

      What is the captured database version and mode of depoyment?

      AWS DocumentDB

      What behaviour do you expect?

      Connector should not fail after snapshotting or when snapshotting is disabled (capture.mode == never)

      What behaviour do you see?

      Connector is failing after snapshotting with the error: 

      Query failed with error code 303 and error message '{$natural: -1} is not supported' on server <redacted documentdb cluster name>.us-east-2.docdb.amazonaws.com:27017 

      Do you see the same behaviour using the latest relesead Debezium version?

      Yes

      Do you have the connector logs, ideally from start till finish?

      2022-10-05 12:45:52.538 [ERROR] 3766 --- [calpkmappings-0] .apache.kafka.connect.runtime.WorkerTask : WorkerSourceTask{id=employer-job-mapping-documentdb-canonicalpkmappings-0} Task threw an uncaught and unrecoverable exception. Task is being killed and will not recover until manually restarted
      org.apache.kafka.connect.errors.ConnectException: An exception occurred in the change event producer. This connector will be stopped.
          at io.debezium.pipeline.ErrorHandler.setProducerThrowable(ErrorHandler.java:50)
          at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:116)
          at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
          at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266)
          at java.util.concurrent.FutureTask.run(FutureTask.java)
          at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
          at java.lang.Thread.run(Thread.java:748)
      Caused by: org.apache.kafka.connect.errors.ConnectException: Error while attempting to get oplog position
          at io.debezium.connector.mongodb.MongoDbStreamingChangeEventSource.lambda$establishConnectionToPrimary$3(MongoDbStreamingChangeEventSource.java:182)
          at io.debezium.connector.mongodb.ConnectionContext$MongoPrimary.execute(ConnectionContext.java:292)
          at io.debezium.connector.mongodb.MongoDbStreamingChangeEventSource.lambda$initializeOffsets$6(MongoDbStreamingChangeEventSource.java:580)
          at java.util.HashMap$Values.forEach(HashMap.java:981)
          at io.debezium.connector.mongodb.ReplicaSets.onEachReplicaSet(ReplicaSets.java:115)
          at io.debezium.connector.mongodb.MongoDbStreamingChangeEventSource.initializeOffsets(MongoDbStreamingChangeEventSource.java:575)
          at io.debezium.connector.mongodb.MongoDbStreamingChangeEventSource.execute(MongoDbStreamingChangeEventSource.java:97)
          at io.debezium.connector.mongodb.MongoDbStreamingChangeEventSource.execute(MongoDbStreamingChangeEventSource.java:59)
          at io.debezium.pipeline.ChangeEventSourceCoordinator.streamEvents(ChangeEventSourceCoordinator.java:174)
          at io.debezium.pipeline.ChangeEventSourceCoordinator.executeChangeEventSources(ChangeEventSourceCoordinator.java:141)
          at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:109)
          ... 6 more
      Caused by: com.mongodb.MongoQueryException: Query failed with error code 303 and error message '{$natural: -1} is not supported' on server employer-job-search-documentdb-qa-1.cbue06mb88c1.us-east-2.docdb.amazonaws.com:27017
          at com.mongodb.internal.operation.FindOperation$1.call(FindOperation.java:663)
          at com.mongodb.internal.operation.FindOperation$1.call(FindOperation.java:653)
          at com.mongodb.internal.operation.OperationHelper.withReadConnectionSource(OperationHelper.java:583)
          at com.mongodb.internal.operation.FindOperation.execute(FindOperation.java:653)
          at com.mongodb.internal.operation.FindOperation.execute(FindOperation.java:81)
          at com.mongodb.client.internal.MongoClientDelegate$DelegateOperationExecutor.execute(MongoClientDelegate.java:184)
          at com.mongodb.client.internal.FindIterableImpl.first(FindIterableImpl.java:200)
          at io.debezium.connector.mongodb.MongoDbStreamingChangeEventSource.lambda$initializeOffsets$5(MongoDbStreamingChangeEventSource.java:582)
          at io.debezium.connector.mongodb.ConnectionContext$MongoPrimary.execute(ConnectionContext.java:288)
          ... 15 more 
      

      Note that this issue is very similar to DBZ-5371 and fix is likely the same

            Unassigned Unassigned
            sage-pierce Sage Pierce (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: