-
Bug
-
Resolution: Done
-
Major
-
None
-
None
What Debezium connector do you use and what version?
1.9.6.Final
What is the connector configuration?
name: my-connector connector.class: io.debezium.connector.mongodb.MongoDbConnector mongodb.name: my-connector mongodb.hosts: <redacted DocumentDB hosts> mongodb.authsource: admin mongodb.user: <redacted username> mongodb.password: <redacted password> tasks.max: 1 snapshot.mode: never capture.mode: change_streams_update_full
What is the captured database version and mode of depoyment?
AWS DocumentDB
What behaviour do you expect?
Connector should not fail after snapshotting or when snapshotting is disabled (capture.mode == never)
What behaviour do you see?
Connector is failing after snapshotting with the error:
Query failed with error code 303 and error message '{$natural: -1} is not supported' on server <redacted documentdb cluster name>.us-east-2.docdb.amazonaws.com:27017
Do you see the same behaviour using the latest relesead Debezium version?
Yes
Do you have the connector logs, ideally from start till finish?
2022-10-05 12:45:52.538 [ERROR] 3766 --- [calpkmappings-0] .apache.kafka.connect.runtime.WorkerTask : WorkerSourceTask{id=employer-job-mapping-documentdb-canonicalpkmappings-0} Task threw an uncaught and unrecoverable exception. Task is being killed and will not recover until manually restarted org.apache.kafka.connect.errors.ConnectException: An exception occurred in the change event producer. This connector will be stopped. at io.debezium.pipeline.ErrorHandler.setProducerThrowable(ErrorHandler.java:50) at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:116) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266) at java.util.concurrent.FutureTask.run(FutureTask.java) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: org.apache.kafka.connect.errors.ConnectException: Error while attempting to get oplog position at io.debezium.connector.mongodb.MongoDbStreamingChangeEventSource.lambda$establishConnectionToPrimary$3(MongoDbStreamingChangeEventSource.java:182) at io.debezium.connector.mongodb.ConnectionContext$MongoPrimary.execute(ConnectionContext.java:292) at io.debezium.connector.mongodb.MongoDbStreamingChangeEventSource.lambda$initializeOffsets$6(MongoDbStreamingChangeEventSource.java:580) at java.util.HashMap$Values.forEach(HashMap.java:981) at io.debezium.connector.mongodb.ReplicaSets.onEachReplicaSet(ReplicaSets.java:115) at io.debezium.connector.mongodb.MongoDbStreamingChangeEventSource.initializeOffsets(MongoDbStreamingChangeEventSource.java:575) at io.debezium.connector.mongodb.MongoDbStreamingChangeEventSource.execute(MongoDbStreamingChangeEventSource.java:97) at io.debezium.connector.mongodb.MongoDbStreamingChangeEventSource.execute(MongoDbStreamingChangeEventSource.java:59) at io.debezium.pipeline.ChangeEventSourceCoordinator.streamEvents(ChangeEventSourceCoordinator.java:174) at io.debezium.pipeline.ChangeEventSourceCoordinator.executeChangeEventSources(ChangeEventSourceCoordinator.java:141) at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:109) ... 6 more Caused by: com.mongodb.MongoQueryException: Query failed with error code 303 and error message '{$natural: -1} is not supported' on server employer-job-search-documentdb-qa-1.cbue06mb88c1.us-east-2.docdb.amazonaws.com:27017 at com.mongodb.internal.operation.FindOperation$1.call(FindOperation.java:663) at com.mongodb.internal.operation.FindOperation$1.call(FindOperation.java:653) at com.mongodb.internal.operation.OperationHelper.withReadConnectionSource(OperationHelper.java:583) at com.mongodb.internal.operation.FindOperation.execute(FindOperation.java:653) at com.mongodb.internal.operation.FindOperation.execute(FindOperation.java:81) at com.mongodb.client.internal.MongoClientDelegate$DelegateOperationExecutor.execute(MongoClientDelegate.java:184) at com.mongodb.client.internal.FindIterableImpl.first(FindIterableImpl.java:200) at io.debezium.connector.mongodb.MongoDbStreamingChangeEventSource.lambda$initializeOffsets$5(MongoDbStreamingChangeEventSource.java:582) at io.debezium.connector.mongodb.ConnectionContext$MongoPrimary.execute(ConnectionContext.java:288) ... 15 more
Note that this issue is very similar to DBZ-5371 and fix is likely the same