Details
-
Bug
-
Resolution: Done
-
Major
-
None
-
None
Description
What Debezium connector do you use and what version?
1.9.6.Final
What is the connector configuration?
name: my-connector connector.class: io.debezium.connector.mongodb.MongoDbConnector mongodb.name: my-connector mongodb.hosts: <redacted DocumentDB hosts> mongodb.authsource: admin mongodb.user: <redacted username> mongodb.password: <redacted password> tasks.max: 1 snapshot.mode: never capture.mode: change_streams_update_full
What is the captured database version and mode of depoyment?
AWS DocumentDB
What behaviour do you expect?
Connector should not fail after snapshotting or when snapshotting is disabled (capture.mode == never)
What behaviour do you see?
Connector is failing after snapshotting with the error:
Query failed with error code 303 and error message '{$natural: -1} is not supported' on server <redacted documentdb cluster name>.us-east-2.docdb.amazonaws.com:27017
Do you see the same behaviour using the latest relesead Debezium version?
Yes
Do you have the connector logs, ideally from start till finish?
2022-10-05 12:45:52.538 [ERROR] 3766 --- [calpkmappings-0] .apache.kafka.connect.runtime.WorkerTask : WorkerSourceTask{id=employer-job-mapping-documentdb-canonicalpkmappings-0} Task threw an uncaught and unrecoverable exception. Task is being killed and will not recover until manually restarted
org.apache.kafka.connect.errors.ConnectException: An exception occurred in the change event producer. This connector will be stopped.
at io.debezium.pipeline.ErrorHandler.setProducerThrowable(ErrorHandler.java:50)
at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:116)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266)
at java.util.concurrent.FutureTask.run(FutureTask.java)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.kafka.connect.errors.ConnectException: Error while attempting to get oplog position
at io.debezium.connector.mongodb.MongoDbStreamingChangeEventSource.lambda$establishConnectionToPrimary$3(MongoDbStreamingChangeEventSource.java:182)
at io.debezium.connector.mongodb.ConnectionContext$MongoPrimary.execute(ConnectionContext.java:292)
at io.debezium.connector.mongodb.MongoDbStreamingChangeEventSource.lambda$initializeOffsets$6(MongoDbStreamingChangeEventSource.java:580)
at java.util.HashMap$Values.forEach(HashMap.java:981)
at io.debezium.connector.mongodb.ReplicaSets.onEachReplicaSet(ReplicaSets.java:115)
at io.debezium.connector.mongodb.MongoDbStreamingChangeEventSource.initializeOffsets(MongoDbStreamingChangeEventSource.java:575)
at io.debezium.connector.mongodb.MongoDbStreamingChangeEventSource.execute(MongoDbStreamingChangeEventSource.java:97)
at io.debezium.connector.mongodb.MongoDbStreamingChangeEventSource.execute(MongoDbStreamingChangeEventSource.java:59)
at io.debezium.pipeline.ChangeEventSourceCoordinator.streamEvents(ChangeEventSourceCoordinator.java:174)
at io.debezium.pipeline.ChangeEventSourceCoordinator.executeChangeEventSources(ChangeEventSourceCoordinator.java:141)
at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:109)
... 6 more
Caused by: com.mongodb.MongoQueryException: Query failed with error code 303 and error message '{$natural: -1} is not supported' on server employer-job-search-documentdb-qa-1.cbue06mb88c1.us-east-2.docdb.amazonaws.com:27017
at com.mongodb.internal.operation.FindOperation$1.call(FindOperation.java:663)
at com.mongodb.internal.operation.FindOperation$1.call(FindOperation.java:653)
at com.mongodb.internal.operation.OperationHelper.withReadConnectionSource(OperationHelper.java:583)
at com.mongodb.internal.operation.FindOperation.execute(FindOperation.java:653)
at com.mongodb.internal.operation.FindOperation.execute(FindOperation.java:81)
at com.mongodb.client.internal.MongoClientDelegate$DelegateOperationExecutor.execute(MongoClientDelegate.java:184)
at com.mongodb.client.internal.FindIterableImpl.first(FindIterableImpl.java:200)
at io.debezium.connector.mongodb.MongoDbStreamingChangeEventSource.lambda$initializeOffsets$5(MongoDbStreamingChangeEventSource.java:582)
at io.debezium.connector.mongodb.ConnectionContext$MongoPrimary.execute(ConnectionContext.java:288)
... 15 more
Note that this issue is very similar to DBZ-5371 and fix is likely the same
