-
Bug
-
Resolution: Done
-
Major
-
1.8.1.Final
-
None
-
False
-
False
-
- mongodb replica set with primary+secondary
- set debezium mongodb 1.8.1, with a simple configuration and `mongodb.hosts`=secondary
-
Upgraded Debezium version from 0.9.5 to 1.8.1, with kafka-connect 2.6, and with a mongodb cluster of primary and 1 secondary (and 1 arbiter), got the error below which does not resolve.
Happens specifically when the provided hostname in the configuration is single, and not the primary, although it seems that discovery does work.
WORKAROUND: configure all hostnames in the seed, where one of them is the primary
io.debezium.DebeziumException: Unable to find primary from MongoDB connection, got '[ServerDescription{address=mongodb-secondary-0.mongodb-headless.default.svc.cluster.local:27017, type=REPLICA_SET_SECONDARY, state=CONNECTED, ok=true, minWireVersion=0, maxWireVersion=7, maxDocumentSize=16777216, logicalSessionTimeoutMinutes=30, roundTripTimeNanos=6254943, setName='rs-test', canonicalAddress=mongodb-secondary-0.mongodb-headless.default.svc.cluster.local:27017, hosts=[mongodb-primary-0.mongodb-headless.default.svc.cluster.local:27017, mongodb-secondary-0.mongodb-headless.default.svc.cluster.local:27017], passives=[], arbiters=[], primary='mongodb-primary-0.mongodb-headless.default.svc.cluster.local:27017', tagSet=TagSet {[]} , electionId=null, setVersion=2, topologyVersion=null, lastWriteDate=Tue Mar 01 09:24:41 UTC 2022, lastUpdateTimeNanos=5786636044107}]' at io.debezium.connector.mongodb.MongoUtil.getPrimaryAddress(MongoUtil.java:378) at io.debezium.connector.mongodb.ConnectionContext.clientForPrimary(ConnectionContext.java:448) at io.debezium.connector.mongodb.ConnectionContext.lambda$primaryClientFor$4(ConnectionContext.java:205) at io.debezium.connector.mongodb.ConnectionContext.lambda$primaryClientFor$5(ConnectionContext.java:214) at io.debezium.connector.mongodb.ConnectionContext$MongoPrimary.execute(ConnectionContext.java:286) at io.debezium.connector.mongodb.MongoDbSnapshotChangeEventSource.lambda$determineSnapshotOffsets$6(MongoDbSnapshotChangeEventSource.java:295) at java.base/java.util.HashMap$Values.forEach(HashMap.java:976) at io.debezium.connector.mongodb.ReplicaSets.onEachReplicaSet(ReplicaSets.java:115) at io.debezium.connector.mongodb.MongoDbSnapshotChangeEventSource.determineSnapshotOffsets(MongoDbSnapshotChangeEventSource.java:290) at io.debezium.connector.mongodb.MongoDbSnapshotChangeEventSource.doExecute(MongoDbSnapshotChangeEventSource.java:99) at io.debezium.connector.mongodb.MongoDbSnapshotChangeEventSource.doExecute(MongoDbSnapshotChangeEventSource.java:52) at io.debezium.pipeline.source.AbstractSnapshotChangeEventSource.execute(AbstractSnapshotChangeEventSource.java:76) at io.debezium.pipeline.ChangeEventSourceCoordinator.doSnapshot(ChangeEventSourceCoordinator.java:153) at io.debezium.pipeline.ChangeEventSourceCoordinator.executeChangeEventSources(ChangeEventSourceCoordinator.java:135) at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:108) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:834)
Reproduced in minikube and TRACE log since KC boot until first error is attached
mongo topology
1 primary
1 replica
1 arbiter
mongodb version 4.0.10
connector config
{ "connector.class" : "io.debezium.connector.mongodb.MongoDbConnector", "capture.mode": "oplog", "tasks.max" : "1", "mongodb.hosts" : "rs-test/mongodb-secondary-0.mongodb-headless.default.svc.cluster.local:27017", "mongodb.name" : "testdb", "mongodb.user" : "..", "mongodb.password" : "..", "database.include.list" : "testdb", "collection.include.list" : "testdb.users", "database.history.kafka.bootstrap.servers" : "data-pipeline-cluster-kafka-brokers:9092", "key.converter" : "io.confluent.connect.avro.AvroConverter", "key.converter.schema.registry.url" : "http://schema-reg-schema-registry:8081", "value.converter" : "org.apache.kafka.connect.json.JsonConverter", "value.converter.schemas.enable": "false", "max.batch.size" : "1024", "schema.compatibility" : "NONE", "errors.tolerance" : "all" }
- relates to
-
DBZ-4733 support mongodb connection string as configuration option
- Resolved