Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-4802

"DebeziumException: Unable to find primary from MongoDB connection" post upgrade to 1.8.1

XMLWordPrintable

    • False
    • False
      1. mongodb replica set with primary+secondary
      2. set debezium mongodb 1.8.1, with a simple configuration and `mongodb.hosts`=secondary

      Upgraded Debezium version from 0.9.5 to 1.8.1, with kafka-connect 2.6, and with a mongodb cluster of primary and 1 secondary (and 1 arbiter), got the error below which does not resolve.

      Happens specifically when the provided hostname in the configuration is single, and not the primary, although it seems that discovery does work.

       

      WORKAROUND: configure all hostnames in the seed, where one of them is the primary

       

       

      io.debezium.DebeziumException: Unable to find primary from MongoDB connection, got '[ServerDescription{address=mongodb-secondary-0.mongodb-headless.default.svc.cluster.local:27017, type=REPLICA_SET_SECONDARY, state=CONNECTED, ok=true, minWireVersion=0, maxWireVersion=7, maxDocumentSize=16777216, logicalSessionTimeoutMinutes=30, roundTripTimeNanos=6254943, setName='rs-test', canonicalAddress=mongodb-secondary-0.mongodb-headless.default.svc.cluster.local:27017, hosts=[mongodb-primary-0.mongodb-headless.default.svc.cluster.local:27017, mongodb-secondary-0.mongodb-headless.default.svc.cluster.local:27017], passives=[], arbiters=[], primary='mongodb-primary-0.mongodb-headless.default.svc.cluster.local:27017', tagSet=TagSet
      {[]}
      , electionId=null, setVersion=2, topologyVersion=null, lastWriteDate=Tue Mar 01 09:24:41 UTC 2022, lastUpdateTimeNanos=5786636044107}]'
              at io.debezium.connector.mongodb.MongoUtil.getPrimaryAddress(MongoUtil.java:378)
              at io.debezium.connector.mongodb.ConnectionContext.clientForPrimary(ConnectionContext.java:448)
              at io.debezium.connector.mongodb.ConnectionContext.lambda$primaryClientFor$4(ConnectionContext.java:205)
              at io.debezium.connector.mongodb.ConnectionContext.lambda$primaryClientFor$5(ConnectionContext.java:214)
              at io.debezium.connector.mongodb.ConnectionContext$MongoPrimary.execute(ConnectionContext.java:286)
              at io.debezium.connector.mongodb.MongoDbSnapshotChangeEventSource.lambda$determineSnapshotOffsets$6(MongoDbSnapshotChangeEventSource.java:295)
              at java.base/java.util.HashMap$Values.forEach(HashMap.java:976)
              at io.debezium.connector.mongodb.ReplicaSets.onEachReplicaSet(ReplicaSets.java:115)
              at io.debezium.connector.mongodb.MongoDbSnapshotChangeEventSource.determineSnapshotOffsets(MongoDbSnapshotChangeEventSource.java:290)
              at io.debezium.connector.mongodb.MongoDbSnapshotChangeEventSource.doExecute(MongoDbSnapshotChangeEventSource.java:99)
              at io.debezium.connector.mongodb.MongoDbSnapshotChangeEventSource.doExecute(MongoDbSnapshotChangeEventSource.java:52)
              at io.debezium.pipeline.source.AbstractSnapshotChangeEventSource.execute(AbstractSnapshotChangeEventSource.java:76)
              at io.debezium.pipeline.ChangeEventSourceCoordinator.doSnapshot(ChangeEventSourceCoordinator.java:153)
              at io.debezium.pipeline.ChangeEventSourceCoordinator.executeChangeEventSources(ChangeEventSourceCoordinator.java:135)
              at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:108)
              at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
              at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
              at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
              at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
              at java.base/java.lang.Thread.run(Thread.java:834)
       
      

       

      Reproduced in minikube and TRACE log since KC boot until first error is attached

      skafka-connect-local.log

      mongo topology

      1 primary

      1 replica

      1 arbiter

      mongodb version 4.0.10

       

       

      connector config 

       

      {
              "connector.class" : "io.debezium.connector.mongodb.MongoDbConnector",
              "capture.mode": "oplog",
              "tasks.max" : "1",
              "mongodb.hosts" : "rs-test/mongodb-secondary-0.mongodb-headless.default.svc.cluster.local:27017",
              "mongodb.name" : "testdb",
              "mongodb.user" : "..",
              "mongodb.password" : "..",
              "database.include.list" : "testdb",
              "collection.include.list" : "testdb.users",
              "database.history.kafka.bootstrap.servers" : "data-pipeline-cluster-kafka-brokers:9092",
              "key.converter" : "io.confluent.connect.avro.AvroConverter",
              "key.converter.schema.registry.url" : "http://schema-reg-schema-registry:8081",
              "value.converter" : "org.apache.kafka.connect.json.JsonConverter",
              "value.converter.schemas.enable": "false",
              "max.batch.size" : "1024",
              "schema.compatibility" : "NONE",
              "errors.tolerance" : "all"
      }
      

       

            jpechane Jiri Pechanec
            agentalpha Arieh L (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated:
              Resolved: