In order to make your issue reports as actionable as possible, please provide the following information, depending on the issue type.

Bug report

The connection of  Pulsar MongoDB Debezium from a docker container to a MongoDB Atlas Cluster is not working

What Debezium connector do you use and what version?

Pulsar Mongo Debezium Source Connector 2.10.1

What is the connector configuration?

tenant: "public"
namespace: "default"
name: "debezium-mongodb-source"
topicName: "debezium-mongodb-topic"
archive: "connectors/pulsar-io-mongodb-source-2.10.1.nar"
parallelism: 1
use-tls: false
 
configs:
    # config for pg, docker image: debezium/example-mongodb:latest
    #mongodb.hosts: "atlas-vdt3wg-shard-0/ramses-shard-00-00.gcd45.mongodb.net:27017,[ramses-shard-00-01.gcd22.mongodb.net:27017|http://ramses-shard-00-01.gcd22.mongodb.net:27017/],[ramses-shard-00-02.gcd45.mongodb.net:27017|http://ramses-shard-00-02.gcd45.mongodb.net:27017/]"
    #mongodb.hosts: "atlas-vdt3wg-shard-0/ramses-shard-00-01.gcd45.mongodb.net:27017"
    mongodb.name: "dbserver3"
    mongodb.connection.string: "mongodb+srv://pulsaradmin:4589...@ramses.gcd22.mongodb.net/?tls=true&serverSelectionTimeoutMS=60000&retryWrites=true&w=majority"
    # Remember to comment the user and password if the user/password authentication is not enabled for your MongoDB server
    mongodb.members.auto.discover: false
    mongodb.ssl.enabled: false
    mongodb.authsource: "admin"
    mongodb.user: "pulsaradmin"
    mongodb.password: " 45896633  "
    mongodb.task.id: "1"
    database.whitelist: "ramses"
    # PULSAR_SERVICE_URL_CONFIG
    #pulsar.service.url: "pulsar://127.0.0.1:6650"
    pulsar.service.url: "pulsar+ssl://127.0.0.1:6651"
    offset.storage.topic: "offset-topic"

What is the captured database version and mode of depoyment?

(E.g. on-premises, with a specific cloud provider, etc.)

<Your answer>

What behaviour do you expect?

A successful connection to MongoDB Atlas cluster

What behaviour do you see?

Abraham DIKA

<andypex2000@gmail.com>
9:36 AM (5 minutes ago) 
 
 
 
to debezium
Hi Team,Happy holidays to all!!
 
For quite some time now, I am trying to connect Pulsar MongoDb Debezium from a docker container to a MongoDB Cluster 
 
Below is the Pulsar Docker compose 
 
version: '3.9'

services:

  pulsar-server:
    command: bin/pulsar standalone
    build:
      context: ./pulsar/
    container_name: pulsar-server
    hostname: pulsar-server
    volumes:
      # CA certificate in the '.pem' format.
      - "./certs/elasticsearch-ca.pem:/pulsar/certs/elasticsearch-ca.pem"
      - "./certs/ca/truststore.jks:/pulsar/certs/ca/truststore.jks"
      - "./certs/ca/keystore.jks:/pulsar/certs/ca/keystore.jks"
      # Copy Debezium - Pulsar Source Connector Files
      - "./util/mongodb-source-config.yaml:/pulsar/connectors/mongodb-source-config.yaml"
      - "./target/pulsar-io-mongodb-source-2.10.1.nar:/pulsar/connectors/pulsar-io-mongodb-source-2.10.1.nar"
      # Copy Pulsar - Elasticsearch Sink Connector files
      - "./util/elasticsearch-sink.yml:/pulsar/connectors/elasticsearch-sink.yml"
      - "./target/pulsar-io-elk-sink-2.10.1.nar:/pulsar/connectors/pulsar-io-elk-sink-2.10.1.nar"
      # Pulsar TLS and JKS trustore files
      - "./certs/mongodb-atlas.pem:/pulsar/certs/tls/mongodb-atlas.pem"
      - "./certs/isrgrootx1.pem:/pulsar/certs/isrgrootx1.pem"
      - "./certs/bundle.p12:/pulsar/certs/bundle.p12"
      - "./certs/certs/ca.cert.pem:/pulsar/certs/ca.cert.pem"
      - "./certs/broker.cert.pem:/pulsar/certs/broker.cert.pem"
      - "./certs/broker.csr.pem:/pulsar/certs/broker.csr.pem"
      - "./certs/broker.key.pem:/pulsar/certs/broker.key.pem"
      - "./certs/broker.key-pk8.pem:/pulsar/certs/broker.key-pk8.pem"
      - "./certs/pulsar-admin.cert.pem:/pulsar/certs/pulsar-admin.cert.pem"
      - "./certs/pulsar-admin.csr.pem:/pulsar/certs/pulsar-admin.csr.pem"
      - "./certs/pulsar-admin.key.pem:/pulsar/certs/pulsar-admin.key.pem"
      - "./certs/pulsar-admin.key-pk8.pem:/pulsar/certs/pulsar-admin.key-pk8.pem"
      - "./certs/client.truststore.jks:/pulsar/certs/client.truststore.jks"
      - "./certs/client.keystore.jks:/pulsar/certs/client.keystore.jks"
      - "./certs/broker.truststore.jks:/pulsar/certs/broker.truststore.jks"
      - "./certs/broker.keystore.jks:/pulsar/certs/broker.keystore.jks"
      #Configuration files
      - "./conf/websocket.conf:/pulsar/conf/websocket.conf"
      - "./conf/proxy.conf:/pulsar/conf/proxy.conf"
      - "./conf/broker.conf:/pulsar/conf/broker.conf"
      - "./conf/client.conf:/pulsar/conf/client.conf"
    expose:
      - 8080
      - 8081
      - 8443
      - 6650
      - 6651
    ports:
    - 8080:8080
    - 8081:8081
    - 8443:8443
    - 6650:6650
    - 6651:6651
    networks:
    - custom-sink-connector
    cap_add:
    - NET_ADMIN
    - NET_RAW
    - SYS_ADMIN
    privileged: true
    # command: >
    #   /bin/bash -c
    #   'bin/pulsar standalone'
    environment:
      - tlsEnabled=true
    healthcheck:
      interval: 10s
      retries: 20
      test: curl --write-out 'HTTP %{http_code}' --fail --silent --output /dev/null http://localhost:8080/admin/v2/clusters/standalone
 
volumes:
  pulsar-io-data:
    driver: local

networks:
  custom-sink-connector:
    driver: bridgeversion: '3.9'

 
 
 Here is the MongoDB Debezium configuration file to connect to MongoDB Atlas Cluster :
 
 
tenant: "public"
namespace: "default"
name: "debezium-mongodb-source"
topicName: "debezium-mongodb-topic"
archive: "connectors/pulsar-io-mongodb-source-2.10.1.nar"
parallelism: 1
use-tls: false

configs:
    # config for pg, docker image: debezium/example-mongodb:latest
    #mongodb.hosts: "atlas-vdt3wg-shard-0/ramses-shard-00-00.gcd45.mongodb.net:27017,[ramses-shard-00-01.gcd22.mongodb.net:27017|http://ramses-shard-00-01.gcd22.mongodb.net:27017/],[ramses-shard-00-02.gcd45.mongodb.net:27017|http://ramses-shard-00-02.gcd45.mongodb.net:27017/]"
    #mongodb.hosts: "atlas-vdt3wg-shard-0/ramses-shard-00-01.gcd45.mongodb.net:27017"
    mongodb.name: "dbserver3"
    mongodb.connection.string: "mongodb+srv://pulsaradmin:4589...@ramses.gcd22.mongodb.net/?tls=true&serverSelectionTimeoutMS=60000&retryWrites=true&w=majority"
    # Remember to comment the user and password if the user/password authentication is not enabled for your MongoDB server
    mongodb.members.auto.discover: false
    mongodb.ssl.enabled: false
    mongodb.authsource: "admin"
    mongodb.user: "pulsaradmin"
    mongodb.password: " 45896633  "
    mongodb.task.id: "1"
    database.whitelist: "ramses"
    # PULSAR_SERVICE_URL_CONFIG
    #pulsar.service.url: "pulsar://127.0.0.1:6650"
    pulsar.service.url: "pulsar+ssl://127.0.0.1:6651"
    offset.storage.topic: "offset-topic"
 
 
Below is the command that I use to  start Pulsar Mongo DB Debezium connection to MongoDB Atlas :
 
export PULSAR_HOME=/workspaces/pulsar-elk-sink
export PATH=${PATH}:$PULSAR_HOME/bin
export PULSAR_CLASSPATH=${PULSAR_HOME}/lib/jars/*:${PULSAR_HOME}/target/classes

#Start MongoDb - Pulsar Source Connector
$PULSAR_HOME/bin/pulsar-admin  source localrun -archive ${PWD}/../connectors/pulsar-io-debezium-mongodb-2.10.1.nar  source-config-file ${PWD}/../util/mongodb-source-config.yaml  tls-trust-cert-path ${PWD}/../certs/certs/ca.cert.pem  use-tls --parallelism 1  -broker-service-url "pulsar+ssl://localhost:6651/"
 
 
Here is the error message I get when launch the connection script :
 
d4672ef, L:/127.0.0.1:38370 ! R:localhost/127.0.0.1:6650] Disconnected
2022-12-29T15:16:15,345+0000 [pulsar-client-io-1-2] WARN  org.apache.pulsar.client.impl.ConnectionPool - [[id: 0xdd4672ef, L:/127.0.0.1:38370 ! R:localhost/127.0.0.1:6650]] Connection handshake failed: org.apache.pulsar.client.api.PulsarClientException: Connection already closed
2022-12-29T15:16:43,679+0000 [function-timer-thread-9-1] ERROR org.apache.pulsar.functions.runtime.RuntimeSpawner - public/default/debezium-mongodb-source Function Container is dead with following exception. Restarting.
java.util.concurrent.CompletionException: org.apache.pulsar.client.api.PulsarClientException: Connection already closed
        at java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:292) ~[?:1.8.0_345]
        at java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:308) ~[?:1.8.0_345]
        at java.util.concurrent.CompletableFuture.uniRun(CompletableFuture.java:714) ~[?:1.8.0_345]
        at java.util.concurrent.CompletableFuture$UniRun.tryFire(CompletableFuture.java:701) ~[?:1.8.0_345]
        at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) ~[?:1.8.0_345]
        at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1990) ~[?:1.8.0_345]
        at org.apache.pulsar.client.impl.ClientCnx.channelInactive(ClientCnx.java:267) ~[pulsar-client-original-2.10.1.jar:2.10.1]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:262) ~[netty-transport-4.1.77.Final.jar:4.1.77.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:248) ~[netty-transport-4.1.77.Final.jar:4.1.77.Final]
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:241) ~[netty-transport-4.1.77.Final.jar:4.1.77.Final]
        at io.netty.handler.codec.ByteToMessageDecoder.channelInputClosed(ByteToMessageDecoder.java:392) ~[netty-codec-4.1.77.Final.jar:4.1.77.Final]
        at io.netty.handler.codec.ByteToMessageDecoder.channelInactive(ByteToMessageDecoder.java:357) ~[netty-codec-4.1.77.Final.jar:4.1.77.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:262) ~[netty-transport-4.1.77.Final.jar:4.1.77.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:248) ~[netty-transport-4.1.77.Final.jar:4.1.77.Final]

I also tried another option on  https with the service url on port 6651, and I get teh erro below:
 
 
2022-12-29T15:30:19,126+0000 [main] INFO  org.apache.pulsar.functions.runtime.thread.ThreadRuntime - ThreadContainer starting function with instanceId 0 functionId 90dc9a02-6776-4ea9-a91d-f2cec73bce6a namespace default
2022-12-29T15:30:20,512+0000 [pulsar-client-io-1-2] INFO  org.apache.pulsar.client.impl.ConnectionPool - [[id: 0xd457af27, L:/127.0.0.1:48866 - R:localhost/127.0.0.1:6651]] Connected to server
2022-12-29T15:30:20,590+0000 [pulsar-client-io-1-2] WARN  org.apache.pulsar.client.impl.ClientCnx - Error during handshake
java.nio.channels.ClosedChannelException: null
        at io.netty.handler.ssl.SslHandler.channelInactive(SslHandler.java:1076) ~[netty-handler-4.1.48.Final.jar:4.1.48.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:262) ~[netty-transport-4.1.77.Final.jar:4.1.77.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:248) ~[netty-transport-4.1.77.Final.jar:4.1.77.Final]
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:241) ~[netty-transport-4.1.77.Final.jar:4.1.77.Final]
        at io.netty.channel.DefaultChannelPipeline$HeadContext.channelInactive(DefaultChannelPipeline.java:1405) ~[netty-transport-4.1.77.Final.jar:4.1.77.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:262) ~[netty-transport-4.1.77.Final.jar:4.1.77.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:248) ~[netty-transport-4.1.77.Final.jar:4.1.77.Final]
        at io.netty.channel.DefaultChannelPipeline.fireChannelInactive(DefaultChannelPipeline.java:901) ~[netty-transport-4.1.77.Final.jar:4.1.77.Final]
        at io.netty.channel.AbstractChannel$AbstractUnsafe$7.run(AbstractChannel.java:813) ~[netty-transport-4.1.77.Final.jar:4.1.77.Final]
        at io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java:174) ~[netty-common-4.1.77.Final.jar:4.1.77.Final]
        at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:167) ~[netty-common-4.1.77.Final.jar:4.1.77.Final]
        at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:470) ~[netty-common-4.1.77.Final.jar:4.1.77.Final]
        at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:384) ~[netty-transport-native-epoll-4.1.48.Final-linux-x86_64.jar:4.1.48.Final]
        at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:995) ~[netty-common-4.1.77.Final.jar:4.1.77.Final]
        at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[netty-common-4.1.77.Final.jar:4.1.77.Final]
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[netty-common-4.1.77.Final.jar:4.1.77.Final]
        at java.lang.Thread.run(Thread.java:750) ~[?:1.8.0_345]
2022-12-29T15:30:20,663+0000 [pulsar-client-io-1-2] INFO  org.apache.pulsar.client.impl.ClientCnx - [id: 0xd457af27, L:/127.0.0.1:48866 ! R:localhost/127.0.0.1:6651] Disconnected
2022-12-29T15:30:20,691+0000 [pulsar-client-io-1-2] WARN  org.apache.pulsar.client.impl.ConnectionPool - [[id: 0xd457af27, L:/127.0.0.1:48866 ! R:localhost/127.0.0.1:6651]] Connection handshake failed: org.apache.pulsar.client.api.PulsarClientException: Connection already closed
2022-12-29T15:30:49,134+0000 [function-timer-thread-9-1] ERROR org.apache.pulsar.functions.runtime.RuntimeSpawner - public/default/debezium-mongodb-source Function Container is dead with following exception. Restarting.
java.util.concurrent.CompletionException: org.apache.pulsar.client.api.PulsarClientException: Connection already closed
        at java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:292) ~[?:1.8.0_345]
        at java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:308) ~[?:1.8.0_345]
        at java.util.concurrent.CompletableFuture.uniRun(CompletableFuture.java:714) ~[?:1.8.0_345]
        at java.util.concurrent.CompletableFuture$UniRun.tryFire(CompletableFuture.java:701) ~[?:1.8.0_345]
        at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) ~[?:1.8.0_345]

Do you see the same behaviour using the latest relesead Debezium version?

(Ideally, also verify with latest Alpha/Beta/CR version)

Yes

Do you have the connector logs, ideally from start till finish?

(You might be asked later to provide DEBUG/TRACE level log)

<Your answer>

How to reproduce the issue using our tutorial deployment?

Yes

Feature request or enhancement

For feature requests or enhancements, provide this information, please:

Which use case/requirement will be addressed by the proposed feature?

<Your answer>

Implementation ideas (optional)

<Your answer>