Details
-
Bug
-
Resolution: Unresolved
-
Major
-
1.8.1.Final
-
None
-
False
-
None
-
False
-
Important
Description
Bug report
Getting this NullPointerException intermittently in a production deployment over an on-premise mongo cluster.
What Debezium connector do you use and what version?
MongoDB Debezium Connector. Version = 1.8.1
What is the connector configuration?
{
"connector.class": "io.debezium.connector.mongodb.MongoDbConnector",
"collection.include.list": "****",
"producer.override.compression.type": "snappy",
"mongodb.password": "******",
"mongodb.user": "*****",
"mongodb.name": "******",
"value.subject.name.strategy": "io.confluent.kafka.serializers.subject.TopicRecordNameStrategy",
"tombstones.on.delete": "false",
"value.converter.schema.registry.url": "http://schema-registry3:8081",
"name": "debezium_2022_10_18",
"value.converter": "io.confluent.connect.avro.AvroConverter",
"mongodb.hosts": "****",
"key.converter": "io.confluent.connect.avro.AvroConverter",
"key.converter.schema.registry.url": "http://schema-registry3:8081",
"database.include.list": "****"
}
What are the captured database version and mode of deployment?
On-premise deployment. MongoDB Version: 5.0.0
What behavior do you expect?
There should not be any NullPointerException
What behavior do you see?
I get a NullPointerException
Do you see the same behavior using the latest released Debezium version?
I can't deploy the latest version in production.
Do you have the connector logs, ideally from the start until finish?
[2022-11-03 08:29:21,225] INFO [debezium_2022_10_18|task-0|offsets] WorkerSourceTask{id=debezium_2022_10_18-0} Either no records were produced by the task since the last offset commit, or every record has been filtered out by a transformation or dropped due to transformation or conversion errors. (org.apache.kafka.connect.runtime.WorkerSourceTask:484)
[2022-11-03 08:29:22,889] INFO [Worker clientId=connect-1, groupId=debezium-cluster] Connector debezium_2022_10_18 target state change (org.apache.kafka.connect.runtime.distributed.DistributedHerder:1711)
[2022-11-03 08:29:22,890] INFO Setting connector debezium_2022_10_18 state to STARTED (org.apache.kafka.connect.runtime.Worker:946)
[2022-11-03 08:29:22,890] INFO [debezium_2022_10_18|task-0] Starting MongoDbConnectorTask with configuration: (io.debezium.connector.common.BaseSourceTask:124)
[2022-11-03 08:29:22,890] INFO [debezium_2022_10_18|task-0] connector.class = io.debezium.connector.mongodb.MongoDbConnector (io.debezium.connector.common.BaseSourceTask:126)
[2022-11-03 08:29:22,890] INFO [debezium_2022_10_18|task-0] collection.include.list = database_1.table_1 (io.debezium.connector.common.BaseSourceTask:126)
[2022-11-03 08:29:22,890] INFO [debezium_2022_10_18|task-0] producer.override.compression.type = snappy (io.debezium.connector.common.BaseSourceTask:126)
[2022-11-03 08:29:22,890] INFO [debezium_2022_10_18|task-0] mongodb.password = *** (io.debezium.connector.common.BaseSourceTask:126)
[2022-11-03 08:29:22,890] INFO [debezium_2022_10_18|task-0] mongodb.user = *** (io.debezium.connector.common.BaseSourceTask:126)
[2022-11-03 08:29:22,890] INFO [debezium_2022_10_18|task-0] mongodb.name = user_1 (io.debezium.connector.common.BaseSourceTask:126)
[2022-11-03 08:29:22,890] INFO [debezium_2022_10_18|task-0] value.subject.name.strategy = io.confluent.kafka.serializers.subject.TopicRecordNameStrategy (io.debezium.connector.common.BaseSourceTask:126)
[2022-11-03 08:29:22,890] INFO [debezium_2022_10_18|task-0] tombstones.on.delete = false (io.debezium.connector.common.BaseSourceTask:126)
[2022-11-03 08:29:22,890] INFO [debezium_2022_10_18|task-0] value.converter.schema.registry.url = http://schema-registry:8081 (io.debezium.connector.common.BaseSourceTask:126)
[2022-11-03 08:29:22,890] INFO [debezium_2022_10_18|task-0] task.class = io.debezium.connector.mongodb.MongoDbConnectorTask (io.debezium.connector.common.BaseSourceTask:126)
[2022-11-03 08:29:22,890] INFO [debezium_2022_10_18|task-0] mongodb.task.id = 0 (io.debezium.connector.common.BaseSourceTask:126)
[2022-11-03 08:29:22,890] INFO [debezium_2022_10_18|task-0] name = debezium_2022_10_18 (io.debezium.connector.common.BaseSourceTask:126)
[2022-11-03 08:29:22,890] INFO [debezium_2022_10_18|task-0] value.converter = io.confluent.connect.avro.AvroConverter (io.debezium.connector.common.BaseSourceTask:126)
[2022-11-03 08:29:22,890] INFO [debezium_2022_10_18|task-0] mongodb.hosts = **** (io.debezium.connector.common.BaseSourceTask:126)
[2022-11-03 08:29:22,890] INFO [debezium_2022_10_18|task-0] key.converter = io.confluent.connect.avro.AvroConverter (io.debezium.connector.common.BaseSourceTask:126)
[2022-11-03 08:29:22,890] INFO [debezium_2022_10_18|task-0] key.converter.schema.registry.url = http://schema-registry:8081 (io.debezium.connector.common.BaseSourceTask:126)
[2022-11-03 08:29:22,890] INFO [debezium_2022_10_18|task-0] database.include.list = database_1 (io.debezium.connector.common.BaseSourceTask:126)
[2022-11-03 08:29:22,893] ERROR [debezium_2022_10_18|task-0] WorkerSourceTask{id=debezium_2022_10_18-0} Task threw an uncaught and unrecoverable exception. Task is being killed and will not recover until manually restarted (org.apache.kafka.connect.runtime.WorkerTask:195)
java.lang.NullPointerException
at io.debezium.connector.mongodb.MongoDbIncrementalSnapshotContext.init(MongoDbIncrementalSnapshotContext.java:178)
at io.debezium.connector.mongodb.MongoDbIncrementalSnapshotContext.load(MongoDbIncrementalSnapshotContext.java:276)
at io.debezium.connector.mongodb.MongoDbOffsetContext$Loader.loadOffsets(MongoDbOffsetContext.java:167)
at io.debezium.connector.mongodb.MongoDbConnectorTask.getPreviousOffset(MongoDbConnectorTask.java:199)
at io.debezium.connector.mongodb.MongoDbConnectorTask.start(MongoDbConnectorTask.java:83)
at io.debezium.connector.common.BaseSourceTask.start(BaseSourceTask.java:130)
at org.apache.kafka.connect.runtime.WorkerSourceTask.initializeAndStart(WorkerSourceTask.java:225)
at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:186)
at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:243)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
[2022-11-03 08:29:22,894] INFO [debezium_2022_10_18|task-0] Stopping down connector (io.debezium.connector.common.BaseSourceTask:238)
[2022-11-03 08:29:22,894] INFO [debezium_2022_10_18|task-0] [Producer clientId=connector-producer-debezium_2022_10_18-0] Closing the Kafka producer with timeoutMillis = 30000 ms. (org.apache.kafka.clients.producer.KafkaProducer:1228)
