-
Enhancement
-
Resolution: Done
-
Major
-
None
-
None
-
False
-
None
-
False
Hi Team,
We're asking for additional features for Incremental Snapshot in Connectors MongoDB.
Bug report
- The official documentation was explaining incremental snapshot using Relational database syntax, which is not applicable to incremental snapshot that uses additional conditions.
- throws UnsupportedOperationException when trying to use additional conditions (https://github.com/debezium/debezium/blob/main/debezium-connector-mongodb/src/main/java/io/debezium/connector/mongodb/MongoDbIncrementalSnapshotChangeEventSource.java#L409)
- Unable to get complete data when querying to an inconsistent PK data type of a collection
What Debezium connector do you use and what version?
Debezium Server 2.4.0.Final
What is the connector configuration?
debezium.format.value=json
debezium.source.connector.class=io.debezium.connector.mongodb.MongoDbConnector
debezium.source.database.history=io.debezium.relational.history.FileDatabaseHistory
debezium.source.tasks.max=1
debezium.source.topic.prefix=cdc
debezium.source.topic.heartbeat.prefix=cdc
debezium.source.mongodb.connection.string=mongodb://cdc:cdc@mongodb:27017/inventory?replicaSet=rs0&authSource=admin debezium.source.database.include.list=inventory
debezium.source.collection.include.list=inventory.customers
debezium.source.signal.data.collection=inventory.debezium_signal
What is the captured database version and mode of deployment?
MongoDB: 6
Do you see the same behaviour using the latest relesead Debezium version?
Yes
How to reproduce the issue using our tutorial deployment?
- Execute debezium_signal with additional-conditions
- Insert 2 records, where the 1st record has _id (int), and the 2nd one is _id (ObjectId). Do an incremental snapshot and you will only get one instead of two records.
Feature request or enhancement
Solving this issue will require us to enable additional-conditions, since we could snapshot based on the data types.
- links to
-
RHEA-2024:139598 Red Hat build of Debezium 2.5.4 release