-
Enhancement
-
Resolution: Done
-
Major
-
None
-
None
-
False
-
None
-
False
Hi Team,
We're asking for additional features for Incremental Snapshot in Connectors MongoDB.
Bug report
- The official documentation was explaining incremental snapshot using Relational database syntax, which is not applicable to incremental snapshot that uses additional conditions.
- throws UnsupportedOperationException when trying to use additional conditions (https://github.com/debezium/debezium/blob/main/debezium-connector-mongodb/src/main/java/io/debezium/connector/mongodb/MongoDbIncrementalSnapshotChangeEventSource.java#L409)
- Unable to get complete data when querying to an inconsistent PK data type of a collection
What Debezium connector do you use and what version?
Debezium Server 2.4.0.Final
What is the connector configuration?
debezium.format.value=json
debezium.source.connector.class=io.debezium.connector.mongodb.MongoDbConnector
debezium.source.database.history=io.debezium.relational.history.FileDatabaseHistory
debezium.source.tasks.max=1
debezium.source.topic.prefix=cdc
debezium.source.topic.heartbeat.prefix=cdc
debezium.source.mongodb.connection.string=mongodb://cdc:cdc@mongodb:27017/inventory?replicaSet=rs0&authSource=admin debezium.source.database.include.list=inventory
debezium.source.collection.include.list=inventory.customers
debezium.source.signal.data.collection=inventory.debezium_signal
What is the captured database version and mode of deployment?
MongoDB: 6
Do you see the same behaviour using the latest relesead Debezium version?
Yes
How to reproduce the issue using our tutorial deployment?
- Execute debezium_signal with additional-conditions
- Insert 2 records, where the 1st record has _id (int), and the 2nd one is _id (ObjectId). Do an incremental snapshot and you will only get one instead of two records.
Feature request or enhancement
Solving this issue will require us to enable additional-conditions, since we could snapshot based on the data types.
- links to
-
RHEA-2024:139598 Red Hat build of Debezium 2.5.4 release
Since the problem described in this issue should be resolved in a recent advisory, it has been closed.
For information on the advisory (Red Hat build of Debezium 2.7.3 release), and where to find the updated files, follow the link below.
If the solution does not work for you, open a new bug report.
https://access.redhat.com/errata/RHEA-2024:7859