Details
-
Bug
-
Resolution: Unresolved
-
Major
-
None
-
None
-
False
-
None
-
False
Description
Bug report
What Debezium connector do you use and what version?
1.8
What is the connector configuration?
mongodb connector
What is the captured database version and mode of depoyment?
mongodb 4.2 sharded cluster
What behaviour do you expect?
the message in kafka topic for sharded collection, key payload should be like :
"payload": { "id": "\"$oid\": \"5d2974673484856dfa2b909a\"" }
What behaviour do you see?
the message in kafka topic for sharded collection, key payload now is :
"payload": { "id": "\":\"201907130000200001\",\"_id\":{\"$oid\": \"5d2974673484856dfa2b909a\"}" }
In change streams, not like normal collection, for sharded collection, mongodb add 'sharded key' in 'documentKey'
// For normal collection "documentKey": { "_id": { "$oid": "62f9e6bcbb4300004f0004cb" } }, // For sharded collection "documentKey": { "caseNo": "201907130000200001", "_id": { "$oid": "5d2974673484856dfa2b909a" } },
The source code in getDocumentIdChangeStream handles only normal collection documentKey。
public String getDocumentIdChangeStream(BsonDocument document) { if (document == null) { return null; } // The serialized value is in format {"_": xxx} so we need to remove the starting dummy field name and closing brace final String keyValue = document.toJson(SIMPLE_JSON_SETTINGS); final int start = 8; final int end = keyValue.length() - 1; if (!(end > start)) { throw new IllegalStateException("Serialized JSON object '" + keyValue + "' is not in expected format"); } return keyValue.substring(start, end); }
