-
Enhancement
-
Resolution: Done
-
Minor
-
None
-
None
-
False
-
-
False
Bug report
MongoDB connector should use wallTime instead of clusterTime for source.ts_ms in change stream events to accurately represent the database operation time.
What Debezium connector do you use and what version?
According to Debezium documentation, source.ts_ms should indicate "the time that the change was made in the database." For MongoDB 6.0+ change streams, source.ts_ms should use wallTime (actual wall-clock time of the database operation) rather than clusterTime (logical oplog timestamp).
Also this ensures accurate lag calculation (ts_ms - source.ts_ms) by measuring the time between the actual database change and Debezium processing.
What is the connector configuration?
NA
What is the captured database version and mode of deployment?
Mongo 8.0.15, onprem
What behavior do you expect?
source.ts_ms should use wallTime from MongoDB change streams
What behavior do you see?
source.ts_ms shows clusterTime from MongoDB change streams
Do you see the same behaviour using the latest released Debezium version?
Yes
Do you have the connector logs, ideally from start till finish?
NA
Implementation ideas (optional)
- Modify SourceInfo.timestamp() to prefer wallTime over clusterTime when available
- Fall back to clusterTime when wallTime is not available (MongoDB < 6.0 or when not provided)
- The wallTime field is already being extracted from ChangeStreamDocument in SourceInfo.changeStreamEvent()