Feature request or enhancement

Which use case/requirement will be addressed by the proposed feature?

More accurate metrics and also source.ts_ms for MySQL >= 8.0.1 with GTID replication mode enabled. At Shopify we're doubling down on lower latency business use cases for CDC and we're aware of the second resolution (int4) timestamp issue in binlog event headers for some time.

This issue and Github pull requests to the binlog client and Debezium is an attempt to engage with the community to report higher resolution metrics and source.ts_ms in a backwards compatible way.

Implementation ideas (optional)

I approached this from the GTID event perspective and not aware of another means to achieve this (injecting timestamps in SQL timestamps as a hack aside).

The binlog client PR is dense with lower level details, MySQL libbinlogevents references and also unpacks additional metadata exposed up until version 8.0.14.

Debezium PR: https://github.com/debezium/debezium/pull/5036

MySQL 8.0.1 introduced 2 new microsecond resolution timestamps on GTID events to track replication lag more accurately:

• immediate_commit_timestamp, microsecond resolution timestamp from the immediate master (or equal to original_commit_timestamp if connecting to master) since MySQL 8.0.1
• original_commit_timestamp, microsecond resolution timestamp of the commit on the originating master (or equal to original_commit_timestamp if connecting to master) since MySQL 8.0.1

original_commit_timestamp is representative of the original commit time on the MySQL writer.

The order of precedence is:

• IF GTID mode is enabled and GtidEventData::getOriginalCommitTimestamp returns a non-zero value (MySQL >= 8.0.1), use the high resolution timestamp as set by the writer
• Otherwise fallback to the second resolution timestamps

Other due diligence checks

• Snapshots set the current time in milliseconds for source.ts_ms already, I did not find any timestamp resolution issues with aligning on millisecond resolution for the streaming path as well