Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-7615

Enhanced event timestamp precision combined with ExtractNewRecordState not working

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Major Major
    • 2.6.0.CR1
    • 2.6.0.Beta1
    • core-library
    • None

      What Debezium connector do you use and what version?

      Oracle Connector 2.6.0.Beta1

      What is the connector configuration?

      "transforms": "changes,moveHeadersToValue,unwrap",
      "transforms.changes.type": "io.debezium.transforms.ExtractChangedRecordState", 
      "transforms.changes.header.changed.name": "Changed", 
      "transforms.moveHeadersToValue.type": "io.debezium.transforms.HeaderToValue", 
      "transforms.moveHeadersToValue.headers": "Changed", 
      "transforms.moveHeadersToValue.fields": "changes", 
      "transforms.moveHeadersToValue.operation": "move", 
      "transforms.unwrap.type": "io.debezium.transforms.ExtractNewRecordState", 
      "transforms.unwrap.add.fields": "source.table:META_SRC_TABLENAME,source.ts_ms:META_SRC_TS,source.ts_us:META_SRC_US,ts_us:META_US,ts_ns:META_NS,op:META_SRC_OP,source.scn:META_SRC_SCN,source.snapshot:META_SRC_SNAPSHOT,source.user_name:META_SRC_USER,changes:META_SRC_CHANGES", 
      "transforms.unwrap.delete.handling.mode": "rewrite", 
      "transforms.unwrap.drop.tombstones": "false",

      I think the rest of the configuration is not required.

      What is the captured database version and mode of depoyment?

      Oracle 19c CDB on prem
      In Kafka Connect we run Java OpenJDK11 (so nanoseconds aren't supported yet)

      What behaviour do you expect?

      "ts_ms": 1709889596702, 
      "ts_us": 1709889596702080, 
      "ts_ns": 1709889596702080000

      The envelope's new ts_us should deliver timestamps with microseconds when used with ExtractNewRecordState. 

      What behaviour do you see?

      "__deleted": "false", 
      "__META_SRC_TABLENAME": "ANTRFRAGENANTW", 
      "__META_SRC_TS_MS": 1709891779000,
      "__META_SRC_TS_US": 1709891779000000,
      "__META_SRC_TS_NS": 1709891779000000000, 
      "__META_TS_MS": 1709891800550, 
      "__META_TS_US": 1709891779000000, 
      "__META_TS_NS": 1709891779000000000, 
      "__META_SRC_OP": "r", 
      "__META_SRC_SCN": "328638752686", 
      "__META_SRC_SNAPSHOT": "true"

      Combined with the ExtractNewRecordState SMT the envelope's ts_us and ts_ns have the same values as in the source's ts_us and ts_ns. 

      The envelope's ts_ms seems to be correct. Without the ExtractNewRecordState all works fine.

       

              ccranfor@redhat.com Chris Cranford
              renerh René Rütter
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Created:
                Updated:
                Resolved: