Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-1588

The ts_ms value is not correct during the snapshot processing

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Major Major
    • 1.0.0.Beta3
    • 0.9.5.Final
    • mysql-connector
    • None
    • Hide

      Steps:
      1. Submit a source task
      2. Submit a sink task
      3. Map the data to hive

      I think this issue is noting to do with the steps, because it will appear for every time

      Thanks

      Show
      Steps: 1. Submit a source task 2. Submit a sink task 3. Map the data to hive I think this issue is noting to do with the steps, because it will appear for every time Thanks

      The ts_ms described as "ts_ms is optional and if present contains the time (using the system clock in the JVM running the Kafka Connect task) at which the connector processed the event." from official documentation with link https://debezium.io/documentation/reference/0.9/connectors/mysql.html

      As it said ts_ms as the processed time fo the event, so I think the during the snapshot every record is an event, so the ts_ms values are not the same during the snapshot, but in fact the ts_ms is the same value, Also I have read the source code io.debezium.connector.mysql.SnapshotReader#execute the variable ts is assign the value from the beginning and use to:
      recorder.recordRow(recordMaker, row, ts); // has no row number!

      So the ts will not change during the snapshot processing, also agains with the official documentation.

              jpechane Jiri Pechanec
              yelijun Lijun Ye (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated:
                Resolved: