Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-1588

The ts_ms value is not correct during the snapshot processing

    XMLWordPrintable

Details

    • Bug
    • Resolution: Done
    • Major
    • 1.0.0.Beta3
    • 0.9.5.Final
    • mysql-connector
    • None
    • Hide

      Steps:
      1. Submit a source task
      2. Submit a sink task
      3. Map the data to hive

      I think this issue is noting to do with the steps, because it will appear for every time

      Thanks

      Show
      Steps: 1. Submit a source task 2. Submit a sink task 3. Map the data to hive I think this issue is noting to do with the steps, because it will appear for every time Thanks

    Description

      The ts_ms described as "ts_ms is optional and if present contains the time (using the system clock in the JVM running the Kafka Connect task) at which the connector processed the event." from official documentation with link https://debezium.io/documentation/reference/0.9/connectors/mysql.html

      As it said ts_ms as the processed time fo the event, so I think the during the snapshot every record is an event, so the ts_ms values are not the same during the snapshot, but in fact the ts_ms is the same value, Also I have read the source code io.debezium.connector.mysql.SnapshotReader#execute the variable ts is assign the value from the beginning and use to:
      recorder.recordRow(recordMaker, row, ts); // has no row number!

      So the ts will not change during the snapshot processing, also agains with the official documentation.

      Attachments

        Activity

          People

            jpechane Jiri Pechanec
            yelijun Lijun Ye (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: