Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-1588

The ts_ms value is not correct during the snapshot processing

    Details

    • Type: Bug
    • Status: Closed (View Workflow)
    • Priority: Major
    • Resolution: Done
    • Affects Version/s: 0.9.5.Final
    • Fix Version/s: 1.0.0.Beta3
    • Component/s: mysql-connector
    • Labels:
      None
    • Environment:

      OS: centos 7, X86_64
      Mysql: 5.7
      JDBC driver: com.mysql.jdbc.Driver

    • Steps to Reproduce:
      Hide

      Steps:
      1. Submit a source task
      2. Submit a sink task
      3. Map the data to hive

      I think this issue is noting to do with the steps, because it will appear for every time

      Thanks

      Show
      Steps: 1. Submit a source task 2. Submit a sink task 3. Map the data to hive I think this issue is noting to do with the steps, because it will appear for every time Thanks

      Description

      The ts_ms described as "ts_ms is optional and if present contains the time (using the system clock in the JVM running the Kafka Connect task) at which the connector processed the event." from official documentation with link https://debezium.io/documentation/reference/0.9/connectors/mysql.html

      As it said ts_ms as the processed time fo the event, so I think the during the snapshot every record is an event, so the ts_ms values are not the same during the snapshot, but in fact the ts_ms is the same value, Also I have read the source code io.debezium.connector.mysql.SnapshotReader#execute the variable ts is assign the value from the beginning and use to:
      recorder.recordRow(recordMaker, row, ts); // has no row number!

      So the ts will not change during the snapshot processing, also agains with the official documentation.

        Gliffy Diagrams

          Attachments

            Activity

              People

              • Assignee:
                jpechanec Jiri Pechanec
                Reporter:
                yelijun Lijun Ye
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: