Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-6388

__source_ts_ms r (read) operation date is set to future for SQL Server

XMLWordPrintable

    • False
    • None
    • False
    • Important

      In order to make your issue reports as actionable as possible, please provide the following information, depending on the issue type.

      Bug report

      For bug reports, provide this information, please:

      What Debezium connector do you use and what version?

      debezium-server-dist-2.1.3.Final-runner.jar

      What is the connector configuration?

      debezium.sink.type=http
      debezium.sink.http.url=http://127.0.0.1:5555
      debezium.source.database.hostname=<READACTED>
      debezium.source.database.port=1433
      debezium.source.database.user=<READACTED>
      debezium.source.database.password=<READACTED>
      debezium.source.database.names=aw2019
      debezium.source.database.encrypt=true
      debezium.source.database.trustServerCertificate=true

      debezium.source.table.include.list=Person.ContactType5

      debezium.source.connector.class=io.debezium.connector.sqlserver.SqlServerConnector
      debezium.source.offset.storage.file.filename=offsets.dat
      debezium.source.offset.flush.interval.ms=0

      debezium.source.schema.history.internal=io.debezium.storage.file.history.FileSchemaHistory
      debezium.source.schema.history.internal.file.filename=FileDatabaseHistory.dat

      quarkus.log.console.json=false

      1. quarkus.log.level=FINEST
        quarkus.log.level=TRACE
        quarkus.naming.enable-jndi=true

      debezium.source.transforms=unwrap
      debezium.source.transforms.unwrap.type=io.debezium.transforms.ExtractNewRecordState
      debezium.source.transforms.unwrap.delete.handling.mode=rewrite
      debezium.source.transforms.unwrap.add.fields=op,table,source.ts_ms
      debezium.source.transforms.unwrap.add.headers=db
      debezium.source.key.converter.schemas.enable=false
      debezium.source.value.converter.schemas.enable=false

      debezium.format.value=json

      What is the captured database version and mode of depoyment?

      On premise Debezium Server, just staring as ./run.sh

       
      if [ z "$JAVA_HOME" ]; thenJAVA_BINARY="java"elseJAVA_BINARY="$JAVA_HOME/bin/java"fiif [ "$OSTYPE" = "msys" ] || [ "$OSTYPE" = "cygwin" ]; thenPATH_SEP=";"elsePATH_SEP=":"fiRUNNER=$(ls debezium-serverrunner.jar)ENABLE_DEBEZIUM_SCRIPTING=${ENABLE_DEBEZIUM_SCRIPTING:-false}{}LIB_PATH="lib/"if [[ "${ENABLE_DEBEZIUM_SCRIPTING}" == "true" ]]; thenLIB_PATH=$LIB_PATH$PATH_SEP"lib_opt/*"fiexec "$JAVA_BINARY" $DEBEZIUM_OPTS $JAVA_OPTS -cp "$RUNNER"$PATH_SEP"conf"$PATH_SEP$LIB_PATH io.debezium.server.Main

      What behaviour do you expect?

      The read operations time ({}source_ts_ms) value is same as current time or UTC time.

      What behaviour do you see?

      The __source_ts_ms value for read operations seems to be set in the future. 

       

      Please see screenshot png with example. The R operation happens first, date is in the future, the U happens soon after and date is correct.

      More information https://groups.google.com/g/debezium/c/NrHQvmeuwh0 

      Do you see the same behaviour using the latest relesead Debezium version?

      Yes, the debezium-server-dist-2.2.0.Final-runner has the same issue.

      Do you have the connector logs, ideally from start till finish?

      Yes, trace log attached.trace3.log

      How to reproduce the issue using our tutorial deployment?

      Run debezium-server-dist-2.1.3.Final-runner.jar with provided properties file. It works with any table in SQL Server that has CDC enabled and with GCP pubsub and http sinks. 

      Feature request or enhancement

      For feature requests or enhancements, provide this information, please:

      R operation suppose to have correct timestamp.

      Which use case/requirement will be addressed by the proposed feature?

      <Your answer>

      Implementation ideas (optional)

      <Your answer>

        1. Screenshot 2023-04-25 at 2.31.23 PM.png
          54 kB
          Liudas Sodonis
        2. trace3.log
          122 kB
          Liudas Sodonis

            Unassigned Unassigned
            liudas.fastloop Liudas Sodonis (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated:
              Resolved: