Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-1194

MySQL connector throw an exception when captured invalid datetime

    Details

      Description

      According to the PR debezium/debezium#114, in case of the "zero-value" timestamp column type, the Java date will convert to epoch time as 1970-01-01, but the datetime column type (not null) - the connector will throw an exception like bellow.

      org.apache.kafka.connect.errors.ConnectException: Tolerance exceeded in error handler
          at org.apache.kafka.connect.runtime.errors.RetryWithToleranceOperator.execAndHandleError(RetryWithToleranceOperator.java:178)
          at org.apache.kafka.connect.runtime.errors.RetryWithToleranceOperator.execute(RetryWithToleranceOperator.java:104)
          at org.apache.kafka.connect.runtime.WorkerSourceTask.convertTransformedRecord(WorkerSourceTask.java:266)
          at org.apache.kafka.connect.runtime.WorkerSourceTask.sendRecords(WorkerSourceTask.java:293)
          at org.apache.kafka.connect.runtime.WorkerSourceTask.execute(WorkerSourceTask.java:228)
          at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:175)
          at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:219)
          at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
          at java.util.concurrent.FutureTask.run(FutureTask.java:266)
          at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
          at java.lang.Thread.run(Thread.java:748)
      Caused by: org.apache.kafka.connect.errors.DataException: Conversion error: null value for field that is required and has no default value
          at org.apache.kafka.connect.json.JsonConverter.convertToJson(JsonConverter.java:586)
          at org.apache.kafka.connect.json.JsonConverter.convertToJson(JsonConverter.java:680)
          at org.apache.kafka.connect.json.JsonConverter.convertToJsonWithEnvelope(JsonConverter.java:567)
          at org.apache.kafka.connect.json.JsonConverter.fromConnectData(JsonConverter.java:320)
          at org.apache.kafka.connect.runtime.WorkerSourceTask.lambda$convertTransformedRecord$1(WorkerSourceTask.java:266)
          at org.apache.kafka.connect.runtime.errors.RetryWithToleranceOperator.execAndRetry(RetryWithToleranceOperator.java:128)
          at org.apache.kafka.connect.runtime.errors.RetryWithToleranceOperator.execAndHandleError(RetryWithToleranceOperator.java:162)
          ... 11 more
      

      The connector knows the datetime column is mandatory, but the debezium connector always convert "zero-value" as nulled value, then the connect schema validate again and it will throw an exception.

        Gliffy Diagrams

          Attachments

            Activity

              People

              • Assignee:
                jpechanec Jiri Pechanec
                Reporter:
                hongphu.it NGUYEN HONG PHU
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: