Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-1430

SMT ExtractNewDocumentState does not support Heartbeat events

    Details

      Description

      The version 0.10.0.Beta3 introduced the heartbeat functionality to the MongoDB Connector, via DBZ-1198

      The SMT for extracting the new document doesn't support the messages which might be routed from the heartbeat, thus breaking it:

      kafka-connect_1               | 2019-08-14 12:50:42,083 ERROR  ||  WorkerSourceTask{id=usabilla-cdc-accounts-0} Task threw an uncaught and unrecoverable exception   [org.apache.kafka.connect.runtime.WorkerTask]
      kafka-connect_1               | org.apache.kafka.connect.errors.ConnectException: Tolerance exceeded in error handler
      kafka-connect_1               | 	at org.apache.kafka.connect.runtime.errors.RetryWithToleranceOperator.execAndHandleError(RetryWithToleranceOperator.java:178)
      kafka-connect_1               | 	at org.apache.kafka.connect.runtime.errors.RetryWithToleranceOperator.execute(RetryWithToleranceOperator.java:104)
      kafka-connect_1               | 	at org.apache.kafka.connect.runtime.TransformationChain.apply(TransformationChain.java:50)
      kafka-connect_1               | 	at org.apache.kafka.connect.runtime.WorkerSourceTask.sendRecords(WorkerSourceTask.java:293)
      kafka-connect_1               | 	at org.apache.kafka.connect.runtime.WorkerSourceTask.execute(WorkerSourceTask.java:229)
      kafka-connect_1               | 	at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:175)
      kafka-connect_1               | 	at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:219)
      kafka-connect_1               | 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
      kafka-connect_1               | 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      kafka-connect_1               | 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
      kafka-connect_1               | 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
      kafka-connect_1               | 	at java.lang.Thread.run(Thread.java:748)
      kafka-connect_1               | Caused by: java.lang.NullPointerException
      kafka-connect_1               | 	at org.apache.kafka.connect.transforms.ExtractField.apply(ExtractField.java:61)
      kafka-connect_1               | 	at io.debezium.connector.mongodb.transforms.ExtractNewDocumentState.apply(ExtractNewDocumentState.java:142)
      kafka-connect_1               | 	at org.apache.kafka.connect.runtime.TransformationChain.lambda$apply$0(TransformationChain.java:50)
      kafka-connect_1               | 	at org.apache.kafka.connect.runtime.errors.RetryWithToleranceOperator.execAndRetry(RetryWithToleranceOperator.java:128)
      kafka-connect_1               | 	at org.apache.kafka.connect.runtime.errors.RetryWithToleranceOperator.execAndHandleError(RetryWithToleranceOperator.java:162)
      kafka-connect_1               | 	... 11 more
      kafka-connect_1               | 2019-08-14 12:50:42,091 ERROR  ||  WorkerSourceTask{id=usabilla-cdc-accounts-0} Task is being killed and will not recover until manually restarted   [org.apache.kafka.connect.runtime.WorkerTask]
      

      I propose to use a similar approach to the other SMTs, like DBZ-1388

        Gliffy Diagrams

          Attachments

            Activity

              People

              • Assignee:
                renatomefi Renato Mefi
                Reporter:
                renatomefi Renato Mefi
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: