Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-3534

LogMiner DELETE operation by ROWID causes DataException

    XMLWordPrintable

Details

    • Bug
    • Resolution: Duplicate
    • Major
    • None
    • 1.5.0.Final, 1.6.0.Alpha1
    • oracle-connector
    • None
    • False
    • False
    • Undefined

    Description

      When a DELETE operation is mined as

      delete from \"DBO\".\"CHANNELS\" where ROWID = 'AAA0DUABEAAHxMoAAH'
      

      will lead to

      {"exception":"org.apache.kafka.connect.errors.DataException: Invalid value: null used for required field: \"CHANNELID\", schema type: INT32\n\tat org.apache.kafka.connect.data.ConnectSchema.validateValue(ConnectSchema.java:220)\n\tat org.apache.kafka.connect.data.Struct.validate(Struct.java:233)\n\tat org.apache.kafka.connect.data.ConnectSchema.validateValue(ConnectSchema.java:253)\n\tat org.apache.kafka.connect.data.Struct.put(Struct.java:216)\n\tat org.apache.kafka.connect.data.Struct.put(Struct.java:203)\n\tat io.debezium.data.Envelope.delete(Envelope.java:341)\n\tat io.debezium.relational.RelationalChangeRecordEmitter.emitDeleteRecord(RelationalChangeRecordEmitter.java:140)\n\tat io.debezium.relational.RelationalChangeRecordEmitter.emitChangeRecords(RelationalChangeRecordEmitter.java:54)\n\tat io.debezium.pipeline.EventDispatcher.dispatchDataChangeEvent(EventDispatcher.java:202)\n\t... 12 common frames omitted\nWrapped by: org.apache.kafka.connect.errors.ConnectException: Error while processing event at offset {commit_scn=26432350098, transaction_id=null, scn=26432406569}\n\tat io.debezium.pipeline.EventDispatcher.dispatchDataChangeEvent(EventDispatcher.java:232)\n\tat io.debezium.connector.oracle.logminer.LogMinerQueryResultProcessor.lambda$processResult$0(LogMinerQueryResultProcessor.java:230)\n\tat io.debezium.connector.oracle.logminer.TransactionalBuffer.commit(TransactionalBuffer.java:154)\n\t... 10 common frames omitted\nWrapped by: org.apache.kafka.connect.errors.ConnectException: An exception occurred in the change event producer. This connector will be stopped.\n\tat io.debezium.pipeline.ErrorHandler.setProducerThrowable(ErrorHandler.java:42)\n\tat io.debezium.connector.oracle.logminer.TransactionalBuffer.commit(TransactionalBuffer.java:168)\n\tat io.debezium.connector.oracle.logminer.TransactionalBuffer.commit(TransactionalBuffer.java:141)\n\tat io.debezium.connector.oracle.logminer.LogMinerQueryResultProcessor.processResult(LogMinerQueryResultProcessor.java:145)\n\tat io.debezium.connector.oracle.logminer.LogMinerStreamingChangeEventSource.execute(LogMinerStreamingChangeEventSource.java:187)\n\tat io.debezium.pipeline.ChangeEventSourceCoordinator.streamEvents(ChangeEventSourceCoordinator.java:140)\n\tat io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:113)\n\tat java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)\n\tat java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)\n\tat java.base/java.lang.Thread.run(Thread.java:834)\n","level":"ERROR","logger":"io.debezium.embedded.EmbeddedEngine","thread":"pool-19-thread-1","message":"Error while trying to run connector class 'io.debezium.connector.oracle.OracleConnector'","mdc":{"DEBEZIUM_PRODUCT_KEY":"core-bt-dev-bt-alpha-1"},"timestamp":"2021-05-18T20:17:36.529Z"} Tue May 18 20:17:36 2021 datalake-debezium-service
      datalake-debezium-service alpha
      alpha 10-64-17-36 cvt_alpha ERROR 1 io.debezium.embedded.EmbeddedEngine
      io.debezium.embedded.EmbeddedEngine /services/cvt/alpha/alpha/datalake-debezium-service/logs/json/datalake-debezium.0ecdd3da1bed.log json mg01-spkidx-102.core.cvent.org
      

      This is because the connector isn't able to resolve any before state and therefore the event is emitted without specifying any values, including the primary key. The connector already logs a warning about this use case but we likely might want to avoid emitting an event for this use case so that the validation that failed isn't triggered since we have no state to provide.

      See https://gitter.im/debezium/user?at=60a42323233397424dce812e

      Attachments

        Issue Links

          Activity

            People

              ccranfor@redhat.com Chris Cranford
              ccranfor@redhat.com Chris Cranford
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: