Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-3593

Transaction commit event dispatch fails if no active transaction in progress.

    XMLWordPrintable

Details

    Description

      We catch a bug with oracle connector. We used the versions stable 1.5 and beta 1.6.

      [2021-06-06
       21:12:32,369] WARN Error while processing event at offset 
       {transaction_id=null, 
       lcr_position=00000a1c4b4ced9d000000010000000100000a1c4b4ced18000000020000000102}
       (io.debezium.pipeline.EventDispatcher:234)
       [2021-06-06 21:12:32,370] ERROR Producer failure 
       (io.debezium.pipeline.ErrorHandler:31)
       org.apache.kafka.connect.errors.DataException: Invalid value: null used 
       for required field: "id", schema type: STRING
       at 
       org.apache.kafka.connect.data.ConnectSchema.validateValue(ConnectSchema.java:220)
       at org.apache.kafka.connect.data.Struct.put(Struct.java:216)
       at org.apache.kafka.connect.data.Struct.put(Struct.java:203)
       at 
       io.debezium.pipeline.txmetadata.TransactionMonitor.endTransaction(TransactionMonitor.java:170)
       at 
       io.debezium.pipeline.txmetadata.TransactionMonitor.transactionComittedEvent(TransactionMonitor.java:132)
       at 
       io.debezium.pipeline.EventDispatcher.dispatchTransactionCommittedEvent(EventDispatcher.java:249)
       at 
       io.debezium.connector.oracle.xstream.LcrEventHandler.dispatchDataChangeEvent(LcrEventHandler.java:107)
       at 
       io.debezium.connector.oracle.xstream.LcrEventHandler.processLCR(LcrEventHandler.java:86)
       at 
       oracle.streams.XStreamOut.XStreamOutReceiveLCRCallbackNative(Native 
       Method)
       at 
       oracle.streams.XStreamOut.receiveLCRCallback(XStreamOut.java:465)
       at 
       io.debezium.connector.oracle.xstream.XstreamStreamingChangeEventSource.execute(XstreamStreamingChangeEventSource.java:94)
       at 
       io.debezium.pipeline.ChangeEventSourceCoordinator.streamEvents(ChangeEventSourceCoordinator.java:140)
       at 
       io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:113)
       at 
       java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
       at 
       java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
       at 
       java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
       at 
       java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
       at java.base/java.lang.Thread.run(Thread.java:834)
      

      We also tried DEBUGGING and saw that the transaction_id in the messages from the database is not empty. The error also appears on other tables

       [2021-06-06 21:59:21,123] DEBUG Processing DML event DefaultRowLCR:
       Header:
       source database name: DB_NAME
       command type: COMMIT
       object owner:
       object name:
       tag: []
       position: [0 0 a 1c 4b 5d 5 ffffffd1 0 0 0 1 0 0 0 1 0 0 a 1c 4b 5d 5 ffffffd1 0 0 0 1 0 0 0 1 2]
       transaction id: 8.14.1587837
       source time: 2021-06-06 21:59:11
       scn: null
       commit scn: null
       lcr flags:
       Statement LCR: false
       IDKeyColumnsOnly LCR: false
       Sequence LCR: false
       Attribute[0]
       Old column values (0):
       New column values (0):
      

      FULL Connect log: connect.log
      DDL:

      create table table_name
      (
          ID                       NUMBER(38) not null
              constraint PK_TABLE_BID
                  primary key,
          ENTITY_ID                NUMBER(38),
          CREATED_DATE             DATE,
          DELETED_DATE             DATE,
          SUBMIT_DATE              DATE,
          LOT_ID                   NUMBER(10)
              constraint BID_LOT_ID
                  references D_LOT_ENTITY,
          REGISTRATION_NUMBER      NVARCHAR2(200),
          PARTICIPANT_ADDRESS      NVARCHAR2(1000),
          DENIAL_REASON            NVARCHAR2(1000),
          ACCORDANCE_REASON        NVARCHAR2(1000),
          BID_STATUS               NUMBER(10),
          BID_TYPE                 NUMBER(10),
          CONTRACT_PRICE           NUMBER(19, 2),
          LIFECYCLE_COST           NUMBER(19, 2),
          PARTICIPANT_POST_ADDRESS VARCHAR2(1020 char),
          INN                      VARCHAR2(80 char),
          KPP                      VARCHAR2(80 char),
          LEGAL_FORM_ID            NUMBER(10),
          CONTACT_PERSON           VARCHAR2(1020 char),
          EMAIL                    VARCHAR2(200 char),
          PHONE                    VARCHAR2(120 char),
          PARTICIPANT_SHORT_NAME   VARCHAR2(1000 char),
          POST_ADDRESS_IS_SAME     NUMBER(1),
          CUSTOMER_ID              NUMBER(10),
          NUM                      NUMBER(10),
          FINAL_RATING             NUMBER(19, 6),
          SUPPLIER_ID              NUMBER(10),
          DENIAL_REASON_ID         NUMBER(10),
          PARTICIPANT_TYPE         NUMBER(2) default '0',
          IS_NOT_PROVIDED          NUMBER(1) default '1',
          VERSION_USER_ID          NUMBER(10),
          RESULT_TYPE              VARCHAR2(100),
          LOADED_FROM_EIS          NUMBER(1) default 0,
          CONTRACT_REJREASON       VARCHAR2(100),
          CONTRACT_REJREASON_TEXT  VARCHAR2(1000 char),
          PARTICIPANT_NAME         VARCHAR2(1000 char),
          PLACE                    NUMBER(3)
      )
      /
      
      create index IDX_D_BID_LOT_ID
          on D_BID (LOT_ID)
      

      Attachments

        1. connect.log
          8 kB
        2. kafka_connect.log
          450 kB

        Activity

          People

            ccranfor@redhat.com Chris Cranford
            chapeauclaque Dmitriy Rum
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: