Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-5712

Oracle Connector - Nested Tables Producer failure (io.debezium.pipeline.ErrorHandler) io.debezium.DebeziumException: java.lang.NullPointerException

    XMLWordPrintable

Details

    • Bug
    • Resolution: Duplicate
    • Major
    • None
    • 1.9.5.Final
    • oracle-connector
    • None
    • False
    • None
    • False

    Description

      In order to make your issue reports as actionable as possible, please provide the following information, depending on the issue type.

      Bug report

      For bug reports, provide this information, please:

      I'm trying to create an initial snapshot on my oracle 11g database and I encounter the following error during the snapshot process. 

       

      [2022-10-11 12:00:54,406] INFO Capturing structure of table PRODSTAR.SYSMAN.MGMT_DELTA_SAVED_COMP
      [2022-10-11 12:00:54,438] INFO Capturing structure of table PRODSTAR.SYSMAN.MGMT_DELTA_SNAP (io.d
      [2022-10-11 12:00:54,469] INFO Capturing structure of table PRODSTAR.SYSMAN.MGMT_DELTA_SUMMARY_ER
      [2022-10-11 12:00:54,470] INFO Snapshot - Final stage (io.debezium.pipeline.source.AbstractSnapsh
      [2022-10-11 12:00:54,470] ERROR Producer failure (io.debezium.pipeline.ErrorHandler)
      io.debezium.DebeziumException: java.lang.NullPointerException
              at io.debezium.pipeline.source.AbstractSnapshotChangeEventSource.execute(AbstractSnapshot
              at io.debezium.pipeline.ChangeEventSourceCoordinator.doSnapshot(ChangeEventSourceCoordina
              at io.debezium.pipeline.ChangeEventSourceCoordinator.executeChangeEventSources(ChangeEven
              at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoor
              at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
              at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
              at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:11
              at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:6
              at java.base/java.lang.Thread.run(Thread.java:829)
      Caused by: java.lang.NullPointerException
              at io.debezium.connector.oracle.OracleSnapshotChangeEventSource.createSchemaChangeEventsF
              at io.debezium.relational.RelationalSnapshotChangeEventSource.doExecute(RelationalSnapsho
              at io.debezium.pipeline.source.AbstractSnapshotChangeEventSource.execute(AbstractSnapshot
              ... 8 more
      [2022-10-11 12:00:54,471] INFO Connected metrics set to 'false' (io.debezium.pipeline.ChangeEvent
      [2022-10-11 12:00:55,351] INFO WorkerSourceTask{id=star-connector-v7-0} flushing 0 outstanding me
      [2022-10-11 12:00:55,353] ERROR WorkerSourceTask{id=star-connector-v7-0} Task threw an uncaught ag.apache.kafka.connect.runtime.WorkerTask)
      org.apache.kafka.connect.errors.ConnectException: An exception occurred in the change event produ
              at io.debezium.pipeline.ErrorHandler.setProducerThrowable(ErrorHandler.java:50)
              at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoor
              at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
              at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
              at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:11
              at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:6
              at java.base/java.lang.Thread.run(Thread.java:829)
      Caused by: io.debezium.DebeziumException: java.lang.NullPointerException
              at io.debezium.pipeline.source.AbstractSnapshotChangeEventSource.execute(AbstractSnapshot
              at io.debezium.pipeline.ChangeEventSourceCoordinator.doSnapshot(ChangeEventSourceCoordina
              at io.debezium.pipeline.ChangeEventSourceCoordinator.executeChangeEventSources(ChangeEven
              at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoor
              ... 5 more
      Caused by: java.lang.NullPointerException
              at io.debezium.connector.oracle.OracleSnapshotChangeEventSource.createSchemaChangeEventsF
              at io.debezium.relational.RelationalSnapshotChangeEventSource.doExecute(RelationalSnapsho
              at io.debezium.pipeline.source.AbstractSnapshotChangeEventSource.execute(AbstractSnapshot
              ... 8 more 

      I think the issue is MGMT_DELTA_SUMMARY_ERRORS is a nested table in Oracle and the nested table is created by the following sql

       

       

      CREATE TABLE SYSMAN.MGMT_DELTA_COMP_SUMMARIES
      (
        DELTA_COMP_GUID  RAW(16),
        COLLECTION_TYPE  VARCHAR2(64 BYTE),
        DIFFERENT_COUNT  INTEGER,
        LEFT_COUNT       INTEGER,
        RIGHT_COUNT      INTEGER,
        ERRORS           SYSMAN.VARCHAR2_TABLE,
        CONSTRAINT COMP_SUMMARY_PK
        PRIMARY KEY
        (DELTA_COMP_GUID, COLLECTION_TYPE)
        USING INDEX
          TABLESPACE SYSAUX
        ENABLE VALIDATE
      ,  CONSTRAINT COMP_DELTA_SUM_FK 
        FOREIGN KEY (DELTA_COMP_GUID) 
        REFERENCES SYSMAN.MGMT_DELTA_SAVED_COMPARISON (DELTA_COMP_GUID)
        ON DELETE CASCADE
        ENABLE VALIDATE
      )
      NESTED TABLE ERRORS STORE AS MGMT_DELTA_SUMMARY_ERRORS
      TABLESPACE SYSAUX
      LOGGING 
      NOCOMPRESS 
      NOCACHE;
      --  There is no statement for index SYSMAN.SYS_C006479.
      --  The object is created when the parent object is created.
       

       

       

      I tried to add the oracle schema SYSMAN in the exclude.schema.list in the connector configuration but the capturing table step still tries to read the table. I need to rollback to 1.8.1 to workaround the issue.

       

      What Debezium connector do you use and what version?

      1.9.5

      What is the connector configuration?

      Oracle Connector Debezium

      What is the captured database version and mode of depoyment?

      (E.g. on-premises, with a specific cloud provider, etc.)

      Oracle 11.2.0.4 on premise

      What behaviour do you expect?

      <Your answer>

      What behaviour do you see?

      <Your answer>

      Do you see the same behaviour using the latest relesead Debezium version?

      (Ideally, also verify with latest Alpha/Beta/CR version)

      I tried to use 1.9.6 but i got a different error not related to this.

      Do you have the connector logs, ideally from start till finish?

      (You might be asked later to provide DEBUG/TRACE level log)

      <Your answer>

      How to reproduce the issue using our tutorial deployment?

      <Your answer>

      Feature request or enhancement

      For feature requests or enhancements, provide this information, please:

      Which use case/requirement will be addressed by the proposed feature?

      <Your answer>

      Implementation ideas (optional)

      <Your answer>

      Attachments

        Issue Links

          Activity

            People

              ccranfor@redhat.com Chris Cranford
              cheungtsl Thomas Cheung (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: