Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-5584

NullPointerException thrown during snapshot of tables

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Obsolete
    • Icon: Major Major
    • None
    • 1.9.3.Final
    • oracle-connector
    • None
    • False
    • None
    • False

      When I try to snapshot a table using Debezium Oracle connector getting Null Pointer exception

      Bug report

      Getting Null pointer exception while trying to snapshot a oracle table. Connector can't start and remains in failed status

      What Debezium connector do you use and what version?

      1.9.3

      What is the connector configuration?

      // code placeholder
      
      {          "name":                                                "cdh2snapshot_ppvlcraw_prod_g5",          "config": {            "database.server.name":                              "cdh2snapshot_ppvlcraw_prod_g5",            "heartbeat.topics.prefix":                           "cdh2snapshot_ppvlcraw_prod_g5.heartbeat",            "database.history.kafka.topic":                      "cdh2snapshot_ppvlcraw_prod_g5.history-topic",            "transaction.topic":                                 "cdh2snapshot_ppvlcraw_prod_g5.transaction",            "transforms.datastreamtopic.regex" :                 "cdh2snapshot_ppvlcraw_prod_g5.VLC_RAW.(?!DBZ_HEARTBEAT$)(.+)$",            "transforms.datastreamtopic.replacement" :           "cdh_datasnapshot_.$1",            "transforms":                                        "datastreamtopic",            "transforms.datastreamtopic.type" :                  "org.apache.kafka.connect.transforms.RegexRouter",            "log.mining.session.max.ms" :                        "14400000",            "connector.class":                                   "io.debezium.connector.oracle.OracleConnector",            "database.dbname":                                   "xxxDB",            "database.user":                                     "c##dbzxxx",            "database.password":                                 "${secrets:kafka-connect-connectors-passwords:cdhextract-password}",            "database.connection.adapter":                       "logminer",            "database.url":                                      "jdbc:oracle:thin:@(DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = xxxx)(PORT = xxxx)) (CONNECT_DATA = (SERVER = DEDICATED) (SERVICE_NAME = xxxx)))",            "database.pdb.name":                                 "xxxxx",            "database.tablename.case.insensitive":               "true",            "snapshot.mode":                                     "initial_only",            "snapshot.locking.mode":                             "none",            "snapshot.fetch.size":                               "20000",            "snapshot.delay.ms":                                 "2000",            "topic.creation.default.replication.factor":         "3",            "topic.creation.default.partitions":                 "1",            "topic.creation.enable":                             "true",            "event.processing.failure.handling.mode":            "warn",            "tasks.max":                                         "1",                    "sanitize.field.names":                              "true",            "heartbeat.interval.ms":                             "10000",            "include.schema.changes":                            "true",            "max.batch.size":                                    "20480",            "max.queue.size":                                    "81290",            "decimal.handling.mode":                             "double",            "time.precision.mode":                               "connect",            "binary.handling.mode":                              "base64",            "log.mining.strategy":                               "online_catalog",            "log.mining.scn.gap.detection.gap.size.min":         "100000",            "log.mining.scn.gap.detection.time.interval.max.ms": "3000",            "log.mining.view.fetch.size":                        "10000",            "log.mining.batch.size.default":                     "20000",            "log.mining.batch.size.max":                         "25000",            "log.mining.batch.size.min":                         "1000",            "database.history.store.only.captured.tables.ddl":   "true",            "database.history.kafka.recovery.poll.interval.ms":  "100000",            "database.history.kafka.query.timeout.ms":           "30000",            "database.history.kafka.bootstrap.servers":          "${env:KAFKA_CONNECT_BOOTSTRAP_SERVERS}",            "database.history.consumer.security.protocol":       "${env:APMT_KAFKA_CONNECT_SECURITY_PROTOCOL}",            "database.history.producer.security.protocol":       "${env:APMT_KAFKA_CONNECT_SECURITY_PROTOCOL}",            "database.history.consumer.sasl.mechanism":          "${env:APMT_KAFKA_SASL_MECHANISM}",            "database.history.producer.sasl.mechanism":          "${env:APMT_KAFKA_SASL_MECHANISM}",            "database.history.consumer.sasl.jaas.config":        "${env:APMT_KAFKA_JAAS_CONFIG}",            "database.history.producer.sasl.jaas.config":        "${env:APMT_KAFKA_JAAS_CONFIG}",            "database.history.consumer.ssl.endpoint.identification.algorithm": "https",            "database.history.producer.ssl.endpoint.identification.algorithm": "https",            "table.include.list":                        "VLC_RAW.ref_unloc_code,VLC_RAW.ref_bizunit_scoped"          }        }   

      What is the captured database version and mode of depoyment?

      Oracle 19c 

      What behaviour do you expect?

      No error and snapshot should begin

      What behaviour do you see?

      Null Pointer exception. Connector can't start.

      Do you see the same behaviour using the latest relesead Debezium version?

      (Ideally, also verify with latest Alpha/Beta/CR version)

      <Your answer>

      Do you have the connector logs, ideally from start till finish?

      (You might be asked later to provide DEBUG/TRACE level log)

      Heading 1
       

      2022-09-06 04:55:55,436 INFO Stopping down connector (io.debezium.connector.common.BaseSourceTask) [task-thread-apmt_cdh_aks1_valencia_cdh2snapshot_ppvlcraw_prod_g5-0]
      Caused by: java.lang.NullPointerException
          ... 5 more
          at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:109)
          at io.debezium.pipeline.ChangeEventSourceCoordinator.executeChangeEventSources(ChangeEventSourceCoordinator.java:137)
          at io.debezium.pipeline.ChangeEventSourceCoordinator.doSnapshot(ChangeEventSourceCoordinator.java:155)
          at io.debezium.pipeline.source.AbstractSnapshotChangeEventSource.execute(AbstractSnapshotChangeEventSource.java:85)
      Caused by: io.debezium.DebeziumException: java.lang.NullPointerException
          at java.base/java.lang.Thread.run(Thread.java:829)
          at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
          at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
          at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
          at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
          at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:116)
          at io.debezium.pipeline.ErrorHandler.setProducerThrowable(ErrorHandler.java:50)
      org.apache.kafka.connect.errors.ConnectException: An exception occurred in the change event producer. This connector will be stopped.
      2022-09-06 04:55:55,435 ERROR WorkerSourceTask{id=apmt_cdh_aks1_valencia_cdh2snapshot_ppvlcraw_prod_g5-0} Task threw an uncaught and unrecoverable exception. Task is being killed and will not recover until manually restarted (org.apache.kafka.connect.runtime.WorkerTask) [task-thread-apmt_cdh_aks1_valencia_cdh2snapshot_ppvlcraw_prod_g5-0]
      2022-09-06 04:55:55,103 INFO Connected metrics set to 'false' (io.debezium.pipeline.ChangeEventSourceCoordinator) [debezium-oracleconnector-apmt_cdh_aks1_valencia_cdh2snapshot_ppvlcraw_prod_g5-change-event-source-coordinator]
      Caused by: java.lang.NullPointerException
          at java.base/java.lang.Thread.run(Thread.java:829)
          at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
          at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
          at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
          at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
          at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:109)
          at io.debezium.pipeline.ChangeEventSourceCoordinator.executeChangeEventSources(ChangeEventSourceCoordinator.java:137)
          at io.debezium.pipeline.ChangeEventSourceCoordinator.doSnapshot(ChangeEventSourceCoordinator.java:155)
          at io.debezium.pipeline.source.AbstractSnapshotChangeEventSource.execute(AbstractSnapshotChangeEventSource.java:85)
      io.debezium.DebeziumException: java.lang.NullPointerException
      2022-09-06 04:55:55,103 ERROR Producer failure (io.debezium.pipeline.ErrorHandler) [debezium-oracleconnector-apmt_cdh_aks1_valencia_cdh2snapshot_ppvlcraw_prod_g5-change-event-source-coordinator]
      2022-09-06 04:55:55,089 INFO Snapshot - Final stage (io.debezium.pipeline.source.AbstractSnapshotChangeEventSource) [debezium-oracleconnector-apmt_cdh_aks1_valencia_cdh2snapshot_ppvlcraw_prod_g5-change-event-source-coordinator]
      2022-09-06 04:55:55,084 INFO Capturing structure of table PPVLCRAW.VLC_TST.inv_unit.temp (io.debezium.connector.oracle.OracleSnapshotChangeEventSource) [debezium-oracleconnector-apmt_cdh_aks1_valencia_cdh2snapshot_ppvlcraw_prod_g5-change-event-source-coordinator]
      2022-09-06 04:55:55,084 INFO Capturing structure of table PPVLCRAW.VLC_RAW.yard_berth_markers (io.debezium.connector.oracle.OracleSnapshotChangeEventSource) [debezium-oracleconnector-apmt_cdh_aks1_valencia_cdh2snapshot_ppvlcraw_prod_g5-change-event-source-coordinator]
      2022-09-06 04:55:55,084 INFO Capturing structure of table PPVLCRAW.VLC_RAW.xps_workassignment (io.debezium.connector.oracle.OracleSnapshotChangeEventSource) [debezium-oracleconnector-apmt_cdh_aks1_valencia_cdh2snapshot_ppvlcraw_prod_g5-change-event-source-coordinator]
      2022-09-06 04:55:55,084 INFO Capturing structure of table PPVLCRAW.VLC_RAW.xps_stackstatus (io.debezium.connector.oracle.OracleSnapshotChangeEventSource) [debezium-oracleconnector-apmt_cdh_aks1_valencia_cdh2snapshot_ppvlcraw_prod_g5-change-event-source-coordinator]
      2022-09-06 04:55:55,084 INFO Capturing structure of table PPVLCRAW.VLC_RAW.xps_stacking_fct_fltr_items (io.debezium.connector.oracle.OracleSnapshotChangeEventSource) [debezium-oracleconnector-apmt_cdh_aks1_valencia_cdh2snapshot_ppvlcraw_prod_g5-change-event-source-coordinator]
      2022-09-06 04:55:53,588 INFO Capturing structure of table PPVLCRAW.VLC_RAW.argo_ar_carrier_visit (io.debezium.connector.oracle.OracleSnapshotChangeEventSource) [debezium-oracleconnector-apmt_cdh_aks1_valencia_cdh2snapshot_ppvlcraw_prod_g5-change-event-source-coordinator]
      2022-09-06 04:55:53,588 INFO Snapshot step 6 - Persisting schema history (io.debezium.relational.RelationalSnapshotChangeEventSource) [debezium-oracleconnector-apmt_cdh_aks1_valencia_cdh2snapshot_ppvlcraw_prod_g5-change-event-source-coordinator]
      2022-09-06 04:55:52,314 INFO Only captured tables schema should be captured, capturing: [PPVLCRAW.VLC_RAW.ref_unloc_code, PPVLCRAW.VLC_RAW.ref_bizunit_scoped] (io.debezium.connector.oracle.OracleSnapshotChangeEventSource) [debezium-oracleconnector-apmt_cdh_aks1_valencia_cdh2snapshot_ppvlcraw_prod_g5-change-event-source-coordinator]
      2022-09-06 04:55:52,314 INFO Snapshot step 5 - Reading structure of captured tables (io.debezium.relational.RelationalSnapshotChangeEventSource) [debezium-oracleconnector-apmt_cdh_aks1_valencia_cdh2snapshot_ppvlcraw_prod_g5-change-event-source-coordinator]
      2022-09-06 04:55:52,314 INFO Snapshot step 4 - Determining snapshot offset (io.debezium.relational.RelationalSnapshotChangeEventSource) [debezium-oracleconnector-apmt_cdh_aks1_valencia_cdh2snapshot_ppvlcraw_prod_g5-change-event-source-coordinator]
      2022-09-06 04:55:52,314 INFO Schema locking was disabled in connector configuration (io.debezium.connector.oracle.OracleSnapshotChangeEventSource) [debezium-oracleconnector-apmt_cdh_aks1_valencia_cdh2snapshot_ppvlcraw_prod_g5-change-event-source-coordinator]
      2022-09-06 04:55:52,314 INFO Snapshot step 3 - Locking captured tables [PPVLCRAW.VLC_RAW.ref_unloc_code, PPVLCRAW.VLC_RAW.ref_bizunit_scoped] (io.debezium.relational.RelationalSnapshotChangeEventSource) [debezium-oracleconnector-apmt_cdh_aks1_valencia_cdh2snapshot_ppvlcraw_prod_g5-change-event-source-coordinator]
      2022-09-06 04:55:52,314 INFO Adding table PPVLCRAW.VLC_RAW.ref_unloc_code to the list of capture schema tables (io.debezium.relational.RelationalSnapshotChangeEventSource) [debezium-oracleconnector-apmt_cdh_aks1_valencia_cdh2snapshot_ppvlcraw_prod_g5-change-event-source-coordinator]
      2022-09-06 04:55:52,314 INFO Adding table PPVLCRAW.VLC_RAW.rail_railroads to the list of capture schema tables (io.debezium.relational.RelationalSnapshotChangeEventSource) [debezium-oracleconnector-apmt_cdh_aks1_valencia_cdh2snapshot_ppvlcraw_prod_g5-change-event-source-coordinator]
      2022-09-06 04:55:52,314 INFO Adding table PPVLCRAW.VLC_RAW.inv_ar_unit to the list of capture schema tables (io.debezium.relational.RelationalSnapshotChangeEventSource) [debezium-oracleconnector-apmt_cdh_aks1_valencia_cdh2snapshot_ppvlcraw_prod_g5-change-event-source-coordinator]
      2022-09-06 04:55:52,314 INFO Adding table PPVLCRAW.VLC_RAW.vsl_crane_statistics_delays to the list of capture schema tables (io.debezium.relational.RelationalSnapshotChangeEventSource) [debezium-oracleconnector-apmt_cdh_aks1_valencia_cdh2snapshot_ppvlcraw_prod_g5-change-event-source-coordinator]
      2022-09-06 04:55:52,314 INFO Adding table PPVLCRAW.VLC_RAW.argo_lanes to the list of capture schema tables (io.debezium.relational.RelationalSnapshotChangeEventSource) [debezium-oracleconnector-apmt_cdh_aks1_valencia_cdh2snapshot_ppvlcraw_prod_g5-change-event-source-coordinator]
      2022-09-06 04:55:52,312 INFO Adding table PPVLCRAW.VLC_RAW.ref_mock_biz_unit to the list of capture schema tables (io.debezium.relational.RelationalSnapshotChangeEventSource) [debezium-oracleconnector-apmt_cdh_aks1_valencia_cdh2snapshot_ppvlcraw_prod_g5-change-event-source-coordinator]
      2022-09-06 04:55:51,929 INFO Snapshot step 2 - Determining captured tables (io.debezium.relational.RelationalSnapshotChangeEventSource) [debezium-oracleconnector-apmt_cdh_aks1_valencia_cdh2snapshot_ppvlcraw_prod_g5-change-event-source-coordinator]
      2022-09-06 04:55:51,929 INFO Previous snapshot was cancelled before completion; a new snapshot will be taken. (io.debezium.relational.RelationalSnapshotChangeEventSource) [debezium-oracleconnector-apmt_cdh_aks1_valencia_cdh2snapshot_ppvlcraw_prod_g5-change-event-source-coordinator]
      2022-09-06 04:55:51,929 INFO Snapshot step 1 - Preparing (io.debezium.relational.RelationalSnapshotChangeEventSource) [debezium-oracleconnector-apmt_cdh_aks1_valencia_cdh2snapshot_ppvlcraw_prod_g5-change-event-source-coordinator]|

      How to reproduce the issue using our tutorial deployment?

      Tried with other tables also.. but same issue

      Feature request or enhancement

      For feature requests or enhancements, provide this information, please:

      Which use case/requirement will be addressed by the proposed feature?

      <Your answer>

      Implementation ideas (optional)

      <Your answer>

              ccranfor@redhat.com Chris Cranford
              surmukho Surajit Mukhopadhyay (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Created:
                Updated:
                Resolved: