-
Bug
-
Resolution: Obsolete
-
Major
-
None
-
1.9.3.Final
-
None
-
False
-
None
-
False
When I try to snapshot a table using Debezium Oracle connector getting Null Pointer exception
Bug report
Getting Null pointer exception while trying to snapshot a oracle table. Connector can't start and remains in failed status
What Debezium connector do you use and what version?
1.9.3
What is the connector configuration?
// code placeholder { "name": "cdh2snapshot_ppvlcraw_prod_g5", "config": { "database.server.name": "cdh2snapshot_ppvlcraw_prod_g5", "heartbeat.topics.prefix": "cdh2snapshot_ppvlcraw_prod_g5.heartbeat", "database.history.kafka.topic": "cdh2snapshot_ppvlcraw_prod_g5.history-topic", "transaction.topic": "cdh2snapshot_ppvlcraw_prod_g5.transaction", "transforms.datastreamtopic.regex" : "cdh2snapshot_ppvlcraw_prod_g5.VLC_RAW.(?!DBZ_HEARTBEAT$)(.+)$", "transforms.datastreamtopic.replacement" : "cdh_datasnapshot_.$1", "transforms": "datastreamtopic", "transforms.datastreamtopic.type" : "org.apache.kafka.connect.transforms.RegexRouter", "log.mining.session.max.ms" : "14400000", "connector.class": "io.debezium.connector.oracle.OracleConnector", "database.dbname": "xxxDB", "database.user": "c##dbzxxx", "database.password": "${secrets:kafka-connect-connectors-passwords:cdhextract-password}", "database.connection.adapter": "logminer", "database.url": "jdbc:oracle:thin:@(DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = xxxx)(PORT = xxxx)) (CONNECT_DATA = (SERVER = DEDICATED) (SERVICE_NAME = xxxx)))", "database.pdb.name": "xxxxx", "database.tablename.case.insensitive": "true", "snapshot.mode": "initial_only", "snapshot.locking.mode": "none", "snapshot.fetch.size": "20000", "snapshot.delay.ms": "2000", "topic.creation.default.replication.factor": "3", "topic.creation.default.partitions": "1", "topic.creation.enable": "true", "event.processing.failure.handling.mode": "warn", "tasks.max": "1", "sanitize.field.names": "true", "heartbeat.interval.ms": "10000", "include.schema.changes": "true", "max.batch.size": "20480", "max.queue.size": "81290", "decimal.handling.mode": "double", "time.precision.mode": "connect", "binary.handling.mode": "base64", "log.mining.strategy": "online_catalog", "log.mining.scn.gap.detection.gap.size.min": "100000", "log.mining.scn.gap.detection.time.interval.max.ms": "3000", "log.mining.view.fetch.size": "10000", "log.mining.batch.size.default": "20000", "log.mining.batch.size.max": "25000", "log.mining.batch.size.min": "1000", "database.history.store.only.captured.tables.ddl": "true", "database.history.kafka.recovery.poll.interval.ms": "100000", "database.history.kafka.query.timeout.ms": "30000", "database.history.kafka.bootstrap.servers": "${env:KAFKA_CONNECT_BOOTSTRAP_SERVERS}", "database.history.consumer.security.protocol": "${env:APMT_KAFKA_CONNECT_SECURITY_PROTOCOL}", "database.history.producer.security.protocol": "${env:APMT_KAFKA_CONNECT_SECURITY_PROTOCOL}", "database.history.consumer.sasl.mechanism": "${env:APMT_KAFKA_SASL_MECHANISM}", "database.history.producer.sasl.mechanism": "${env:APMT_KAFKA_SASL_MECHANISM}", "database.history.consumer.sasl.jaas.config": "${env:APMT_KAFKA_JAAS_CONFIG}", "database.history.producer.sasl.jaas.config": "${env:APMT_KAFKA_JAAS_CONFIG}", "database.history.consumer.ssl.endpoint.identification.algorithm": "https", "database.history.producer.ssl.endpoint.identification.algorithm": "https", "table.include.list": "VLC_RAW.ref_unloc_code,VLC_RAW.ref_bizunit_scoped" } }
What is the captured database version and mode of depoyment?
Oracle 19c
What behaviour do you expect?
No error and snapshot should begin
What behaviour do you see?
Null Pointer exception. Connector can't start.
Do you see the same behaviour using the latest relesead Debezium version?
(Ideally, also verify with latest Alpha/Beta/CR version)
<Your answer>
Do you have the connector logs, ideally from start till finish?
(You might be asked later to provide DEBUG/TRACE level log)
Heading 1 |
---|
2022-09-06 04:55:55,436 INFO Stopping down connector (io.debezium.connector.common.BaseSourceTask) [task-thread-apmt_cdh_aks1_valencia_cdh2snapshot_ppvlcraw_prod_g5-0]
Caused by: java.lang.NullPointerException
... 5 more
at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:109)
at io.debezium.pipeline.ChangeEventSourceCoordinator.executeChangeEventSources(ChangeEventSourceCoordinator.java:137)
at io.debezium.pipeline.ChangeEventSourceCoordinator.doSnapshot(ChangeEventSourceCoordinator.java:155)
at io.debezium.pipeline.source.AbstractSnapshotChangeEventSource.execute(AbstractSnapshotChangeEventSource.java:85)
Caused by: io.debezium.DebeziumException: java.lang.NullPointerException
at java.base/java.lang.Thread.run(Thread.java:829)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:116)
at io.debezium.pipeline.ErrorHandler.setProducerThrowable(ErrorHandler.java:50)
org.apache.kafka.connect.errors.ConnectException: An exception occurred in the change event producer. This connector will be stopped.
2022-09-06 04:55:55,435 ERROR WorkerSourceTask{id=apmt_cdh_aks1_valencia_cdh2snapshot_ppvlcraw_prod_g5-0} Task threw an uncaught and unrecoverable exception. Task is being killed and will not recover until manually restarted (org.apache.kafka.connect.runtime.WorkerTask) [task-thread-apmt_cdh_aks1_valencia_cdh2snapshot_ppvlcraw_prod_g5-0]
2022-09-06 04:55:55,103 INFO Connected metrics set to 'false' (io.debezium.pipeline.ChangeEventSourceCoordinator) [debezium-oracleconnector-apmt_cdh_aks1_valencia_cdh2snapshot_ppvlcraw_prod_g5-change-event-source-coordinator]
Caused by: java.lang.NullPointerException
at java.base/java.lang.Thread.run(Thread.java:829)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:109)
at io.debezium.pipeline.ChangeEventSourceCoordinator.executeChangeEventSources(ChangeEventSourceCoordinator.java:137)
at io.debezium.pipeline.ChangeEventSourceCoordinator.doSnapshot(ChangeEventSourceCoordinator.java:155)
at io.debezium.pipeline.source.AbstractSnapshotChangeEventSource.execute(AbstractSnapshotChangeEventSource.java:85)
io.debezium.DebeziumException: java.lang.NullPointerException
2022-09-06 04:55:55,103 ERROR Producer failure (io.debezium.pipeline.ErrorHandler) [debezium-oracleconnector-apmt_cdh_aks1_valencia_cdh2snapshot_ppvlcraw_prod_g5-change-event-source-coordinator]
2022-09-06 04:55:55,089 INFO Snapshot - Final stage (io.debezium.pipeline.source.AbstractSnapshotChangeEventSource) [debezium-oracleconnector-apmt_cdh_aks1_valencia_cdh2snapshot_ppvlcraw_prod_g5-change-event-source-coordinator]
2022-09-06 04:55:55,084 INFO Capturing structure of table PPVLCRAW.VLC_TST.inv_unit.temp (io.debezium.connector.oracle.OracleSnapshotChangeEventSource) [debezium-oracleconnector-apmt_cdh_aks1_valencia_cdh2snapshot_ppvlcraw_prod_g5-change-event-source-coordinator]
2022-09-06 04:55:55,084 INFO Capturing structure of table PPVLCRAW.VLC_RAW.yard_berth_markers (io.debezium.connector.oracle.OracleSnapshotChangeEventSource) [debezium-oracleconnector-apmt_cdh_aks1_valencia_cdh2snapshot_ppvlcraw_prod_g5-change-event-source-coordinator]
2022-09-06 04:55:55,084 INFO Capturing structure of table PPVLCRAW.VLC_RAW.xps_workassignment (io.debezium.connector.oracle.OracleSnapshotChangeEventSource) [debezium-oracleconnector-apmt_cdh_aks1_valencia_cdh2snapshot_ppvlcraw_prod_g5-change-event-source-coordinator]
2022-09-06 04:55:55,084 INFO Capturing structure of table PPVLCRAW.VLC_RAW.xps_stackstatus (io.debezium.connector.oracle.OracleSnapshotChangeEventSource) [debezium-oracleconnector-apmt_cdh_aks1_valencia_cdh2snapshot_ppvlcraw_prod_g5-change-event-source-coordinator]
2022-09-06 04:55:55,084 INFO Capturing structure of table PPVLCRAW.VLC_RAW.xps_stacking_fct_fltr_items (io.debezium.connector.oracle.OracleSnapshotChangeEventSource) [debezium-oracleconnector-apmt_cdh_aks1_valencia_cdh2snapshot_ppvlcraw_prod_g5-change-event-source-coordinator]
2022-09-06 04:55:53,588 INFO Capturing structure of table PPVLCRAW.VLC_RAW.argo_ar_carrier_visit (io.debezium.connector.oracle.OracleSnapshotChangeEventSource) [debezium-oracleconnector-apmt_cdh_aks1_valencia_cdh2snapshot_ppvlcraw_prod_g5-change-event-source-coordinator]
2022-09-06 04:55:53,588 INFO Snapshot step 6 - Persisting schema history (io.debezium.relational.RelationalSnapshotChangeEventSource) [debezium-oracleconnector-apmt_cdh_aks1_valencia_cdh2snapshot_ppvlcraw_prod_g5-change-event-source-coordinator]
2022-09-06 04:55:52,314 INFO Only captured tables schema should be captured, capturing: [PPVLCRAW.VLC_RAW.ref_unloc_code, PPVLCRAW.VLC_RAW.ref_bizunit_scoped] (io.debezium.connector.oracle.OracleSnapshotChangeEventSource) [debezium-oracleconnector-apmt_cdh_aks1_valencia_cdh2snapshot_ppvlcraw_prod_g5-change-event-source-coordinator]
2022-09-06 04:55:52,314 INFO Snapshot step 5 - Reading structure of captured tables (io.debezium.relational.RelationalSnapshotChangeEventSource) [debezium-oracleconnector-apmt_cdh_aks1_valencia_cdh2snapshot_ppvlcraw_prod_g5-change-event-source-coordinator]
2022-09-06 04:55:52,314 INFO Snapshot step 4 - Determining snapshot offset (io.debezium.relational.RelationalSnapshotChangeEventSource) [debezium-oracleconnector-apmt_cdh_aks1_valencia_cdh2snapshot_ppvlcraw_prod_g5-change-event-source-coordinator]
2022-09-06 04:55:52,314 INFO Schema locking was disabled in connector configuration (io.debezium.connector.oracle.OracleSnapshotChangeEventSource) [debezium-oracleconnector-apmt_cdh_aks1_valencia_cdh2snapshot_ppvlcraw_prod_g5-change-event-source-coordinator]
2022-09-06 04:55:52,314 INFO Snapshot step 3 - Locking captured tables [PPVLCRAW.VLC_RAW.ref_unloc_code, PPVLCRAW.VLC_RAW.ref_bizunit_scoped] (io.debezium.relational.RelationalSnapshotChangeEventSource) [debezium-oracleconnector-apmt_cdh_aks1_valencia_cdh2snapshot_ppvlcraw_prod_g5-change-event-source-coordinator]
2022-09-06 04:55:52,314 INFO Adding table PPVLCRAW.VLC_RAW.ref_unloc_code to the list of capture schema tables (io.debezium.relational.RelationalSnapshotChangeEventSource) [debezium-oracleconnector-apmt_cdh_aks1_valencia_cdh2snapshot_ppvlcraw_prod_g5-change-event-source-coordinator]
2022-09-06 04:55:52,314 INFO Adding table PPVLCRAW.VLC_RAW.rail_railroads to the list of capture schema tables (io.debezium.relational.RelationalSnapshotChangeEventSource) [debezium-oracleconnector-apmt_cdh_aks1_valencia_cdh2snapshot_ppvlcraw_prod_g5-change-event-source-coordinator]
2022-09-06 04:55:52,314 INFO Adding table PPVLCRAW.VLC_RAW.inv_ar_unit to the list of capture schema tables (io.debezium.relational.RelationalSnapshotChangeEventSource) [debezium-oracleconnector-apmt_cdh_aks1_valencia_cdh2snapshot_ppvlcraw_prod_g5-change-event-source-coordinator]
2022-09-06 04:55:52,314 INFO Adding table PPVLCRAW.VLC_RAW.vsl_crane_statistics_delays to the list of capture schema tables (io.debezium.relational.RelationalSnapshotChangeEventSource) [debezium-oracleconnector-apmt_cdh_aks1_valencia_cdh2snapshot_ppvlcraw_prod_g5-change-event-source-coordinator]
2022-09-06 04:55:52,314 INFO Adding table PPVLCRAW.VLC_RAW.argo_lanes to the list of capture schema tables (io.debezium.relational.RelationalSnapshotChangeEventSource) [debezium-oracleconnector-apmt_cdh_aks1_valencia_cdh2snapshot_ppvlcraw_prod_g5-change-event-source-coordinator]
2022-09-06 04:55:52,312 INFO Adding table PPVLCRAW.VLC_RAW.ref_mock_biz_unit to the list of capture schema tables (io.debezium.relational.RelationalSnapshotChangeEventSource) [debezium-oracleconnector-apmt_cdh_aks1_valencia_cdh2snapshot_ppvlcraw_prod_g5-change-event-source-coordinator]
2022-09-06 04:55:51,929 INFO Snapshot step 2 - Determining captured tables (io.debezium.relational.RelationalSnapshotChangeEventSource) [debezium-oracleconnector-apmt_cdh_aks1_valencia_cdh2snapshot_ppvlcraw_prod_g5-change-event-source-coordinator]
2022-09-06 04:55:51,929 INFO Previous snapshot was cancelled before completion; a new snapshot will be taken. (io.debezium.relational.RelationalSnapshotChangeEventSource) [debezium-oracleconnector-apmt_cdh_aks1_valencia_cdh2snapshot_ppvlcraw_prod_g5-change-event-source-coordinator]
2022-09-06 04:55:51,929 INFO Snapshot step 1 - Preparing (io.debezium.relational.RelationalSnapshotChangeEventSource) [debezium-oracleconnector-apmt_cdh_aks1_valencia_cdh2snapshot_ppvlcraw_prod_g5-change-event-source-coordinator]|
How to reproduce the issue using our tutorial deployment?
Tried with other tables also.. but same issue
Feature request or enhancement
For feature requests or enhancements, provide this information, please:
Which use case/requirement will be addressed by the proposed feature?
<Your answer>
Implementation ideas (optional)
<Your answer>
- duplicates
-
DBZ-5270 database.history.store.only.captured.tables.ddl not suppressing logs
- Closed