Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-4654

Oracle DDL parser failure with supplemental log group clause with a custom name

XMLWordPrintable

      Hi, 
      I am trying to add a new table in Debezium oracle cdc connector and as soon as I trigger incremental snapshot, I get following errors in the cluster logs and it fail the connector. 

      [2022-01-31 21:34:02,648] ERROR [ TESTDB-oracdc-test-002-nonprod|task-0] Mining session stopped due to the {} (io.debezium.connector.oracle.logminer.LogMinerHelper:111)
      org.apache.kafka.connect.errors.ConnectException: Error while processing event at offset {commit_scn=6763072949260, transaction_id=null, scn=6763072949249}
          at io.debezium.pipeline.EventDispatcher.dispatchDataChangeEvent(EventDispatcher.java:252)
          at io.debezium.connector.oracle.logminer.processor.AbstractLogMinerEventProcessor$1.accept(AbstractLogMinerEventProcessor.java:397)
          at io.debezium.connector.oracle.logminer.processor.AbstractLogMinerEventProcessor$1.accept(AbstractLogMinerEventProcessor.java:353)
          at io.debezium.connector.oracle.logminer.processor.TransactionCommitConsumer.dispatchChangeEvent(TransactionCommitConsumer.java:144)
          at io.debezium.connector.oracle.logminer.processor.TransactionCommitConsumer.close(TransactionCommitConsumer.java:100)
          at io.debezium.connector.oracle.logminer.processor.AbstractLogMinerEventProcessor.handleCommit(AbstractLogMinerEventProcessor.java:417)
          at io.debezium.connector.oracle.logminer.processor.AbstractLogMinerEventProcessor.processRow(AbstractLogMinerEventProcessor.java:261)
          at io.debezium.connector.oracle.logminer.processor.AbstractLogMinerEventProcessor.processResults(AbstractLogMinerEventProcessor.java:239)
          at io.debezium.connector.oracle.logminer.processor.AbstractLogMinerEventProcessor.process(AbstractLogMinerEventProcessor.java:185)
          at io.debezium.connector.oracle.logminer.LogMinerStreamingChangeEventSource.execute(LogMinerStreamingChangeEventSource.java:168)
          at io.debezium.connector.oracle.logminer.LogMinerStreamingChangeEventSource.execute(LogMinerStreamingChangeEventSource.java:57)
          at io.debezium.pipeline.ChangeEventSourceCoordinator.streamEvents(ChangeEventSourceCoordinator.java:172)
          at io.debezium.pipeline.ChangeEventSourceCoordinator.executeChangeEventSources(ChangeEventSourceCoordinator.java:139)
          at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:108)
          at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
          at java.util.concurrent.FutureTask.run(FutureTask.java:266)
          at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
          at java.lang.Thread.run(Thread.java:748)
      Caused by: java.lang.IllegalArgumentException: No enum constant io.debezium.pipeline.signal.ExecuteSnapshot.SnapshotType.incremental
          at java.lang.Enum.valueOf(Enum.java:238)
          at io.debezium.pipeline.signal.ExecuteSnapshot$SnapshotType.valueOf(ExecuteSnapshot.java:36)
          at io.debezium.pipeline.signal.ExecuteSnapshot.getSnapshotType(ExecuteSnapshot.java:66)
          at io.debezium.pipeline.signal.ExecuteSnapshot.arrived(ExecuteSnapshot.java:52)
          at io.debezium.pipeline.signal.Signal.process(Signal.java:137)
          at io.debezium.pipeline.signal.Signal.process(Signal.java:175)
          at io.debezium.pipeline.EventDispatcher$2.changeRecord(EventDispatcher.java:226)
          at io.debezium.relational.RelationalChangeRecordEmitter.emitCreateRecord(RelationalChangeRecordEmitter.java:78)
          at io.debezium.relational.RelationalChangeRecordEmitter.emitChangeRecords(RelationalChangeRecordEmitter.java:46)
          at io.debezium.pipeline.EventDispatcher.dispatchDataChangeEvent(EventDispatcher.java:215)
          ... 18 more
      [2022-01-31 21:34:02,649] ERROR [ TESTDB-oracdc-test-002-nonprod|task-0] Producer failure (io.debezium.pipeline.ErrorHandler:31)
      org.apache.kafka.connect.errors.ConnectException: Error while processing event at offset {commit_scn=6763072949260, transaction_id=null, scn=6763072949249}
          at io.debezium.pipeline.EventDispatcher.dispatchDataChangeEvent(EventDispatcher.java:252)
          at io.debezium.connector.oracle.logminer.processor.AbstractLogMinerEventProcessor$1.accept(AbstractLogMinerEventProcessor.java:397)
          at io.debezium.connector.oracle.logminer.processor.AbstractLogMinerEventProcessor$1.accept(AbstractLogMinerEventProcessor.java:353)
          at io.debezium.connector.oracle.logminer.processor.TransactionCommitConsumer.dispatchChangeEvent(TransactionCommitConsumer.java:144)
          at io.debezium.connector.oracle.logminer.processor.TransactionCommitConsumer.close(TransactionCommitConsumer.java:100)
          at io.debezium.connector.oracle.logminer.processor.AbstractLogMinerEventProcessor.handleCommit(AbstractLogMinerEventProcessor.java:417)
          at io.debezium.connector.oracle.logminer.processor.AbstractLogMinerEventProcessor.processRow(AbstractLogMinerEventProcessor.java:261)
          at io.debezium.connector.oracle.logminer.processor.AbstractLogMinerEventProcessor.processResults(AbstractLogMinerEventProcessor.java:239)
          at io.debezium.connector.oracle.logminer.processor.AbstractLogMinerEventProcessor.process(AbstractLogMinerEventProcessor.java:185)
          at io.debezium.connector.oracle.logminer.LogMinerStreamingChangeEventSource.execute(LogMinerStreamingChangeEventSource.java:168)
          at io.debezium.connector.oracle.logminer.LogMinerStreamingChangeEventSource.execute(LogMinerStreamingChangeEventSource.java:57)
          at io.debezium.pipeline.ChangeEventSourceCoordinator.streamEvents(ChangeEventSourceCoordinator.java:172)
          at io.debezium.pipeline.ChangeEventSourceCoordinator.executeChangeEventSources(ChangeEventSourceCoordinator.java:139)
          at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:108)
          at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
          at java.util.concurrent.FutureTask.run(FutureTask.java:266)
          at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
          at java.lang.Thread.run(Thread.java:748)
      Caused by: java.lang.IllegalArgumentException: No enum constant io.debezium.pipeline.signal.ExecuteSnapshot.SnapshotType.incremental
          at java.lang.Enum.valueOf(Enum.java:238)
          at io.debezium.pipeline.signal.ExecuteSnapshot$SnapshotType.valueOf(ExecuteSnapshot.java:36)
          at io.debezium.pipeline.signal.ExecuteSnapshot.getSnapshotType(ExecuteSnapshot.java:66)
          at io.debezium.pipeline.signal.ExecuteSnapshot.arrived(ExecuteSnapshot.java:52)
          at io.debezium.pipeline.signal.Signal.process(Signal.java:137)
          at io.debezium.pipeline.signal.Signal.process(Signal.java:175)
          at io.debezium.pipeline.EventDispatcher$2.changeRecord(EventDispatcher.java:226)
          at io.debezium.relational.RelationalChangeRecordEmitter.emitCreateRecord(RelationalChangeRecordEmitter.java:78)
          at io.debezium.relational.RelationalChangeRecordEmitter.emitChangeRecords(RelationalChangeRecordEmitter.java:46)
          at io.debezium.pipeline.EventDispatcher.dispatchDataChangeEvent(EventDispatcher.java:215)
          ... 18 more

        [2022-01-31 21:34:02,888] ERROR [ TESTDB-oracdc-test-002-nonprod|task-0] WorkerSourceTask{id= TESTDB-oracdc-test-002-nonprod-0} Task threw an uncaught and unrecoverable exception (org.apache.kafka.connect.runtime.WorkerTask:187)
      org.apache.kafka.connect.errors.ConnectException: An exception occurred in the change event producer. This connector will be stopped.
          at io.debezium.pipeline.ErrorHandler.setProducerThrowable(ErrorHandler.java:42)
          at io.debezium.connector.oracle.logminer.LogMinerStreamingChangeEventSource.execute(LogMinerStreamingChangeEventSource.java:181)
          at io.debezium.connector.oracle.logminer.LogMinerStreamingChangeEventSource.execute(LogMinerStreamingChangeEventSource.java:57)
          at io.debezium.pipeline.ChangeEventSourceCoordinator.streamEvents(ChangeEventSourceCoordinator.java:172)
          at io.debezium.pipeline.ChangeEventSourceCoordinator.executeChangeEventSources(ChangeEventSourceCoordinator.java:139)
          at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:108)
          at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
          at java.util.concurrent.FutureTask.run(FutureTask.java:266)
          at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
          at java.lang.Thread.run(Thread.java:748)
      Caused by: org.apache.kafka.connect.errors.ConnectException: Error while processing event at offset {commit_scn=6763072949260, transaction_id=null, scn=6763072949249}
          at io.debezium.pipeline.EventDispatcher.dispatchDataChangeEvent(EventDispatcher.java:252)
          at io.debezium.connector.oracle.logminer.processor.AbstractLogMinerEventProcessor$1.accept(AbstractLogMinerEventProcessor.java:397)
          at io.debezium.connector.oracle.logminer.processor.AbstractLogMinerEventProcessor$1.accept(AbstractLogMinerEventProcessor.java:353)
          at io.debezium.connector.oracle.logminer.processor.TransactionCommitConsumer.dispatchChangeEvent(TransactionCommitConsumer.java:144)
          at io.debezium.connector.oracle.logminer.processor.TransactionCommitConsumer.close(TransactionCommitConsumer.java:100)
          at io.debezium.connector.oracle.logminer.processor.AbstractLogMinerEventProcessor.handleCommit(AbstractLogMinerEventProcessor.java:417)
          at io.debezium.connector.oracle.logminer.processor.AbstractLogMinerEventProcessor.processRow(AbstractLogMinerEventProcessor.java:261)
          at io.debezium.connector.oracle.logminer.processor.AbstractLogMinerEventProcessor.processResults(AbstractLogMinerEventProcessor.java:239)
          at io.debezium.connector.oracle.logminer.processor.AbstractLogMinerEventProcessor.process(AbstractLogMinerEventProcessor.java:185)
          at io.debezium.connector.oracle.logminer.LogMinerStreamingChangeEventSource.execute(LogMinerStreamingChangeEventSource.java:168)
          ... 9 more
      Caused by: java.lang.IllegalArgumentException: No enum constant io.debezium.pipeline.signal.ExecuteSnapshot.SnapshotType.incremental
          at java.lang.Enum.valueOf(Enum.java:238)
          at io.debezium.pipeline.signal.ExecuteSnapshot$SnapshotType.valueOf(ExecuteSnapshot.java:36)
          at io.debezium.pipeline.signal.ExecuteSnapshot.getSnapshotType(ExecuteSnapshot.java:66)
          at io.debezium.pipeline.signal.ExecuteSnapshot.arrived(ExecuteSnapshot.java:52)
          at io.debezium.pipeline.signal.Signal.process(Signal.java:137)
          at io.debezium.pipeline.signal.Signal.process(Signal.java:175)
          at io.debezium.pipeline.EventDispatcher$2.changeRecord(EventDispatcher.java:226)
          at io.debezium.relational.RelationalChangeRecordEmitter.emitCreateRecord(RelationalChangeRecordEmitter.java:78)
          at io.debezium.relational.RelationalChangeRecordEmitter.emitChangeRecords(RelationalChangeRecordEmitter.java:46)
          at io.debezium.pipeline.EventDispatcher.dispatchDataChangeEvent(EventDispatcher.java:215)
          ... 18 more
      [2022-01-31 21:34:02,888] ERROR [ TESTDB-oracdc-test-002-nonprod|task-0] WorkerSourceTask{id= TESTDB-oracdc-test-002-nonprod-0} Task is being killed and will not recover until manually restarted (org.apache.kafka.connect.runtime.WorkerTask:188)
      [2022-01-31 21:34:02,889] INFO [ TESTDB-oracdc-test-002-nonprod|task-0] Stopping down connector (io.debezium.connector.common.BaseSourceTask:238)
      [2022-01-31 21:34:02,910] INFO [ TESTDB-oracdc-test-002-nonprod|task-0] Connection gracefully closed (io.debezium.jdbc.JdbcConnection:965)
      [2022-01-31 21:34:02,910] INFO [ TESTDB-oracdc-test-002-nonprod|task-0] [Producer clientId= TESTDB-oracdc-dbhistory] Closing the Kafka producer with timeoutMillis = 30000 ms. (org.apache.kafka.clients.producer.KafkaProducer:1182)
      [2022-01-31 21:34:02,911] INFO [ TESTDB-oracdc-test-002-nonprod|task-0] [Producer clientId=connector-producer- TESTDB-oracdc-test-002-nonprod-0] Closing the Kafka producer with timeoutMillis = 30000 ms. (org.apache.kafka.clients.producer.KafkaProducer:1182)

      Below is the kafka config:

      {{{}}
        "connector.class": "io.debezium.connector.oracle.OracleConnector",
        "snapshot.locking.mode": "none",
        "max.queue.size": "1265",
        "max.batch.size": "640",
        "tasks.max": "1",
        "value.converter.schema.registry.basic.auth.user.info": "******",
        "value.converter.schema.registry.url": "*********",
        "confluent.topic.bootstrap.servers": "*********",
        "key.converter": "org.apache.kafka.connect.storage.StringConverter",
        "value.converter": "io.confluent.connect.avro.AvroConverter",
        "value.converter.basic.auth.credentials.source": "USER_INFO",
        "confluent.topic.sasl.mechanism": "PLAIN",
        "database.history.skip.unparseable.ddl": "true",
        "database.history.kafka.bootstrap.servers": "********",
        "database.history.consumer.security.protocol": "SASL_SSL",
        "database.history.consumer.ssl.endpoint.identification.algorithm": "https",
        "database.history.consumer.sasl.mechanism": "PLAIN",
        "database.history.consumer.sasl.jaas.config": "org.apache.kafka.common.security.plain.PlainLoginModule required username=\"********\" password=\"*****",
        "database.history.producer.security.protocol": "SASL_SSL",
        "database.history.producer.ssl.endpoint.identification.algorithm": "https",
        "database.history.producer.sasl.mechanism": "PLAIN",
        "database.history.producer.sasl.jaas.config": "org.apache.kafka.common.security.plain.PlainLoginModule required username=\"*****\" password=\"*****",
        "confluent.topic.security.protocol": "SASL_SSL",
        "confluent.topic.sasl.jaas.config": "org.apache.kafka.common.security.plain.PlainLoginModule required username=\"*****\" password=\"*****",
        "schema.include.list": "ORACDC",
        "lob.enabled": "true",
        "errors.log.enable": "true",
        "database.user": "ORACDC",
        "database.dbname": "TESTDB",
        "database.url": "jdbc:oracle:thin:@(description=(address=(protocol=tcp)(host=******)(port=1521))(connect_data=(server=dedicated)(service_name=TESTDB)))",
        "database.server.name": "TESTDB-oracdc",
        "database.history.kafka.topic": "TESTDB-oracdc-history",
        "database.password": "********",
        "table.include.list": "ORACDC.EMPLOYEE,ORACDC.DEPARTMENT,ORACDC.DEBEZIUM_SIGNAL",",
        "signal.data.collection": "TESTDB.ORACDC.DEBEZIUM_SIGNAL",
        "snapshot.mode": "initial"
      }

      As soon as I trigger incremental snapshot with following command, it throws above errors:

      INSERT INTO ORACDC.debezium_signal (id, type, data) VALUES('ad-hoc-1', 'execute-snapshot', '{"data-collections": ["ORACDC.DEPARTMENT", "ORACDC.LOCATION"],"type":"incremental"}');
      commit;

              ccranfor@redhat.com Chris Cranford
              fk36@hotmail.com Farooq Khalid (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

                Created:
                Updated:
                Resolved: