Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-6650

ConcurrentModificationException thrown in Debezium 2.3

XMLWordPrintable

    • Critical

      In order to make your issue reports as actionable as possible, please provide the following information, depending on the issue type.

      Bug report

      For bug reports, provide this information, please:

      Log output:

       

      [2023-07-11 14:55:23,118] INFO Starting streaming (io.debezium.pipeline.ChangeEventSourceCoordinator:204) 
      [2023-07-11 14:55:23,235] INFO Database Version: Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production (io.debezium.connector.oracle.OracleConnection:82) 
      [2023-07-11 14:55:23,670] INFO Getting database metadata for table 'TFTONG.TFT_TSM.TEMP_LCQ_09' (io.debezium.connector.oracle.xstream.LcrEventHandler:305) 
      [2023-07-11 14:55:24,176] INFO Connection gracefully closed (io.debezium.jdbc.JdbcConnection:947) 
      [2023-07-11 14:55:24,177] INFO Table TFTONG.TFT_TSM.TEMP_LCQ_09 will be captured. (io.debezium.connector.oracle.xstream.LcrEventHandler:182) 
      [2023-07-11 14:55:24,190] INFO Already applied 586 database changes (io.debezium.relational.history.SchemaHistoryMetrics:140) 
      [2023-07-11 14:55:24,191] ERROR Producer failure (io.debezium.pipeline.ErrorHandler:57) 
      java.util.ConcurrentModificationException 
              at java.base/java.util.ArrayList.forEach(ArrayList.java:1543) 
              at io.debezium.connector.oracle.OracleSchemaChangeEventEmitter.lambda$emitSchemaChangeEvent$1(OracleSchemaChangeEventEmitter.java:100) 
              at io.debezium.relational.ddl.DdlChanges.getEventsByDatabase(DdlChanges.java:57) 
              at io.debezium.connector.oracle.OracleSchemaChangeEventEmitter.emitSchemaChangeEvent(OracleSchemaChangeEventEmitter.java:99) 
              at io.debezium.pipeline.EventDispatcher.dispatchSchemaChangeEvent(EventDispatcher.java:392) 
              at io.debezium.connector.oracle.xstream.LcrEventHandler.dispatchSchemaChangeEvent(LcrEventHandler.java:257) 
              at io.debezium.connector.oracle.xstream.LcrEventHandler.processLCR(LcrEventHandler.java:120) 
              at oracle.streams.XStreamOut.XStreamOutReceiveLCRCallbackNative(Native Method) 
              at oracle.streams.XStreamOut.receiveLCRCallback(Unknown Source) 
              at io.debezium.connector.oracle.xstream.XstreamStreamingChangeEventSource.execute(XstreamStreamingChangeEventSource.java:125) 
              at io.debezium.connector.oracle.xstream.XstreamStreamingChangeEventSource.execute(XstreamStreamingChangeEventSource.java:45) 
              at io.debezium.pipeline.ChangeEventSourceCoordinator.streamEvents(ChangeEventSourceCoordinator.java:205) 
              at io.debezium.pipeline.ChangeEventSourceCoordinator.executeChangeEventSources(ChangeEventSourceCoordinator.java:172) 
              at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:118) 
              at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) 
              at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) 
              at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) 
              at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) 
              at java.base/java.lang.Thread.run(Thread.java:834) 
      [2023-07-11 14:55:24,612] INFO WorkerSourceTask{id=oracle_tsm-0} Committing offsets (org.apache.kafka.connect.runtime.WorkerSourceTask:478) 
      [2023-07-11 14:55:24,612] INFO WorkerSourceTask{id=oracle_tsm-0} flushing 415 outstanding messages for offset commit (org.apache.kafka.connect.runtime.WorkerSourceTask:495) 
      [2023-07-11 14:55:24,656] INFO WorkerSourceTask{id=oracle_tsm-0} Finished commitOffsets successfully in 44 ms (org.apache.kafka.connect.runtime.WorkerSourceTask:574) 
      [2023-07-11 14:55:24,656] ERROR WorkerSourceTask{id=oracle_tsm-0} Task threw an uncaught and unrecoverable exception. Task is being killed and will not recover until manually restarted (org.apache.kafka.connect.runtime.WorkerTask:187) 
      org.apache.kafka.connect.errors.ConnectException: An exception occurred in the change event producer. This connector will be stopped. 
              at io.debezium.pipeline.ErrorHandler.setProducerThrowable(ErrorHandler.java:72) 
              at io.debezium.connector.oracle.xstream.LcrEventHandler.processLCR(LcrEventHandler.java:130) 
              at oracle.streams.XStreamOut.XStreamOutReceiveLCRCallbackNative(Native Method) 
              at oracle.streams.XStreamOut.receiveLCRCallback(Unknown Source) 
              at io.debezium.connector.oracle.xstream.XstreamStreamingChangeEventSource.execute(XstreamStreamingChangeEventSource.java:125) 
              at io.debezium.connector.oracle.xstream.XstreamStreamingChangeEventSource.execute(XstreamStreamingChangeEventSource.java:45) 
              at io.debezium.pipeline.ChangeEventSourceCoordinator.streamEvents(ChangeEventSourceCoordinator.java:205) 
              at io.debezium.pipeline.ChangeEventSourceCoordinator.executeChangeEventSources(ChangeEventSourceCoordinator.java:172) 
              at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:118) 
              at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) 
              at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) 
              at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) 
              at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) 
              at java.base/java.lang.Thread.run(Thread.java:834) 
      Caused by: java.util.ConcurrentModificationException 
              at java.base/java.util.ArrayList.forEach(ArrayList.java:1543) 
              at io.debezium.connector.oracle.OracleSchemaChangeEventEmitter.lambda$emitSchemaChangeEvent$1(OracleSchemaChangeEventEmitter.java:100) 
              at io.debezium.relational.ddl.DdlChanges.getEventsByDatabase(DdlChanges.java:57) 
              at io.debezium.connector.oracle.OracleSchemaChangeEventEmitter.emitSchemaChangeEvent(OracleSchemaChangeEventEmitter.java:99) 
              at io.debezium.pipeline.EventDispatcher.dispatchSchemaChangeEvent(EventDispatcher.java:392) 
              at io.debezium.connector.oracle.xstream.LcrEventHandler.dispatchSchemaChangeEvent(LcrEventHandler.java:257) 
              at io.debezium.connector.oracle.xstream.LcrEventHandler.processLCR(LcrEventHandler.java:120) 
              ... 12 more 
      [2023-07-11 14:55:24,657] INFO Stopping down connector (io.debezium.connector.common.BaseSourceTask:294) 
      [2023-07-11 14:55:25,205] INFO Connection gracefully closed (io.debezium.jdbc.JdbcConnection:947) 
      [2023-07-11 14:55:25,205] INFO Finished streaming (io.debezium.pipeline.ChangeEventSourceCoordinator:206)

       

       

      What Debezium connector do you use and what version?

      Debezium Oracle Plugin 2.3.0

      What is the connector configuration?

      {
             "name": "oracle_tsm",
             "config": {
                 "connector.class": "io.debezium.connector.oracle.OracleConnector",
                 "database.user": "********",
                 "database.dbname": "********",
                 "rac.nodes": "********,********",
                 "tasks.max": "1",
                 "database.connection.adapter": "xstream",
                 "schema.history.internal.kafka.bootstrap.servers": "********:9093,********:9093,********:9093",
                 "database.port": "1521",
                 "tombstones.on.delete": "false",
                 "key.converter.schemas.enable": "false",
                 "topic.prefix": "oracle_tsm",
                 "decimal.handling.mode": "string",
                 "schema.history.internal.kafka.topic": "oracle_tsm.schema-changes.inventory",
                 "database.hostname": "********",
                 "database.password": "********",
                 "value.converter.schemas.enable": "false",
                 "name": "oracle_tsm",
                 "schema.history.internal.skip.unparseable.ddl": "true",
                 "database.out.server.name": "dbzxout",
                 "schema.include.list": "tft_tsm",
                 "value.converter": "org.apache.kafka.connect.json.JsonConverter",
                 "key.converter": "org.apache.kafka.connect.json.JsonConverter",
                 "snapshot.mode": "schema_only",
                 "signal.poll.interval.ms":"5000",
                 "message.key.columns": "*********"
             }
          } 

      What is the captured database version and mode of depoyment?

      Oracle 11.2.0.4 Rac mode

      What behaviour do you expect?

      No exceptions.

      What behaviour do you see?

      DBA truncated table TFTONG.TFT_TSM.TEMP_LCQ_09, then errors happened

      Do you see the same behaviour using the latest relesead Debezium version?

      Yes.

      Do you have the connector logs, ideally from start till finish?

      If need, I will provide it.

            ccranfor@redhat.com Chris Cranford
            ddsr8013 ddsr 8013
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: