Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-6457

Oracle parallel snapshots do not properly set PDB context when using multitenancy

    XMLWordPrintable

Details

    Description

      In order to make your issue reports as actionable as possible, please provide the following information, depending on the issue type.

      Bug report

      For bug reports, provide this information, please:

      What Debezium connector do you use and what version?

      v2.2.0.Alpha3 (also tried with v2.2.0.Final)

      What is the connector configuration?

       

      Starting OracleConnectorTask with configuration:
      connector.class = io.debezium.connector.oracle.OracleConnector
      auto.create.topics.enable = false
      schema.history.internal.consumer.sasl.jaas.config = ********
      bootstrap.servers = <snipped>
      schema.history.internal.producer.security.protocol = SASL_SSL
      sasl.mechanism = AWS_MSK_IAM
      schema.history.internal.producer.retry.backoff.ms = 1000
      lob.enabled = true
      offset.storage.topic = <snipped>
      interval.handling.mode = string
      errors.retry.delay.initial.ms = 300
      key.converter = org.apache.kafka.connect.json.JsonConverter
      schema.history.internal.producer.sasl.mechanism = AWS_MSK_IAM
      schema.history.internal.consumer.sasl.client.callback.handler.class = software.amazon.msk.auth.iam.IAMClientCallbackHandler
      database.dbname = ORCLCDB
      database.user = c##dbzuser
      offset.storage = org.apache.kafka.connect.storage.KafkaOffsetBackingStore
      schema.history.internal.producer.sasl.client.callback.handler.class = software.amazon.msk.auth.iam.IAMClientCallbackHandler
      internal.log.mining.transaction.snapshot.boundary.mode = all
      schema.history.internal.kafka.bootstrap.servers = <snipped>
      errors.max.retries = -1
      database.password = ********
      name = oracle
      schema.history.internal.producer.sasl.jaas.config = ********
      schema.history.internal.consumer.sasl.mechanism = AWS_MSK_IAM
      schema.history.internal.producer.bootstrap.servers = <snipped>
      max.batch.size = 2048
      skipped.operations = none
      snapshot.mode = initial
      schema.history.internal.consumer.security.protocol = SASL_SSL
      max.queue.size = 8192
      tasks.max = 1
      retry.backoff.ms = 1000
      tombstones.on.delete = false
      topic.prefix = <snipped>
      offset.storage.file.filename =
      schema.history.internal.kafka.topic = <snipped>
      sasl.jaas.config = ********
      offset.storage.partitions = 1
      sasl.client.callback.handler.class = software.amazon.msk.auth.iam.IAMClientCallbackHandler
      value.converter = org.apache.kafka.connect.json.JsonConverter
      log.mining.batch.size.max = 300000
      database.pdb.name = ORCLPDB1
      schema.history.internal.consumer.bootstrap.servers = <snipped>
      log.mining.view.fetch.size = 100000
      offset.flush.timeout.ms = 2000
      errors.retry.delay.max.ms = 10000
      snapshot.max.threads = 4
      database.port = 1521
      schema.history.internal.consumer.retry.backoff.ms = 1000
      offset.flush.interval.ms = 10000
      security.protocol = SASL_SSL
      driver.connection.adapter = logminer
      schema.history.internal = io.debezium.storage.kafka.history.KafkaSchemaHistory
      database.hostname = <snipped>
      schema.name.adjustment.mode = avro
      log.mining.batch.size.min = 10000
      log.mining.batch.size.default = 20000
      offset.storage.replication.factor = 1
      driver.out.server.name = dbzxout
      table.include.list = DEBEZIUM.CUSTOMER,DEBEZIUM.DISTRICT,DEBEZIUM.HISTORY,DEBEZIUM.ITEM,DEBEZIUM.NEW_ORDER,DEBEZIUM.ORDER_LINE,DEBEZIUM.ORDERS,DEBEZIUM.STOCK,DEBEZIUM.WAREHOUSE
      

       

       

       

      What is the captured database version and mode of depoyment?

      (E.g. on-premises, with a specific cloud provider, etc.)

      Oracle 19c running in a Docker Container in AWS ECS

      What behaviour do you expect?

      Parallel Snapshot to succeed when `snapshot.max.threads` is greater than 1.

      What behaviour do you see?

      Running snapshot with `snapshot.max.threads` greater than 1 causes an error: `ORA-00942: table or view does not exist`

      Do you see the same behaviour using the latest relesead Debezium version?

      (Ideally, also verify with latest Alpha/Beta/CR version)

      Yes, I tried upgrading to v2.2.0.Final and the error still occurs

      Do you have the connector logs, ideally from start till finish?

      [io.deb.rel.RelationalSnapshotChangeEventSource] (pool-137-thread-1) Exporting data from table 'ORCLPDB1.DEBEZIUM.CUSTOMER' (1 of 9 tables)
      [io.deb.rel.RelationalSnapshotChangeEventSource] (pool-137-thread-2) Exporting data from table 'ORCLPDB1.DEBEZIUM.DISTRICT' (2 of 9 tables)
      [io.deb.rel.RelationalSnapshotChangeEventSource] (pool-137-thread-3) Exporting data from table 'ORCLPDB1.DEBEZIUM.HISTORY' (3 of 9 tables)
      [io.deb.rel.RelationalSnapshotChangeEventSource] (pool-137-thread-4) Exporting data from table 'ORCLPDB1.DEBEZIUM.ITEM' (4 of 9 tables)
      [io.deb.rel.RelationalSnapshotChangeEventSource] (debezium-oracleconnector-<snipped>-change-event-source-coordinator) Error during snapshot: java.util.concurrent.ExecutionException: org.apache.kafka.connect.errors.ConnectException: Snapshotting of table ORCLPDB1.DEBEZIUM.ITEM failed
              at java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122)
              at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:191)
              at io.debezium.relational.RelationalSnapshotChangeEventSource.createDataEvents(RelationalSnapshotChangeEventSource.java:440)
              at io.debezium.relational.RelationalSnapshotChangeEventSource.doExecute(RelationalSnapshotChangeEventSource.java:157)
              at io.debezium.pipeline.source.AbstractSnapshotChangeEventSource.execute(AbstractSnapshotChangeEventSource.java:76)
              at io.debezium.pipeline.ChangeEventSourceCoordinator.doSnapshot(ChangeEventSourceCoordinator.java:155)
              at io.debezium.pipeline.ChangeEventSourceCoordinator.executeChangeEventSources(ChangeEventSourceCoordinator.java:137)
              at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:109)
              at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
              at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
              at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
              at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
              at java.base/java.lang.Thread.run(Thread.java:833)
      Caused by: org.apache.kafka.connect.errors.ConnectException: Snapshotting of table ORCLPDB1.DEBEZIUM.ITEM failed
              at io.debezium.relational.RelationalSnapshotChangeEventSource.doCreateDataEventsForTable(RelationalSnapshotChangeEventSource.java:564)
              at io.debezium.relational.RelationalSnapshotChangeEventSource.lambda$createDataEventsForTableCallable$5(RelationalSnapshotChangeEventSource.java:492)
              at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
              ... 5 more
      Caused by: java.sql.SQLSyntaxErrorException: ORA-00942: table or view does not exist        at oracle.jdbc.driver.T4CTTIoer11.processError(T4CTTIoer11.java:630)
              at oracle.jdbc.driver.T4CTTIoer11.processError(T4CTTIoer11.java:564)
              at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:1231)
              at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:772)
              at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:299)
              at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:512)
              at oracle.jdbc.driver.T4CStatement.doOall8(T4CStatement.java:123)
              at oracle.jdbc.driver.T4CStatement.executeForDescribe(T4CStatement.java:969)
              at oracle.jdbc.driver.OracleStatement.prepareDefineBufferAndExecute(OracleStatement.java:1271)
              at oracle.jdbc.driver.OracleStatement.executeMaybeDescribe(OracleStatement.java:1149)
              at oracle.jdbc.driver.OracleStatement.executeSQLSelect(OracleStatement.java:1661)
              at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1470)
              at oracle.jdbc.driver.OracleStatement.executeQuery(OracleStatement.java:2055)
              at oracle.jdbc.driver.OracleStatementWrapper.executeQuery(OracleStatementWrapper.java:394)
              at io.debezium.relational.RelationalSnapshotChangeEventSource.doCreateDataEventsForTable(RelationalSnapshotChangeEventSource.java:518)
              ... 7 more
      Caused by: Error : 942, Position : 72, Sql = SELECT "I_ID", "I_IM_ID", "I_NAME", "I_PRICE", "I_DATA" FROM "DEBEZIUM"."ITEM" AS OF SCN 2489610, OriginalSql = SELECT "I_ID", "I_IM_ID", "I_NAME", "I_PRICE", "I_DATA" FROM "DEBEZIUM"."ITEM" AS OF SCN 2489610, Error Msg = ORA-00942: table or view does not exist        at oracle.jdbc.driver.T4CTTIoer11.processError(T4CTTIoer11.java:637)
              ... 21 more 

       

       

      From the log above position 72 is the `.` in `"DEBEZIUM"."ITEM"` and a search on past issues turned up something similar - https://issues.redhat.com/browse/DBZ-4744

      Attachments

        Issue Links

          Activity

            People

              ccranfor@redhat.com Chris Cranford
              bshore-edb Brandon Shore (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: