Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-2624

SqlExceptions using dbz with Oracle on RDS online logs and logminer

    XMLWordPrintable

Details

    • False
    • False
    • Undefined
    • Hide
      1. Download [instant client and extract
      2. Build docker image with dbz-dockerfile
      3. Create Oracle Instance on RDS
      4. Create tables with dummy data in Oracle
      5. Submit the following payload for oracle connector:
      {    "name": "oracle-cdc",    "config": {        "connector.class" : "io.debezium.connector.oracle.OracleConnector",        "tasks.max" : "1",        "database.server.name" : "cdc-4",        "database.hostname" : "*",        "database.port" : "1521",        "database.user" : "*",        "database.schema" : "TESTER",        "database.password" : "*",        "database.dbname" : "*",        "database.out.server.name" : "dbzxout",        "database.history.kafka.bootstrap.servers" : "*:9092",        "table.include.list": "TESTER.EMPLOYEES,TESTER.COUNTRIES,TESTER.INVENTORIES",        "database.history.kafka.topic": "oracle_cdc",        "database.connection.adapter": "logminer",        "log.mining.strategy": "online_catalog"    }}

       

      Show
      Download  [instant client  and extract Build docker image with dbz-dockerfile Create Oracle Instance on RDS Create tables with dummy data in Oracle Submit the following payload for oracle connector: { "name" : "oracle-cdc" , "config" : { "connector.class" : "io.debezium.connector.oracle.OracleConnector" , "tasks.max" : "1" , "database.server.name" : "cdc-4" , "database.hostname" : "*" , "database.port" : "1521" , "database.user" : "*" , "database.schema" : "TESTER" , "database.password" : "*" , "database.dbname" : "*" , "database.out.server.name" : "dbzxout" , "database.history.kafka.bootstrap.servers" : "*:9092" , "table.include.list" : "TESTER.EMPLOYEES,TESTER.COUNTRIES,TESTER.INVENTORIES" , "database.history.kafka.topic" : "oracle_cdc" , "database.connection.adapter" : "logminer" , "log.mining.strategy" : "online_catalog" }}  

    Description

      I have been testing out the Debezium Oracle connector with Oracle on RDS v12 and v19, and am using instantclient v12.2.0.1.0.

      On Oracle RDS v12:

      The debezium connector fails with the following error:

      {"log":"org.apache.kafka.connect.errors.ConnectException: An exception occurred in the change event producer. This connector will be stopped.\n","stream":"stdout","time":"2020-09-29T12:04:15.219012535Z"}
{"log":"\u0009at io.debezium.pipeline.ErrorHandler.setProducerThrowable(ErrorHandler.java:42)\n","stream":"stdout","time":"2020-09-29T12:04:15.219027901Z"}
{"log":"\u0009at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:121)\n","stream":"stdout","time":"2020-09-29T12:04:15.219033471Z"}
{"log":"\u0009at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)\n","stream":"stdout","time":"2020-09-29T12:04:15.219038019Z"}
{"log":"\u0009at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)\n","stream":"stdout","time":"2020-09-29T12:04:15.219042309Z"}
{"log":"\u0009at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)\n","stream":"stdout","time":"2020-09-29T12:04:15.219048461Z"}
{"log":"\u0009at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)\n","stream":"stdout","time":"2020-09-29T12:04:15.21905273Z"}
{"log":"\u0009at java.base/java.lang.Thread.run(Thread.java:834)\n","stream":"stdout","time":"2020-09-29T12:04:15.219057141Z"}
{"log":"Caused by: java.lang.RuntimeException: java.sql.SQLException: Stream has already been closed\n","stream":"stdout","time":"2020-09-29T12:04:15.219061419Z"}
{"log":"\u0009at io.debezium.pipeline.source.AbstractSnapshotChangeEventSource.execute(AbstractSnapshotChangeEventSource.java:76)\n","stream":"stdout","time":"2020-09-29T12:04:15.219065694Z"}
{"log":"\u0009at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:105)\n","stream":"stdout","time":"2020-09-29T12:04:15.219070438Z"}
{"log":"\u0009... 5 more\n","stream":"stdout","time":"2020-09-29T12:04:15.219074818Z"}

      On Oracle RDS v19:

      The debezium connector fails with the following error:

      2020-10-02 03:56:01,306 INFO   ||  Snapshot step 1 - Preparing   [io.debezium.relational.RelationalSnapshotChangeEventSource]
2020-10-02 03:56:01,353 INFO   ||  Snapshot step 2 - Determining captured tables   [io.debezium.relational.RelationalSnapshotChangeEventSource]
2020-10-02 03:56:01,544 INFO   ||  Snapshot step 3 - Locking captured tables   [io.debezium.relational.RelationalSnapshotChangeEventSource]
2020-10-02 03:56:01,545 INFO   ||  Snapshot step 4 - Determining snapshot offset   [io.debezium.relational.RelationalSnapshotChangeEventSource]
2020-10-02 03:56:01,551 INFO   ||  Snapshot step 5 - Reading structure of captured tables   [io.debezium.relational.RelationalSnapshotChangeEventSource]
2020-10-02 03:56:01,551 INFO   ||  Snapshot step 6 - Persisting schema history   [io.debezium.relational.RelationalSnapshotChangeEventSource]
2020-10-02 03:56:01,552 INFO   ||  Snapshot step 7 - Snapshotting data   [io.debezium.relational.RelationalSnapshotChangeEventSource]
2020-10-02 03:56:01,553 INFO   ||  Snapshot - Final stage   [io.debezium.pipeline.source.AbstractSnapshotChangeEventSource]
2020-10-02 03:56:01,553 INFO   ||  Snapshot ended with SnapshotResult [status=COMPLETED, offset=OracleOffsetContext [scn=1727612]]   [io.debezium.pipeline.ChangeEventSourceCoordinator]
2020-10-02 03:56:01,553 WARN   ||  After applying the include/exclude list filters, no changes will be captured. Please check your configuration!   [io.debezium.relational.RelationalDatabaseSchema]
2020-10-02 03:56:01,554 INFO   ||  Connected metrics set to 'true'   [io.debezium.pipeline.metrics.StreamingChangeEventSourceMetrics]
2020-10-02 03:56:01,554 INFO   ||  Starting streaming   [io.debezium.pipeline.ChangeEventSourceCoordinator]
2020-10-02 03:56:01,555 INFO   ||  Requested thread factory for connector OracleConnector, id = cdc-1 named = transactional-buffer   [io.debezium.util.Threads]
2020-10-02 03:56:01,555 WARN   ||   Cannot apply 0 whitelisted tables condition   [io.debezium.connector.oracle.logminer.SqlUtils]
2020-10-02 03:56:01,611 ERROR  ||  Mining session stopped due to the java.lang.IllegalStateException: The online log files do not contain offset SCN: 1727612, re-snapshot is required.    [io.debezium.connector.oracle.logminer.LogMinerHelper]
2020-10-02 03:56:01,612 ERROR  ||  Producer failure   [io.debezium.pipeline.ErrorHandler]
java.lang.IllegalStateException: The online log files do not contain offset SCN: 1727612, re-snapshot is required.
    at io.debezium.connector.oracle.logminer.LogMinerHelper.setRedoLogFilesForMining(LogMinerHelper.java:318)
    at io.debezium.connector.oracle.logminer.LogMinerStreamingChangeEventSource.execute(LogMinerStreamingChangeEventSource.java:135)
    at io.debezium.pipeline.ChangeEventSourceCoordinator.streamEvents(ChangeEventSourceCoordinator.java:140)
    at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:113)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:834)

      What I've noticed:

      1. I originally used instantclient V19.8 with Oracle RDS v19, but I got the same error that I'm getting with Oracle RDS v12 (Stream has already closed). Downgrading instantclient to v12.2 resulted in the error shown above. (The online log files do not contain offset SCN: 1727612, re-snapshot is required.).
      2. I tried testing different versions of instantclient with Oracle RDS v12. But nothing seems to work.
      3. For the current issue that I am facing with Oracle RDS v19. As seen in the error logs, the oracle-debezium-connector successfully takes snapshots of the table. But is unable to find the SCN that it determined.

      Attachments

        Issue Links

          is duplicated by

          Bug - A problem that impairs or prevents the functions of the product. DBZ-2677 SQLException: Stream has already been closed during Schema Read

          • Critical - To be worked on immediately following BLOCKER issues.
          • Closed
          is related to

          Task - Represents a small unit of work that is not end-user facing. DBZ-2690 Pull oracle specific changes for reading table column metadata into debezium-core

          • Minor - To be worked after urgent, high, and medium priorities are resolved. This term is chosen when the severity of the issue is low, or the complexity or effort to correct it may be higher, relatively speaking. For low priority issues, known workarounds exist or are not needed due to the trivial effort needed to address the issue.
          • Closed

          Activity

            People

              ccranfor@redhat.com Chris Cranford
              mngeow Marcus ngeow (Inactive)
              Votes:
              3 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: