Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-4424

Oracle Connector not receiving DELETE operations

    XMLWordPrintable

Details

    • False
    • False
    • Hide
      1. Setup logminer on database
      2. Create database, tables and user
      3. Grant permissions
      4. Create debezium user
      5. Grant permissions
      6. Setup Kafka-Connect environment with debezium image & Oracle drivers
      7. POST Oracle connector
      8. Start making changes to data
      Show
      Setup logminer on database Create database, tables and user Grant permissions Create debezium user Grant permissions Setup Kafka-Connect environment with debezium image & Oracle drivers POST Oracle connector Start making changes to data

    Description

      Source Connector is working fine with Inserts, Updates but is not catching any DELETE op.

      We've tested on both 11g and 19c dbs trying to figure out if it was something with incompatibility between versions. We've also follow configurations steps properly for logminer and debezium properties.

      Source Connector:

      {
          "name": "inventory-connector",
          "config": {
            "connector.class": "io.debezium.connector.oracle.OracleConnector",
            "log.mining.strategy": "online_catalog",
            "log.mining.archive.log.hours": "1",
            "database.server.name": "inventory",
            "database.hostname": "x.x.x.x",
            "database.user": "c##dbzuser",
            "database.password": "xx",
            "database.port": "1521",
            "database.history.kafka.topic": "schema-changes.inventory",
            "database.history.kafka.bootstrap.servers": "kafka:9092",
            "database.url": "jdbc:oracle:thin:@//x.x.x.x:1521/XE",
            "database.dbname": "XE",
            "table.include.list": "DEMO.PRUEBA",
            "transforms": "extract",
            "transforms.extract.add.fields": "op",
            "transforms.extract.type": "io.debezium.transforms.ExtractNewRecordState",
            "decimal.handling.mode":"double",
            "tasks.max": "1"
          }
      }

      debezium log (19c)

      2021-12-08 19:34:31,969 INFO   ||  1 records sent during previous 00:01:53.51, last recorded offset: {commit_scn=3458085, transaction_id=null, scn=3458084}   [io.debezium.connector.common.BaseSourceTask]
      2021-12-08 19:34:34,785 INFO   Oracle|ORCLCDB|streaming  Oracle Session UGA 18.09MB (max = 47.34MB), PGA 21.14MB (max = 101.33MB)   [io.debezium.connector.oracle.logminer.LogMinerStreamingChangeEventSource]
      2021-12-08 19:34:37,780 INFO   Oracle|ORCLCDB|streaming  Oracle Session UGA 18.09MB (max = 47.34MB), PGA 21.14MB (max = 101.33MB)   [io.debezium.connector.oracle.logminer.LogMinerStreamingChangeEventSource]
      2021-12-08 19:34:40,979 INFO   Oracle|ORCLCDB|streaming  Oracle Session UGA 18.09MB (max = 47.34MB), PGA 21.14MB (max = 101.33MB)   [io.debezium.connector.oracle.logminer.LogMinerStreamingChangeEventSource]
      2021-12-08 19:34:43,978 INFO   Oracle|ORCLCDB|streaming  Oracle Session UGA 18.09MB (max = 47.34MB), PGA 21.14MB (max = 101.33MB)   [io.debezium.connector.oracle.logminer.LogMinerStreamingChangeEventSource]
      2021-12-08 19:34:43,995 INFO   ||  1 records sent during previous 00:00:12.026, last recorded offset: {commit_scn=3458142, transaction_id=null, scn=3458141}   [io.debezium.connector.common.BaseSourceTask]
      2021-12-08 19:34:47,166 INFO   Oracle|ORCLCDB|streaming  Oracle Session UGA 18.09MB (max = 47.34MB), PGA 21.14MB (max = 101.33MB)   [io.debezium.connector.oracle.logminer.LogMinerStreamingChangeEventSource]
      2021-12-08 19:34:50,155 INFO   Oracle|ORCLCDB|streaming  Oracle Session UGA 18.09MB (max = 47.34MB), PGA 21.14MB (max = 101.33MB)   [io.debezium.connector.oracle.logminer.LogMinerStreamingChangeEventSource]
      2021-12-08 19:34:53,347 INFO   Oracle|ORCLCDB|streaming  Oracle Session UGA 18.09MB (max = 47.34MB), PGA 21.14MB (max = 101.33MB)   [io.debezium.connector.oracle.logminer.LogMinerStreamingChangeEventSource]
      2021-12-08 19:34:56,339 INFO   Oracle|ORCLCDB|streaming  Oracle Session UGA 18.09MB (max = 47.34MB), PGA 21.14MB (max = 101.33MB)   [io.debezium.connector.oracle.logminer.LogMinerStreamingChangeEventSource]

      kafka topic (19c)

      [root@kfkbrokerfr3 source]# docker exec -it kafka_kafka_1 kafka-console-consumer --bootstrap-server kafka:9092 --
      topic ORCLCDB.C__DEMO.PRUEBA --from-beginning
      {"schema":{"type":"struct","fields":[{"type":"double","optional":false,"field":"PERSON_ID"},{"type":"string","optional":false,"field":"FIRST_NAME"},{"type":"string","optional":false,"field":"LAST_NAME"},{"type":"string","optional":true,"field":"__op"}],"optional":false,"name":"ORCLCDB.C__DEMO.PRUEBA.Value"},"payload":{"PERSON_ID":2.0,"FIRST_NAME":"Alejandro","LAST_NAME":"Melendez","__op":"r"}}
      {"schema":{"type":"struct","fields":[{"type":"double","optional":false,"field":"PERSON_ID"},{"type":"string","optional":false,"field":"FIRST_NAME"},{"type":"string","optional":false,"field":"LAST_NAME"},{"type":"string","optional":true,"field":"__op"}],"optional":false,"name":"ORCLCDB.C__DEMO.PRUEBA.Value"},"payload":{"PERSON_ID":1.0,"FIRST_NAME":"Ale","LAST_NAME":"Melendez","__op":"r"}}
      {"schema":{"type":"struct","fields":[{"type":"double","optional":false,"field":"PERSON_ID"},{"type":"string","optional":false,"field":"FIRST_NAME"},{"type":"string","optional":false,"field":"LAST_NAME"},{"type":"string","optional":true,"field":"__op"}],"optional":false,"name":"ORCLCDB.C__DEMO.PRUEBA.Value"},"payload":{"PERSON_ID":7.0,"FIRST_NAME":"Prueba","LAST_NAME":"Ejemplo","__op":"c"}}
      {"schema":{"type":"struct","fields":[{"type":"double","optional":false,"field":"PERSON_ID"},{"type":"string","optional":false,"field":"FIRST_NAME"},{"type":"string","optional":false,"field":"LAST_NAME"},{"type":"string","optional":true,"field":"__op"}],"optional":false,"name":"ORCLCDB.C__DEMO.PRUEBA.Value"},"payload":{"PERSON_ID":7.0,"FIRST_NAME":"Nombre","LAST_NAME":"Ejemplo","__op":"u"}} 

      Attachments

        Activity

          People

            Unassigned Unassigned
            r.lam Ricardo Lam (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: