Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-7474

Oracle ReselectColumnsProcessor does not work with CLOB/BLOB columns when lob.enabled=false

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Not a Bug
    • Icon: Major Major
    • None
    • 2.5.1.Final
    • oracle-connector
    • None
    • False
    • None
    • False

      Bug report

      For bug reports, provide this information, please:

      What Debezium connector do you use and what version?

      debezium-connector-oracle version 2.5.1.Final

      What is the connector configuration?

      {
        "name": "source-test-connector",
        "config": {
          "connector.class": "io.debezium.connector.oracle.OracleConnector",
          "tasks.max": "1",
          "database.hostname": "oracle",
          "database.port": "1521",
          "database.user": "c##dbzuser",
          "database.password": "dbz",
          "database.dbname": "orclcdb",
          "database.pdb.name": "orclpdb1",
          "database.connection.adapter": "logminer",
          "topic.prefix": "dbz",
          "schema.name.adjustment.mode": "avro",
          "table.include.list": "C##DBZUSER.TEST_TABLE",
          "include.schema.changes": "false",
          "schema.history.internal.kafka.bootstrap.servers" : "kafka:9092",
          "schema.history.internal.kafka.topic": "schema-changes.test",
          "heartbeat.interval.ms": "60000",
          "log.mining.strategy": "online_catalog",
          "log.mining.query.filter.mode": "in",
          "post.processors": "reselector",
          "reselector.type": "io.debezium.processors.reselect.ReselectColumnsPostProcessor",
          "reselector.reselect.columns.include.list": "C##DBZUSER.TEST_TABLE:DATA",
          "custom.metric.tags": "connector=source-test-connector",
          "transforms": "unwrap",
          "transforms.unwrap.type": "io.debezium.transforms.ExtractNewRecordState",
          "key.converter": "org.apache.kafka.connect.json.JsonConverter",
          "key.converter.schemas.enable": "false",
          "value.converter": "org.apache.kafka.connect.json.JsonConverter",
          "value.converter.schemas.enable": "false"
        }
      }
      

      What is the captured database version and mode of deployment?

      Oracle Database 19, Docker

      What behaviour do you expect?

      Oracle connector reselects CLOB/BLOB columns by ReselectColumnsProcessor.

      What behaviour do you see?

      Oracle connector ignores reselection for CLOB/BLOB columns by ReselectColumnsProcessor when lob.enabled=false.

      Do you see the same behaviour using the latest relesead Debezium version?

      Yes

      How to reproduce the issue using our tutorial deployment?

      1. Create a new table:

      CREATE TABLE c##dbzuser.test_table (
        id   NUMBER(10) NOT NULL PRIMARY KEY,
        data CLOB
      );
      

      2. Create a new connector with ReselectColumnsPostProcessor for CLOB column:

      curl -X POST -H "Accept:application/json" -H "Content-Type:application/json" http://localhost:8083/connectors -d '
      {
        "name": "source-test-connector",
        "config": {
          "connector.class": "io.debezium.connector.oracle.OracleConnector",
          "tasks.max": "1",
          "database.hostname": "oracle",
          "database.port": "1521",
          "database.user": "c##dbzuser",
          "database.password": "dbz",
          "database.dbname": "orclcdb",
          "database.pdb.name": "orclpdb1",
          "database.connection.adapter": "logminer",
          "topic.prefix": "dbz",
          "schema.name.adjustment.mode": "avro",
          "table.include.list": "C##DBZUSER.TEST_TABLE",
          "include.schema.changes": "false",
          "schema.history.internal.kafka.bootstrap.servers" : "kafka:9092",
          "schema.history.internal.kafka.topic": "schema-changes.test",
          "heartbeat.interval.ms": "60000",
          "log.mining.strategy": "online_catalog",
          "log.mining.query.filter.mode": "in",
          "post.processors": "reselector",
          "reselector.type": "io.debezium.processors.reselect.ReselectColumnsPostProcessor",
          "reselector.reselect.columns.include.list": "C##DBZUSER.TEST_TABLE:DATA",
          "custom.metric.tags": "connector=source-test-connector",
          "transforms": "unwrap",
          "transforms.unwrap.type": "io.debezium.transforms.ExtractNewRecordState",
          "key.converter": "org.apache.kafka.connect.json.JsonConverter",
          "key.converter.schemas.enable": "false",
          "value.converter": "org.apache.kafka.connect.json.JsonConverter",
          "value.converter.schemas.enable": "false"
        }
      }'
      

      3. Insert a new record into the table:

      INSERT INTO c##dbzuser.test_table (id, data)
      VALUES (1, TO_CLOB('data'));
      
      commit;
      

      4. Check messages in the topic:
      ER:

      {
      	"ID": 1,
      	"DATA": "data"
      }
      

      AR:

      {
      	"ID": 1,
      	"DATA": "__debezium_unavailable_value"
      }
      

      Feature request or enhancement

      <Your answer>

      Implementation ideas (optional)

      This may be by design, but it seems that OracleValueConverters used in the ReselectColumnsPostProcessor must always be configured with LOB support enabled.

            ccranfor@redhat.com Chris Cranford
            andrey.pustovetov@gmail.com Andrey Pustovetov
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: