Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-4891

Single quotes replication

    XMLWordPrintable

Details

    Description

      Hello!
       
      I have found the bug in single quote repilacion '.
      When there is update or insert event then:
      The value in oracle=Test'quote


      Value in kafka topic = Test''quote

      {"schema":\{"type":"struct","fields":[{"type":"string","optional":false,"field":"ID"}],"optional":false,"name":"dev_bodb.MZIGISOVS.QUOTE_TEST.Key"},"payload":\{"ID":"2"}} \{"schema":{"type":"struct","fields":[{"type":"string","optional":true,"field":"QUOTE_FIELD"},\{"type":"string","optional":false,"field":"ID"}],"optional":false,"name":"dev_bodb.MZIGISOVS.QUOTE_TEST.Value"},"payload":\{"QUOTE_FIELD":"test''quote","ID":"2"}}

       
      If there is initial snapshot:
      The value in oracle=test'quote

      Value in kafka topic = test'quote

      {"schema":\{"type":"struct","fields":[{"type":"string","optional":false,"field":"ID"}],"optional":false,"name":"dev_bodb.MZIGISOVS.QUOTE_TEST.Key"},"payload":\{"ID":"2"}} \{"schema":{"type":"struct","fields":[{"type":"string","optional":true,"field":"QUOTE_FIELD"},\{"type":"string","optional":false,"field":"ID"}],"optional":false,"name":"dev_bodb.MZIGISOVS.QUOTE_TEST.Value"},"payload":\{"QUOTE_FIELD":"test'quote","ID":"2"}}

      I have made the debug of this logminerParser and found that parser reads the value from oracle redo log and make mistake in parser, as I correctly understand perhaps the problem is here, could you take look:

      debezium-connector-oracle\src\main\java\io\debezium\connector\oracle\logminer\parser\LogMinerDmlParser.java -> 363 row

      How to reproduce:

      Create table: 

      CREATE TABLE QUOTE_TEST (
      QUOTE_FIELD VARCHAR(20),
      ID NUMBER,
      PRIMARY KEY(ID)
      );
      INSERT INTO QUOTE_TEST (QUOTE_FIELD,ID)
      VALUES ('test''quote',1);

      Create config.properties: 

      name=oracle-source-quote-test
      connector.class=io.debezium.connector.oracle.OracleConnector
      database.hostname=
      database.port=
      database.user=
      database.password=
      database.dbname=
      database.server.name=
      tasks.max=1
      heartbeat.interval.ms=60000
      database.history.kafka.bootstrap.servers=localhost:9092
      database.history.kafka.topic=schema-changes.inventory
      database.history.skip.unparseable.ddl=true
      database.history.store.only.captured.tables.ddl=true
      schema.include.list={{yourShema}}
      table.include.list={{yourShema}}.QUOTE_TEST
      column.include.list={{yourShema}}.QUOTE_TEST.QUOTE_FIELD,{{yourShema}}.QUOTE_TEST.ID
      database.connection.adapter=logminer
      decimal.handling.mode=string
      log.mining.strategy=online_catalog
      event.processing.failure.handling.mode=skip
      time.precision.mode=connect
      include.schema.changes=false
      
      transforms=unwrap,route
      
      transforms.unwrap.type=io.debezium.transforms.ExtractNewRecordState
      transforms.unwrap.drop.tombstones=false
      transforms.unwrap.delete.handling.mode=none
      
      transforms.route.type=org.apache.kafka.connect.transforms.RegexRouter
      transforms.route.regex=([^.]+)\\.([^.]+)\\.([^.]+)
      transforms.route.replacement=$3

      Start zookeeper/kafka/debezium

      Wait for initial snapshot and check value of QUOTE_TEST.QUOTE_FIELD.

      make update or insert:

      update {{yourSchema}}.QUOTE_TEST set QUOTE_FIELD = 'test_quo''te' where id = 1;

      check value of QUOTE_TEST.QUOTE_FIELD.

      Thank you!

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              mzigisov maksim zigisov (Inactive)
              Votes:
              1 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: