Details
-
Bug
-
Resolution: Done
-
Major
-
None
-
None
-
False
-
None
-
False
Description
Hello!
I have found the bug in single quote repilacion '.
When there is update or insert event then:
The value in oracle=Test'quote
Value in kafka topic = Test''quote
{"schema":\{"type":"struct","fields":[{"type":"string","optional":false,"field":"ID"}],"optional":false,"name":"dev_bodb.MZIGISOVS.QUOTE_TEST.Key"},"payload":\{"ID":"2"}} \{"schema":{"type":"struct","fields":[{"type":"string","optional":true,"field":"QUOTE_FIELD"},\{"type":"string","optional":false,"field":"ID"}],"optional":false,"name":"dev_bodb.MZIGISOVS.QUOTE_TEST.Value"},"payload":\{"QUOTE_FIELD":"test''quote","ID":"2"}}
If there is initial snapshot:
The value in oracle=test'quote
Value in kafka topic = test'quote
{"schema":\{"type":"struct","fields":[{"type":"string","optional":false,"field":"ID"}],"optional":false,"name":"dev_bodb.MZIGISOVS.QUOTE_TEST.Key"},"payload":\{"ID":"2"}} \{"schema":{"type":"struct","fields":[{"type":"string","optional":true,"field":"QUOTE_FIELD"},\{"type":"string","optional":false,"field":"ID"}],"optional":false,"name":"dev_bodb.MZIGISOVS.QUOTE_TEST.Value"},"payload":\{"QUOTE_FIELD":"test'quote","ID":"2"}}
I have made the debug of this logminerParser and found that parser reads the value from oracle redo log and make mistake in parser, as I correctly understand perhaps the problem is here, could you take look:
debezium-connector-oracle\src\main\java\io\debezium\connector\oracle\logminer\parser\LogMinerDmlParser.java -> 363 row
How to reproduce:
Create table:
CREATE TABLE QUOTE_TEST ( QUOTE_FIELD VARCHAR(20), ID NUMBER, PRIMARY KEY(ID) );
INSERT INTO QUOTE_TEST (QUOTE_FIELD,ID) VALUES ('test''quote',1);
Create config.properties:
name=oracle-source-quote-test connector.class=io.debezium.connector.oracle.OracleConnector database.hostname= database.port= database.user= database.password= database.dbname= database.server.name= tasks.max=1 heartbeat.interval.ms=60000 database.history.kafka.bootstrap.servers=localhost:9092 database.history.kafka.topic=schema-changes.inventory database.history.skip.unparseable.ddl=true database.history.store.only.captured.tables.ddl=true schema.include.list={{yourShema}} table.include.list={{yourShema}}.QUOTE_TEST column.include.list={{yourShema}}.QUOTE_TEST.QUOTE_FIELD,{{yourShema}}.QUOTE_TEST.ID database.connection.adapter=logminer decimal.handling.mode=string log.mining.strategy=online_catalog event.processing.failure.handling.mode=skip time.precision.mode=connect include.schema.changes=false transforms=unwrap,route transforms.unwrap.type=io.debezium.transforms.ExtractNewRecordState transforms.unwrap.drop.tombstones=false transforms.unwrap.delete.handling.mode=none transforms.route.type=org.apache.kafka.connect.transforms.RegexRouter transforms.route.regex=([^.]+)\\.([^.]+)\\.([^.]+) transforms.route.replacement=$3
Start zookeeper/kafka/debezium
Wait for initial snapshot and check value of QUOTE_TEST.QUOTE_FIELD.
make update or insert:
update {{yourSchema}}.QUOTE_TEST set QUOTE_FIELD = 'test_quo''te' where id = 1;
check value of QUOTE_TEST.QUOTE_FIELD.
Thank you!
