Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-9132

'||' in ORACLE NVARCHAR data will cause exception

XMLWordPrintable

    • Important

      In order to make your issue reports as actionable as possible, please provide the following information, depending on the issue type.

      Bug report

      For bug reports, provide this information, please:

      What Debezium connector do you use and what version?

       
      debezium-connector-oracle:3.2.0.Beta2
       

      What is the connector configuration?

       

      Map<String, String> props = new HashMap<>();
props.put("connector.class","io.debezium.connector.oracle.OracleConnector");
props.put("tasks.max","1");
// 内存存储schema
props.put("schema.history.internal","io.debezium.relational.history.MemorySchemaHistory");

props.put("transforms.unwrap.delete.handling.mode","drop");
props.put("log.mining.transaction.retention.hours","2");

props.put("unavailable.value.placeholder","NA");
// LOB开关
props.put("lob.enabled","false");
props.put("log.mining.strategy","hybrid");
props.put("topic.prefix","G_D_C");
props.put("decimal.handling.mode","precise");
// DDL解析不了不要报错终止任务
props.put("database.history.skip.unparseable.ddl","true");
props.put("errors.max.retries","3");

props.put("time.precision.mode","connect");
props.put("database.server.name","task1_1553_server");
props.put("event.processing.failure.handling.mode","warn");
props.put("heartbeat.interval.ms","1000");
props.put("database.ssl.mode","disabled");
props.put("log.mining.batch.size.default", "30000");

props.put("snapshotSchema","true");
props.put("snapshot.mode","schema_only");

props.put("database.sidSeparator","/");
props.put("database.hostname","10.69.66.75");
props.put("database.port","8314");
props.put("database.dbname","helowin");
props.put("database.user","system");
props.put("database.password","system");
props.put("schema.include.list","TEST");
props.put("table.include.list","TEST.A_TEST");

// 基于配置的snapshoter
props.put("snapshot.mode.custom.name","custom");
// 禁止锁表
props.put("snapshot.locking.mode", "none");
props.put("snapshot.mode.configuration.based.snapshot.schema","true");
// 只对需要捕获的表进行结构加载
props.put("schema.history.internal.store.only.captured.tables.ddl", "true");
// 事务查询模式
props.put("log.mining.query.filter.mode", "in");
// 事务缓存模式 memory infinispan_embedded
props.put("log.mining.buffer.type","memory");

      What is the captured database version and mode of deployment?

      (E.g. on-premises, with a specific cloud provider, etc.)

      Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - 64bit Production

      on virtual machine

      What behavior do you expect?

      when i save chinese character '中国||武汉' in to NVARCHAR2(500), I wish get the real value '中国||武汉'

      What behavior do you see?

      Actually  It return me this values.

      UNISTR('\4E2D\56FD||\6B66\6C49')UNISTR('\4E2D\56FD||\6B66\6C49')

      Do you see the same behaviour using the latest released Debezium version?

      (Ideally, also verify with latest Alpha/Beta/CR version)

      it's exists from  2.6.1.Final

      Do you have the connector logs, ideally from start till finish?

      (You might be asked later to provide DEBUG/TRACE level log)

      this contants result to the failure.

      io.debezium.connector.oracle.logminer.UnistrHelper#CONCATENATION_PATTERN

       

      this code use '||' to split data, but if the business data contains '||' ,it will result failure.

      How to reproduce the issue using our tutorial deployment?

      create a table column , specify the type with NVARCHAR2(500);

      store value '中国||武汉' into this column.

      And then we can't recieive the real value '中国||武汉'

      Feature request or enhancement

      For feature requests or enhancements, provide this information, please:

      Which use case/requirement will be addressed by the proposed feature?

      <Your answer>

      Implementation ideas (optional)

      io.debezium.connector.oracle.logminer.UnistrHelper#CONCATENATION_PATTERN

      private static final Pattern CONCATENATION_PATTERN = Pattern.compile("\\|\\|(?=(?:[^']|'[^']*')*$)");



// this regrex pattern means if || quoted by ' , it can't be a splliter.

       
       

              ccranfor@redhat.com Chris Cranford
              williamxww xiang vv (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Created:
                Updated:
                Resolved: