Bug report

When we enabled TRACE logging and ran the application, we noticed Parsing errors especially if the DDL of Oracle table includes DEFAULT keyword expression. For example , DEFAULT (0), DEFAULT(9999.99), etc gives Conveter issues. One of the examples is shown below.

{"@timestamp":"2024-09-23T18:54:11.445Z","ecs.version":"1.2.0","log.level":"DEBUG","message":"Building schema for column HAS_PHOTOS of type 2 named NUMBER with constraints (1,Optional[0])","process.thread.name":" debezium-oracleconnector-ed50-scan1-change-event-source-coordinator","log.logger":"io.debezium.connector.oracle.OracleValueConverters","dbz.connectorContext":"snapshot","dbz.connectorName":"ed50-scan1","dbz.conne ctorType":"Oracle","dbz.databaseName":"MYTABLE","dbz.taskId":"0"} {"@timestamp":"2024-09-23T18:54:11.445Z","ecs.version":"1.2.0","log.level":"TRACE","message":"Value from data object: *** (0) ***","process.thread.name":"debezium-oracleconnector-ed50-scan1-change-event-source-co ordinator","log.logger":"io.debezium.connector.oracle.OracleValueConverters","dbz.connectorContext":"snapshot","dbz.connectorName":"ed50-scan1","dbz.connectorType":"Oracle","dbz.databaseName":"MYTABLE","dbz.task Id":"0"}

*

{"@timestamp":"2024-09-23T18:54:11.445Z","ecs.version":"1.2.0","log.level":"WARN","message":"Cannot parse column default value '(0)' to type '2'.  Expression evaluation is not supported.","process.thread.name":"d* ebezium-oracleconnector-ed50-scan1-change-event-source-coordinator","log.logger":"io.debezium.connector.oracle.OracleDefaultValueConverter","dbz.connectorContext":"snapshot","dbz.connectorName":"ed50-scan1","dbz. connectorType":"Oracle","dbz.databaseName":"MYTABLE","dbz.taskId":"0"}

*

{"@timestamp":"2024-09-23T18:54:11.446Z","ecs.version":"1.2.0","log.level":"DEBUG","message":"Parsing failed due to error","process.thread.name":"debezium-oracleconnector-ed50-scan1-change-event-source-coordinator","log.logger":"io.debezium.connector.oracle.OracleDefaultValueConverter","dbz.connectorContext":"snapshot","dbz.connectorName":"ed50-scan1","dbz.connectorType":"Oracle","dbz.databaseName":"MYTABLE","dbz.taskId":"0","error.type":"java.lang.NumberFormatException","error.message":"Character ( is neither a decimal digit number, decimal point, nor \"e\" notation exponential mark.","error.stack_trace":"java.lang.NumberFormatException: Character ( is neither a decimal digit number, decimal point, nor \"e\" notation exponential mark.\n\tat java.base/java.math.BigDecimal.<init>(Unknown Source)\n\tat java.base/java.math.BigDecimal.<init>(Unknown Source)\n\tat java.base/java.math.BigDecimal.<init>(Unknown Source)\n\tat io.debezium.jdbc.JdbcValueConverters.lambda$toBigDecimal$46(JdbcValueConverters.java:1094)\n\tat* io.debezium.jdbc.JdbcValueConverters.convertValue(JdbcValueConverters.java:1335)\n\tat io.debezium.jdbc.JdbcValueConverters.toBigDecimal(JdbcValueConverters.java:1068)\n\tat io.debezium.connector.oracle.OracleValueConverters.convertDecimal(OracleValueConverters.java:428)\n\tat io.debezium.connector.oracle.OracleValueConverters.convertNumeric(OracleValueConverters.java:442)\n\tat io.debezium.connector.oracle.OracleValueConverters.convertVariableScale(OracleValueConverters.java:558)\n\tat io.debezium.connector.oracle.OracleValueConverters.lambda$getNumericConverter$10(OracleValueConverters.java:239)\n\tat io.debezium.connector.oracle.OracleDefaultValueConverter.convertDefaultValue(OracleDefaultValueConverter.java:94)\n\tat io.debezium.connector.oracle.OracleDefaultValueConverter.parseDefaultValue(OracleDefaultValueConverter.java:67)\n\tat io.debezium.relational.TableSchemaBuilder.lambda$parseDefaultValue$9(TableSchemaBuilder.java:505)\n\tat java.base/java.util.Optional.flatMap(Unknown Source)\n\tat io.debezium.relational.TableSchemaBuilder.parseDefaultValue(TableSchemaBuilder.java:505)\n\tat io.debezium.relational.TableSchemaBuilder.addField(TableSchemaBuilder.java:439)\n\tat io.debezium.relational.TableSchemaBuilder.lambda$create$2(TableSchemaBuilder.java:200)\n\tat java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(Unknown Source)\n\tat java.base/java.util.stream.ReferencePipeline$2$1.accept(Unknown Source)\n\tat java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(Unknown Source)\n\tat java.base/java.util.stream.AbstractPipeline.copyInto(Unknown Source)\n\tat java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(Unknown Source)\n\tat java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(Unknown Source)\n\tat java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(Unknown Source)\n\tat java.base/java.util.stream.AbstractPipeline.evaluate(Unknown Source)\n\tat java.base/java.util.stream.ReferencePipeline.forEach(Unknown Source)\n\tat io.debezium.relational.TableSchemaBuilder.create(TableSchemaBuilder.java:198)\n\tat io.debezium.relational.RelationalDatabaseSchema.buildAndRegisterSchema(RelationalDatabaseSchema.java:122)\n\tat io.debezium.connector.oracle.OracleDatabaseSchema.buildAndRegisterSchema(OracleDatabaseSchema.java:127)\n\tat io.debezium.connector.oracle.OracleDatabaseSchema.lambda$applySchemaChange$0(OracleDatabaseSchema.java:101)\n\tat java.base/java.lang.Iterable.forEach(Unknown Source)\n\tat io.debezium.connector.oracle.OracleDatabaseSchema.applySchemaChange(OracleDatabaseSchema.java:100)\n\tat io.debezium.pipeline.EventDispatcher$SchemaChangeEventReceiver.schemaChangeEvent(EventDispatcher.java:680)\n\tat io.debezium.relational.RelationalSnapshotChangeEventSource.lambda$createSchemaChangeEventsForTables$3(RelationalSnapshotChangeEventSource.java:451)\n\tat io.debezium.pipeline.EventDispatcher.dispatchSchemaChangeEvent(EventDispatcher.java:388)\n\tat io.debezium.relational.RelationalSnapshotChangeEventSource.createSchemaChangeEventsForTables(RelationalSnapshotChangeEventSource.java:449)\n\tat io.debezium.relational.RelationalSnapshotChangeEventSource.doExecute(RelationalSnapshotChangeEventSource.java:168)\n\tat io.debezium.pipeline.source.AbstractSnapshotChangeEventSource.execute(AbstractSnapshotChangeEventSource.java:96)\n\tat io.debezium.pipeline.ChangeEventSourceCoordinator.doSnapshot(ChangeEventSourceCoordinator.java:286)\n\tat io.debezium.pipeline.ChangeEventSourceCoordinator.doSnapshot(ChangeEventSourceCoordinator.java:270)\n\tat io.debezium.pipeline.ChangeEventSourceCoordinator.executeChangeEventSources(ChangeEventSourceCoordinator.java:192)\n\tat io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:143)\n\tat java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)\n\tat java.base/java.util.concurrent.FutureTask.run(Unknown Source)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)\n\tat java.base/java.lang.Thread.run(Unknown Source)\n"} {"@timestamp":"2024-09-23T18:54:11.446Z","ecs.version":"1.2.0","log.level":"DEBUG","message":"Building schema for column HAS_PHOTOS of type 2 named NUMBER with constraints (1,Optional[0])","process.thread.name":"debezium-oracleconnector-ed50-scan1-change-event-source-coordinator","log.logger":"io.debezium.connector.oracle.OracleValueConverters","dbz.connectorContext":"snapshot","dbz.connectorName":"ed50-scan1","dbz.connectorType":"Oracle","dbz.databaseName":"MYTABLE","dbz.taskId":"0"} {"@timestamp":"2024-09-23T18:54:11.446Z","ecs.version":"1.2.0","log.level":"DEBUG","message":"- field 'HAS_PHOTOS' (STRING) from column HAS_PHOTOS NUMBER(1, 0) DEFAULT VALUE (0)","process.thread.name":"debezium-oracleconnector-ed50-scan1-change-event-source-coordinator","log.logger":"io.debezium.relational.TableSchemaBuilder","dbz.connectorContext":"snapshot","dbz.connectorName":"ed50-scan1","dbz.connectorType":"Oracle","dbz.databaseName":"MYTABLE","dbz.taskId":"0"}

{"@timestamp":"2024-09-23T18:54:11.446Z","ecs.version":"1.2.0","log.level":"DEBUG","message":"Mapped primary key for table 'MYTABLE.WM_LM.LPN' to schema: {\"name\" : \"ed50-scan1.WM_LM.LPN.Key\", \"type\" : \"STRUCT\", \"optional\" : \"false\", \"default\" : null, \"fields\" : [{\"name\" : \"LPN_ID\", \"index\" : \"0\", \"schema\" : {\"type\" : \"STRING\", \"optional\" : \"false\", \"default\" : null}}]}","process.thread.name":"debezium-oracleconnector-ed50-scan1-change-event-source-coordinator","log.logger":"io.debezium.relational.TableSchemaBuilder","dbz.connectorContext":"snapshot","dbz.connectorName":"ed50-scan1","dbz.connectorType":"Oracle","dbz.databaseName":"MYTABLE","dbz.taskId":"0"}

What Debezium connector do you use and what version?

It is 2.7.3, but here are helpful additional library information we use in our Java Springboot application.

implementation 'io.debezium:debezium-api:2.7.3.Final'
implementation'io.debezium:debezium-embedded:2.7.3.Final'
implementation 'org.apache.kafka:connect-runtime:3.6.1'
implementation 'io.debezium:debezium-connector-oracle:2.7.3.Final'
implementation 'com.oracle.database.jdbc:ojdbc8:21.6.0.0'
implementation 'io.debezium:debezium-storage-kafka:2.7.3.Final'

What is the connector configuration?

connector:
decimalHandlingMode: string
name: <<NAME>>
class: io.debezium.connector.oracle.OracleConnector
offsetTopic: <<OFFSETTOPICNAME>>
offsetTopicNumberOfPartitions: 1
offsetTopicReplicationFactor: 1
heartbeatIntervalInMsecs: 100000
heartbeatQuery: SELECT * FROM MYTABLE WHERE ROWNUM <= 1

What is the captured database version and mode of deployment?

Data base in running on a data center and is an Oracle database of version 19c.

DB Server on Dev Database runs on a 2 node RAC. The Archived log files (which Log Miner reads from) are shared to both nodes in RAC as a shared path.

What behavior do you expect?

Expression language error should not arrive. Not sure what else is the impact of this. Do messages get lost in that case ?

What behavior do you see?

Default values should be handled right and should not impact the flow

Do you see the same behaviour using the latest released Debezium version?

Didn't try yet. Is 2.7.3.Final OK ?

Do you have the connector logs, ideally from start till finish?

No, it was TRACE enabled so it is hard to capture logs as they dissapear soon.But useful parts are captured.

How to reproduce the issue using our tutorial deployment?

Yes, please see bug report above.

 

Also have a question as to how much impact does it make.