Debezium Builder added a comment - 2021/03/12 5:21 AM Released

Chris Cranford added a comment - 2021/02/25 4:49 PM - edited By default, Oracle identifiers are not case-sensitive in queries except when they're explicitly used in a string-predicate, i.e. "WHERE TABLE_NAME = 'table'", or if the identifier was created with double-quotes. So with that in mind, we can take the identifier name as-is from the JDBC metadata results and store it in that case. When we then use the names in a string-predicate, they'll "just work". When we use the names in a SQL clause, i.e. "FROM table", we should double-quote the name to be compatible with both lower/mixed-case and the default upper-case scenarios. Additionally, when using database.tablename.case.insensitive=true , Xstreams still fails because the LcrEventHandler doesn't compute the TableId correctly and only enforces the catalog and table name to lower case but ignores the schema name: return new TableId(sourceDatabaseName.toLowerCase(), lcr.getObjectOwner(), lcr.getObjectName().toLowerCase()); I think we should drop database.tablename.case.insensitive entirely. It doesn't work except in the specific corner case where the schema is created with double-quotes in all lower-case, which may have been the case when tested by the contributor locally. Removing the option should be completely backward compatible.

Nikolay Nedkov (Inactive) added a comment - 2021/02/25 12:06 PM - edited Hello, I got NullPointerException without showing which exact field is missing value for. My config with censored sensitive data: ``` { "connector.class": "io.debezium.connector.oracle.OracleConnector", "topic.prefix": "", "tasks.max": "1", "database.server.name": "XX", "table.include.list": "PAN.BE__CLE_THIRD_PARTIES", "database.include.list": "XX", "numeric.mapping": "best_fit", "database.url": "jdbc:oracle:thin:@xxxx:1521:xxx", "database.user": "XX", "database.schema": "XX", "database.password": "XXXX", "database.dbname": "XXX", "database.tablename.case.insensitive": true, "database.out.server.name": "XXXout", "database.history.kafka.bootstrap.servers": "kafka:9092", "database.history.kafka.topic": "schema-changes.BE__CLE_THIRD_PARTIES", "database.connection.adapter": "logminer", "transforms": "ValueToKey,unwrap", "transforms.ValueToKey.type": "org.apache.kafka.connect.transforms.ValueToKey", "transforms.ValueToKey.fields": "THP_SERIAL", "transforms.unwrap.type": "io.debezium.transforms.ExtractNewRecordState", "transforms.unwrap.drop.tombstones": "false", "transforms.unwrap.delete.handling.mode": "rewrite" } ``` Here is the log: ``` eb 25, 2021 @ 18:39:38.832 2021-02-25 16:39:38,831 ERROR || WorkerSourceTask {id=oracle-source-BE__CLE_THIRD_PARTIES-0} Task threw an uncaught and unrecoverable exception [org.apache.kafka.connect.runtime.WorkerTask] be-debezium Feb 25, 2021 @ 18:39:38.832 org.apache.kafka.connect.errors.ConnectException: An exception occurred in the change event producer. This connector will be stopped. be-debezium Feb 25, 2021 @ 18:39:38.832 at io.debezium.pipeline.ErrorHandler.setProducerThrowable(ErrorHandler.java:42) be-debezium Feb 25, 2021 @ 18:39:38.832 at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:121) be-debezium Feb 25, 2021 @ 18:39:38.832 at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) be-debezium Feb 25, 2021 @ 18:39:38.832 at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) be-debezium Feb 25, 2021 @ 18:39:38.832 at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) be-debezium Feb 25, 2021 @ 18:39:38.832 at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) be-debezium Feb 25, 2021 @ 18:39:38.832 at java.base/java.lang.Thread.run(Thread.java:834) be-debezium Feb 25, 2021 @ 18:39:38.832 Caused by: io.debezium.DebeziumException: java.lang.NullPointerException be-debezium Feb 25, 2021 @ 18:39:38.832 at io.debezium.pipeline.source.AbstractSnapshotChangeEventSource.execute(AbstractSnapshotChangeEventSource.java:82) be-debezium Feb 25, 2021 @ 18:39:38.832 at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:105) be-debezium Feb 25, 2021 @ 18:39:38.832 ... 5 more be-debezium Feb 25, 2021 @ 18:39:38.832 Caused by: java.lang.NullPointerException be-debezium Feb 25, 2021 @ 18:39:38.832 at io.debezium.relational.RelationalSnapshotChangeEventSource.createSchemaChangeEventsForTables(RelationalSnapshotChangeEventSource.java:271) be-debezium Feb 25, 2021 @ 18:39:38.832 at io.debezium.relational.RelationalSnapshotChangeEventSource.doExecute(RelationalSnapshotChangeEventSource.java:127) be-debezium Feb 25, 2021 @ 18:39:38.832 at io.debezium.pipeline.source.AbstractSnapshotChangeEventSource.execute(AbstractSnapshotChangeEventSource.java:71) be-debezium Feb 25, 2021 @ 18:39:38.832 ... 6 more be-debezium Feb 25, 2021 @ 18:39:38.831 2021-02-25 16:39:38,831 INFO || WorkerSourceTask{id=oracle-source-BE__CLE_THIRD_PARTIES-0} Committing offsets [org.apache.kafka.connect.runtime.WorkerSourceTask] be-debezium Feb 25, 2021 @ 18:39:38.720 2021-02-25 16:39:38,719 ERROR || Producer failure [io.debezium.pipeline.ErrorHandler] ```

Gunnar Morling added a comment - 2021/02/25 5:27 AM I'm definitely pro automating this, and removing the option eventually. I'd suggest this: derive the setting automatically if the option is explicitly given, use its value deprecate the option for removal

Chris Cranford added a comment - 2021/02/23 10:38 PM gunnar.morling / jpechane I'm a bit torn on this because on one hand I think having this documented is in part satisfactory; however, the fact the connector doesn't recognize any miss configuration here and doesn't even give the user a hint there's something wrong until a query later in the runtime fails obscurely is not right either. On one hand we could validate the config option against the new OracleDatabaseVersion implementation. If we detect the database connection is to an Oracle 12+ instance, we could choose to throw a DebeziumException so at least users get a heads up that the configuration is invalid. We could also simply log a warning and omit that configuration option for Oracle 12+ environments too. Another idea would be we deprecate/remove this option entirely and we rely on the OracleDatabaseVersion implementation to provide the resolution of whether the schema is to be case insensitive or not based on the major database version. What do either of you think? A strong preference to one or the other? Any other ideas come to mind?