-
Bug
-
Resolution: Done
-
Major
-
2.2.0.Final
-
None
-
False
-
None
-
False
What Debezium connector do you use and what version?
debezium-connector-oracle-2.2.0.Final
What is the connector configuration?
{ "connector.class": "io.debezium.connector.oracle.OracleConnector", "snapshot.locking.mode": "none", "tasks.max": "1", "schema.include.list": "TEST", "log.mining.strategy": "online_catalog", "schema.history.internal.store.only.captured.tables.ddl": "true", "topic.prefix": "TEST", "decimal.handling.mode": "double", "schema.history.internal.kafka.topic": "DBHISTORY. TEST01", "signal.data.collection": "ORCL. TEST. DEBEZIUM_SIGNAL", "log.mining.archive.log.only.mode": "false", "log.mining.archive.log.hours": "0", "log.mining.transaction.retention.hours": "12", "database.user": "user", "database.dbname": "ORCL", "time.precision.mode": "connect", "schema.history.internal.kafka.bootstrap.servers": "broker:9092", "event.processing.failure.handling.mode": "warn", "schema.history.internal.skip.unparseable.ddl": "true", "database.port": "1521", "database.serverTimezone": "UTC", " database.hostname": "172.0.1.1", "database.password": "password", "table.include.list": "TEST. TBTEST,TEST. DEBEZIUM_SIGNAL", "snapshot.mode": "initial" }What behavior do you expect?
In the fifth step of the snapshot, if you have a specified table, you do not need to get all the table metadata to save snapshot time.
What behavior do you see?
I have over 30000,100 tables in my database, and I only need to capture more than <> tables. During the snapshot, I found out that it took more than an hour. It's been too long.
At the same time, I examined the code in version 2.2 and found it in the OracleSnapshotChangeEventSource class, the readTableStructure method, and jdbcConnection. readSchema, passed the TableFilter parameter is empty. In the jdbcConnection class, readSchema method, if (tableFilter==null | | tableFilter. isInclusion (tableId)) {" was not filtered, resulting in a long time.
Do you have connector logs, preferably from start to finish?
(You may be asked to provide debug/trace level logs later)
[2023-05-05 17:57:18,472] INFO Snapshot step 5 - Reading structure of captured tables (io.debezium.relational.RelationalSnapshotChangeEventSource)
.....
[2023-05-05 19:00:39,289] INFO Snapshot step 5.a - Creating connection pool (io.debezium.relational.RelationalSnapshotChangeEventSource)
- links to
-
RHEA-2023:120698 Red Hat build of Debezium 2.3.4 release