-
Bug
-
Resolution: Done
-
Major
-
2.2.1.Final, 2.3.0.Final
-
False
-
None
-
False
-
Moderate
Bug report
What Debezium connector do you use and what version?
2.2.1Final (same code exist also at latest)
What is the connector configuration?
relevant configuration:
database.include.list: <database_name>
connector.class: io.debezium.connector.mysql.MySqlConnector
schema.history.internal.store.only.captured.databases.ddl: true
schema.history.internal.store.only.captured.tables.ddl: false
snapshot.include.collection.list: <database_name>.*
snapshot.mode: schema_only\schema_only_recovery
table.include.list: <database_name>.table_test1,<database_name>.table_test2,<database_name>.cdc_dummy
What is the captured database version and mode of depoyment?
MySql v5.7
What behaviour do you expect?
I accept the schema.history.internal.store.only.captured.databases.ddl To be respected while snapshot is filtering out tables and collect only included databases tables structure into the <connector_name>.history topic.
What behaviour do you see?
debezium is collecting all databases tables into history topic
Do you see the same behaviour using the latest relesead Debezium version?
relevant Code as not been changed to support it (RelationalTableFilters.java,RelationalSnapshotChangeEventSource.java)
was recreated also in 2.3.0Final version.
How to reproduce the issue using our tutorial deployment?
running the above configuration on MySQL Instance which contains more than one application database. but include only one of the databases in the database.include.list
- Reset and run a connector to take a new snapshot.
- Add a column to one of the tables in the second (untracked) database.
- make som DML changes to the tracked database make sure new binlog offset is recorded in the connector offset topic.
- restart the connector
- Alter column type - parseException should occur.
Feature request or enhancement
schema.history.internal.store.only.captured.databases.ddl flag to control the include of only tracked databases(include.database.list) ddl in history topic.
Which use case/requirement will be addressed by the proposed feature?
There are several methods this situation can be prevented but consider the documentation regarding schema.history.internal.store.only.captured.databases.ddl flag it should be handle on that level.
Without this behavior for stability you must include schema.history.internal.store.only.captured.tables.ddl and perform a schema_only_recovery procedure whenever new table is added to the include list.
Implementation ideas (optional)
the quickest and cleanest way is to fix it in RelationalTableFilters.java
the eligibleForSchemaDataCollectionFilter should return a schemaSnapshotFilter which takes into consideration the schema.history.internal.store.only.captured.databases.ddl flag value
- is related to
-
DBZ-6070 DDL events not stored in schema history topic for excluded tables
- Closed
- links to
-
RHEA-2024:129636 Red Hat build of Debezium 2.5.4 release