-
Bug
-
Resolution: Not a Bug
-
Major
-
None
-
2.0.0.Final
-
None
-
False
-
None
-
False
In order to make your issue reports as actionable as possible, please provide the following information, depending on the issue type.
Bug report
For bug reports, provide this information, please:
What Debezium connector do you use and what version?
we upgrade from 1.8 to 2.0
What is the connector configuration?
{ "connector.class": "io.debezium.connector.mysql.MySqlConnector", "snapshot.locking.mode": "none", "_comment": "Connector config for zcs-04_qas", "database.history.kafka.recovery.poll.interval.ms": "86400000", "drop.tombstones": "false", "database.history.skip.unparseable.ddl": "true", "drop.deletes": "false", "_comment": "MySQL settings", "database.hostname": "xxxxxx", "database.port": "xxxxx", "database.user": "xxxxxx", "database.password": "xxxxx", "database.connectionTimeZone": "Europe/Brussels", "database.server.id": "2329", "database.server.name": "zcs-04_qas", "database.include.list": "zcs", "schema.history.internal.kafka.topic": "zcs-04_qas.history", "schema.history.internal.kafka.bootstrap.servers": "xxxxxx.hrx.erp:9092", "database.history.kafka.bootstrap.servers": "xxxxxxx.hrx.erp:9092", "ddl.parser.mode": "antlr", "include.schema.changes": "true", "decimal.handling.mode": "string", "connect.keep.alive": "true", "snapshot.new.tables": "parallel", "topic.prefix": "zcs-04_qas", "snapshot.mode": "when_needed", "table.include.list": "zcs.conf_pay_elements,zcs.conf_wage_type_category_pay_elements,zcs.employments,zcs.enterprise_store_communications,zcs.enterprise_store_cost_assignments,zcs.enterprise_store_deployments,zcs.enterprise_store_personal_data,zcs.enterprise_store_profiles,zcs.fcs_countries,zcs.gccs,zcs.lccs,zcs.pay_calendars,zcs.pay_groups,zcs.pay_periods,zcs.wage_type_categories,zcs.wage_type_report_sources,zcs.users,zcs.lccs_users,zcs.glrep_rows_staging2,zcs.glrep_file2", "_comment": "Kakfa connector settings", "poll.interval.ms": "5000", "_comment": "Kafka converter settings", "value.converter.schema.registry.url": "http://xxxxxxxx.hrx.erp:8081", "value.converter": "io.confluent.connect.avro.AvroConverter", "key.converter": "io.confluent.connect.avro.AvroConverter", "key.converter.schema.registry.url": "http://xxxxxxxxx.hrx.erp:8081", "internal.key.converter": "org.apache.kafka.connect.json.JsonConverter", "internal.value.converter": "org.apache.kafka.connect.json.JsonConverter" }What is the captured database version and mode of depoyment?
(E.g. on-premises, with a specific cloud provider, etc.)
MySQL 7.x
What behaviour do you expect?
The table and schema no listed/set in the connector configuration are IGNORED.
When a database as a lot of schemas it generates a lot of unnecessary data collect, logs.
Based on the configuration of the connector,
- only the schema/db specified should be taken into account
- only the tables list in the table list parameter should be taken into account (if not set then all)
What behaviour do you see?
- We see in the Kafka Connectors log an error message when we start a connector for schema_X with table list Table_1, Table2
- about schema_Y ???? --> why never set in a connector ?
- about table_Y ??? --> why never set in a connector ?
Do you see the same behaviour using the latest relesead Debezium version?
(Ideally, also verify with latest Alpha/Beta/CR version)
seems in the past that it was a WARNING message now an ERROR (not sure)
Do you have the connector logs, ideally from start till finish?
(You might be asked later to provide DEBUG/TRACE level log)
"tasks": [
],
"type": "source"
}
You can find some historical info in case: DBZ-5840
How to reproduce the issue using our tutorial deployment?
<Your answer>
Feature request or enhancement
For feature requests or enhancements, provide this information, please:
Which use case/requirement will be addressed by the proposed feature?
<Your answer>
Implementation ideas (optional)
None listed schema/tables should be IGNORED or be a WARNING but should never stop/block a connector for a specific schema and specific tables (=SINGLETON, ignore other)
- impacts account
-
DBZ-5342 Internal schema may get corrupted due to duplicates in database history topic
- Pull Request Sent