Details
-
Bug
-
Resolution: Done
-
Major
-
2.3.4.Final
-
None
-
False
-
None
-
False
-
Important
Description
Bug report
Presently, we are running Debezium to source data from MariaDB into Kafka. Our setup was working fine for some weeks up until we got the following:
Caused by: io.debezium.DebeziumException: Error processing binlog event ... 7 more Caused by: io.debezium.text.ParsingException: DDL statement couldn't be parsed. Please open a Jira issue with the statement 'alter table user_details add index if not exists `country_id_index` (country_id), algorithm=NOCOPY' no viable alternative at input 'alter table user_details add index if not exists `country_id_index` (country_id), algorithm=NOCOPY'
To address this, we enabled the database.history.skip.unparseable.ddl setting to skip the DDL event from the MariaDB binlog. However, even after applying this parameter to true, the issue persists.
What Debezium connector do you use and what version?
MySQL Connector (2.3.4.Final)
What is the connector configuration?
{ "connector.class":"io.debezium.connector.mysql.MySqlConnector", "database.user":"${secrets:mariadb-secret:MARIADB_KAFKA_USERNAME}", "transforms.Reroute.type":"io.debezium.transforms.ByLogicalTableRouter", "database.server.id":"202308031027", "tasks.max":"1", "transforms":"Reroute", "schema.history.internal.kafka.bootstrap.servers":"rootz-strimzi-kafka-kafka-bootstrap.cloudflow.svc:9092", "transforms.Reroute.topic.regex":"(.*)_(.*)", "database.port":"3306", "include.schema.changes":"false", "value.converter.schema.registry.url":"https://kafka-sr.stg.rootz.com/", "topic.prefix":"mariadb", "transforms.Reroute.topic.replacement":"$1-$2", "schema.history.internal.kafka.topic":"mariadb.casinoapi.payment-transaction.history", "database.hostname":"mariadb.stg.rootz.com", "database.password":"${secrets:mariadb-secret:MARIADB_KAFKA_PASSWORD}", "schema.name.adjustment.mode":"avro", "name":"mariadb.casinoapi.payment-transaction.source-v1", "database.history.skip.unparseable.ddl":"true", "table.include.list":"casinoapi.payment_transaction", "value.converter":"io.confluent.connect.avro.AvroConverter", "key.converter":"io.confluent.connect.avro.AvroConverter", "key.converter.schema.registry.url":"https://kafka-sr.stg.rootz.com/", "database.include.list":"casinoapi" }
What is the captured database version and mode of depoyment?
Both MariaDB and the Kafka / Kafka Connect are deployed on premise.
What behaviour do you expect?
The database.history.skip.unparseable.ddl should skip the erroneous DDL statement and the SOURCE connector should resume it's data load.
What behaviour do you see?
The database.history.skip.unparseable.ddl is not skipping the erroneous DDL statement.
Do you see the same behaviour using the latest released Debezium version?
On 2.3.4.FINAL, we see the same behaviour.
Do you have the connector logs, ideally from start till finish?
Attachments
Issue Links
- links to
-
RHEA-2024:129636
Red Hat build of Debezium 2.5.4 release