Type: Bug
Resolution: Done
Priority: Major
Fix Version/s: 2.7.3.Final, 3.0.0.CR2
Affects Version/s: 2.7.1.Final
Component/s: mariadb-connector
Labels:
- ddl
- debezium
- parsing

Blocked:
False
Blocked Reason:
None
Ready:
False
Git Pull Request:
https://github.com/debezium/debezium/pull/5874

SFDC Cases Counter:
SFDC Cases Open:
SFDC Cases Links:

In order to make your issue reports as actionable as possible, please provide the following information, depending on the issue type.

Bug report

For bug reports, provide this information, please:

What Debezium connector do you use and what version?

MariaDB Source Connector
Debezium 2.7.1.Final With the Embedded Engine.

What is the connector configuration?

props.setProperty("name", applicationName);
props.setProperty("bootstrap.servers", bootstrapServers);
props.setProperty("offset.storage", "org.apache.kafka.connect.storage.KafkaOffsetBackingStore");
props.setProperty("offset.storage.topic", systemTopicPrefix + ".offset_storage");
props.setProperty("offset.storage.partitions", "1");
props.setProperty("offset.storage.replication.factor", "2");
props.setProperty("offset.flush.interval.ms", "20000"); // 20 seconds
props.setProperty("offset.flush.timeout.ms", "10000"); // 10 seconds
props.setProperty("topic.prefix", topicPrefix);
props.setProperty("database.server.id", generateServerId(applicationName));
props.setProperty("schema.history.internal.kafka.bootstrap.servers", bootstrapServers);
props.setProperty("schema.history.internal.kafka.topic", systemTopicPrefix + ".db_history");
props.setProperty("snapshot.max.threads", String.valueOf(snapshotMaxThreads));
props.setProperty("signal.enabled.channels", "kafka");
props.setProperty("signal.kafka.topic", systemTopicPrefix + ".debezium_signals");
props.setProperty("signal.kafka.bootstrap.servers", bootstrapServers);
props.setProperty("connector.class", "io.debezium.connector.mariadb.MariaDbConnector");
props.setProperty("connector.adapter", "mariadb");
props.setProperty("database.hostname", debeziumSourceDatabaseConfig.getHostname());
props.setProperty("database.port", debeziumSourceDatabaseConfig.getPort());
props.setProperty("database.user", debeziumSourceDatabaseConfig.getUsername());
props.setProperty("database.password", debeziumSourceDatabaseConfig.getPassword());
props.setProperty("database.server.name", applicationName.replace('.', '_'));
props.setProperty("database.history.skip.unparseable.ddl", String.valueOf(true));
props.setProperty("database.history.producer.max.request.size", String.valueOf(MAX_HISTORY_MESSAGE_SIZE));
props.setProperty("database.history.producer.buffer.memory", String.valueOf(MAX_HISTORY_MESSAGE_SIZE));
props.setProperty("database.history.consumer.max.partition.fetch.bytes", String.valueOf(MAX_HISTORY_MESSAGE_SIZE));
props.setProperty("database.history.store.only.captured.tables.ddl", String.valueOf(false));
props.setProperty("database.include.list", debeziumSourceDatabaseConfig.getSchemaWhitelist());
props.setProperty("database.timezone", "disable");
props.setProperty("database.protocol", "jdbc:mariadb");
props.setProperty("database.jdbc.driver", "org.mariadb.jdbc.Driver");
props.setProperty("database.ssl.mode", "disabled");
props.setProperty("snapshot.mode", debeziumSourceDatabaseConfig.getSnapshotMode());
props.setProperty("include.schema.changes", "true");
props.setProperty("connect.keep.alive", "true");
props.setProperty("connections.max.idle.ms", "540000");

What is the captured database version and mode of deployment?

On premise MariaDB

What behavior do you expect?

A DDL was executed on the source database. which caused the Debezium to crash due to the following error.

// io.debezium.DebeziumException: Error processing binlog event
	at io.debezium.connector.binlog.BinlogStreamingChangeEventSource.handleEvent(BinlogStreamingChangeEventSource.java:579)
	at io.debezium.connector.binlog.BinlogStreamingChangeEventSource.lambda$execute$17(BinlogStreamingChangeEventSource.java:209)
	at com.github.shyiko.mysql.binlog.BinaryLogClient.notifyEventListeners(BinaryLogClient.java:1268)
	at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:1094)
	at com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:653)
	at com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:954)
	at java.base/java.lang.Thread.run(Thread.java:840)
Caused by: io.debezium.text.ParsingException: DDL statement couldn't be parsed. Please open a Jira issue with the statement 'ALTER TABLE my_table
    ADD CONSTRAINT my_table_to_dealers_fk
        FOREIGN KEY IF NOT EXISTS (dealer_id) REFERENCES n_dealers (id),
    ADD CONSTRAINT my_table_trigger_ranking_class_version_dealer_uk UNIQUE KEY IF NOT EXISTS (trigger_id, ranking_criteria, classification_id, version, dealer_id),
    ADD CONSTRAINT trigger_classification_to_trigger_name_fk FOREIGN KEY IF NOT EXISTS (trigger_id) REFERENCES n_vulnerability_triggers (id)'
no viable alternative at input 'ALTER TABLE my_table\n    ADD CONSTRAINT my_table_to_dealers_fk\n        FOREIGN KEY IF NOT EXISTS (dealer_id) REFERENCES n_dealers (id),\n    ADD CONSTRAINT my_table_trigger_ranking_class_version_dealer_uk UNIQUE KEY IF NOT
'
	at io.debezium.antlr.ParsingErrorListener.syntaxError(ParsingErrorListener.java:43)
	at org.antlr.v4.runtime.ProxyErrorListener.syntaxError(ProxyErrorListener.java:41)
	at org.antlr.v4.runtime.Parser.notifyErrorListeners(Parser.java:543)
	at org.antlr.v4.runtime.DefaultErrorStrategy.reportNoViableAlternative(DefaultErrorStrategy.java:310)
	at org.antlr.v4.runtime.DefaultErrorStrategy.reportError(DefaultErrorStrategy.java:136)
	at io.debezium.ddl.parser.mariadb.generated.MariaDBParser.sqlStatements(MariaDBParser.java:1266)
	at io.debezium.ddl.parser.mariadb.generated.MariaDBParser.root(MariaDBParser.java:986)
	at io.debezium.connector.mariadb.antlr.MariaDbAntlrDdlParser.parseTree(MariaDbAntlrDdlParser.java:80)
	at io.debezium.connector.mariadb.antlr.MariaDbAntlrDdlParser.parseTree(MariaDbAntlrDdlParser.java:47)
	at io.debezium.antlr.AntlrDdlParser.parse(AntlrDdlParser.java:76)
	at io.debezium.connector.binlog.BinlogDatabaseSchema.parseDdl(BinlogDatabaseSchema.java:311)
	at io.debezium.connector.binlog.BinlogDatabaseSchema.parseStreamingDdl(BinlogDatabaseSchema.java:258)
	at io.debezium.connector.binlog.BinlogStreamingChangeEventSource.handleQueryEvent(BinlogStreamingChangeEventSource.java:726)
	at io.debezium.connector.binlog.BinlogStreamingChangeEventSource.lambda$execute$5(BinlogStreamingChangeEventSource.java:179)
	at io.debezium.connector.binlog.BinlogStreamingChangeEventSource.handleEvent(BinlogStreamingChangeEventSource.java:559)
	... 6 common frames omitted
Caused by: org.antlr.v4.runtime.NoViableAltException: null
	at org.antlr.v4.runtime.atn.ParserATNSimulator.noViableAlt(ParserATNSimulator.java:2028)
	at org.antlr.v4.runtime.atn.ParserATNSimulator.execATN(ParserATNSimulator.java:467)
	at org.antlr.v4.runtime.atn.ParserATNSimulator.adaptivePredict(ParserATNSimulator.java:393)
	at io.debezium.ddl.parser.mariadb.generated.MariaDBParser.sqlStatements(MariaDBParser.java:1062)
	... 15 common frames omitted

This is the DDL that was executed

ALTER TABLE my_table
    MODIFY COLUMN IF EXISTS dealer_id SMALLINT UNSIGNED NOT NULL;
ALTER TABLE my_table
    ADD CONSTRAINT my_table_to_dealers_fk
        FOREIGN KEY IF NOT EXISTS (dealer_id) REFERENCES n_dealers (id),
    ADD CONSTRAINT my_table_trigger_ranking_class_version_dealer_uk UNIQUE KEY IF NOT EXISTS (trigger_id, ranking_criteria, classification_id, version, dealer_id),
    ADD CONSTRAINT trigger_classification_to_trigger_name_fk FOREIGN KEY IF NOT EXISTS (trigger_id) REFERENCES n_vulnerability_triggers (id);

This is part of the binary log on the position that failed

// # at 2467229
#240913 16:58:09 server id 89655002  end_log_pos 2467271 CRC32 0x3f8a826d   GTID 0-89655002-80866054 ddl
/*!100001 SET @@session.gtid_seq_no=80866054*//*!*/;
# at 2467271
#240913 16:58:09 server id 89655002  end_log_pos 2467863 CRC32 0xe50b506f   Query thread_id=2362197 exec_time=0 error_code=0  xid=27261143
SET TIMESTAMP=1726235889/*!*/;
ALTER TABLE my_table
    ADD CONSTRAINT my_table_to_dealers_fk
        FOREIGN KEY IF NOT EXISTS (dealer_id) REFERENCES n_dealers (id),
    ADD CONSTRAINT my_table_trigger_ranking_class_version_dealer_uk UNIQUE KEY IF NOT EXISTS (trigger_id, ranking_criteria, classification_id, version, dealer_id),
    ADD CONSTRAINT trigger_classification_to_trigger_name_fk FOREIGN KEY IF NOT EXISTS (trigger_id) REFERENCES n_vulnerability_triggers (id)
/*!*/;
# at 2467863
#240913 16:58:09 server id 89655002  end_log_pos 2467905 CRC32 0x89328edd   GTID 0-89655002-80866055 trans
/*!100001 SET @@session.gtid_seq_no=80866055*//*!*/;
START TRANSACTION
/*!*/;
# at 2467905

What behavior do you see?

My connector crashed and i had to initiate an initial snapshot.

Do you see the same behaviour using the latest released Debezium version?

yes

Do you have the connector logs, ideally from start till finish?

pasted them above

links to

RHEA-2024:139598 Red Hat build of Debezium 2.5.4 release

Details

Description

Bug report

What Debezium connector do you use and what version?

What is the connector configuration?

What is the captured database version and mode of deployment?

What behavior do you expect?

This is the DDL that was executed

Do you see the same behaviour using the latest released Debezium version?

Do you have the connector logs, ideally from start till finish?

Attachments

Issue Links

Easy Agile Planning Poker

Activity

People

Dates