Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-9444

MariaDB connector get schema mismatch for system-versioned table

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Duplicate
    • Icon: Major Major
    • None
    • 3.2.1.Final
    • mariadb-connector
    • None
    • False
    • Hide

      None

      Show
      None
    • False
    • Important

      In order to make your issue reports as actionable as possible, please provide the following information, depending on the issue type.

      Bug report

      For bug reports, provide this information, please:

      What Debezium connector do you use and what version?

      MariaDB, version 3.2.1.Final

      What is the connector configuration?

       

      snapshot.mode: no_data
      schema.history.internal.skip.unparseable.ddl: true
      schema.history.internal.store.only.captured.tables.ddl: false
      schema.history.internal.store.only.captured.databases.ddl: true
      include.schema.changes: false
      include.schema.comments: false 

       

       

      and here's the DDL from the source table

      CREATE TABLE `questionnaire_answers` (
        `id` int(11) NOT NULL AUTO_INCREMENT,
        `questionnaire_id` char(36) NOT NULL,
        `answer_id` int(11) NOT NULL,
        `free_text_answer` mediumtext DEFAULT NULL,
        `date_created` timestamp NOT NULL DEFAULT current_timestamp(),
        PRIMARY KEY (`id`),
        UNIQUE KEY `uq_questionnaire_answers_questionnaire_id` (`questionnaire_id`,`answer_id`),
        KEY `answer_id` (`answer_id`),
        CONSTRAINT `questionnaire_answers_ibfk_1` FOREIGN KEY (`questionnaire_id`) REFERENCES `questionnaires` (`id`),
        CONSTRAINT `questionnaire_answers_ibfk_2` FOREIGN KEY (`answer_id`) REFERENCES `answers` (`id`)
      ) ENGINE=InnoDB AUTO_INCREMENT=228529 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_general_ci WITH SYSTEM VERSIONING; 

       

       

      What is the captured database version and mode of deployment?

      MariaDB

      What behavior do you expect?

      It should be able to handle system versioned table really well

      What behavior do you see?

      It always gets this error after a few minutes, while the table schema doesn't change

      Caused by: io.debezium.DebeziumException: Error processing row in questionnaire_answers, internal schema size 5, but row size 7 , restart connector with schema recovery mode.
      	at io.debezium.connector.binlog.BinlogStreamingChangeEventSource.validateChangeEventWithTable(BinlogStreamingChangeEventSource.java:1069)
      	at io.debezium.connector.binlog.BinlogStreamingChangeEventSource.lambda$handleInsert$28(BinlogStreamingChangeEventSource.java:841)
      	at io.debezium.connector.binlog.BinlogStreamingChangeEventSource.handleChange(BinlogStreamingChangeEventSource.java:1105)
      	at io.debezium.connector.binlog.BinlogStreamingChangeEventSource.handleInsert(BinlogStreamingChangeEventSource.java:836)
      	at io.debezium.connector.binlog.BinlogStreamingChangeEventSource.lambda$execute$7(BinlogStreamingChangeEventSource.java:182)
      	at io.debezium.connector.binlog.BinlogStreamingChangeEventSource.handleEvent(BinlogStreamingChangeEventSource.java:570)
      	... 6 more
      2025-09-01 06:39:49,293 INFO [mysql-source-kyc|task-0] Error processing binlog event, and propagating to Kafka Connect so it stops this connector. Future binlog events read before connector is shutdown will be ignored. (io.debezium.connector.binlog.BinlogStreamingChangeEventSource) [HOSTNAME:3306]
      2025-09-01 06:39:49,721 ERROR [mysql-source-kyc|task-0] WorkerSourceTask{id=mysql-source-kyc-0} Task threw an uncaught and unrecoverable exception. Task is being killed and will not recover until manually restarted (org.apache.kafka.connect.runtime.WorkerTask) [task-thread-mysql-source-kyc-0]
      org.apache.kafka.connect.errors.ConnectException: An exception occurred in the change event producer. This connector will be stopped.
      	at io.debezium.pipeline.ErrorHandler.setProducerThrowable(ErrorHandler.java:67)
      	at io.debezium.connector.binlog.BinlogStreamingChangeEventSource.handleEvent(BinlogStreamingChangeEventSource.java:590)
      	at io.debezium.connector.binlog.BinlogStreamingChangeEventSource.lambda$execute$17(BinlogStreamingChangeEventSource.java:208)
      	at com.github.shyiko.mysql.binlog.BinaryLogClient.notifyEventListeners(BinaryLogClient.java:1281)
      	at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:1103)
      	at com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:657)
      	at com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:959)
      	at java.base/java.lang.Thread.run(Thread.java:840)
      Caused by: io.debezium.DebeziumException: Error processing binlog event
      	... 7 more
      Caused by: io.debezium.DebeziumException: Error processing row in questionnaire_answers, internal schema size 5, but row size 7 , restart connector with schema recovery mode.
      	at io.debezium.connector.binlog.BinlogStreamingChangeEventSource.validateChangeEventWithTable(BinlogStreamingChangeEventSource.java:1069)
      	at io.debezium.connector.binlog.BinlogStreamingChangeEventSource.lambda$handleInsert$28(BinlogStreamingChangeEventSource.java:841)
      	at io.debezium.connector.binlog.BinlogStreamingChangeEventSource.handleChange(BinlogStreamingChangeEventSource.java:1105)
      	at io.debezium.connector.binlog.BinlogStreamingChangeEventSource.handleInsert(BinlogStreamingChangeEventSource.java:836)
      	at io.debezium.connector.binlog.BinlogStreamingChangeEventSource.lambda$execute$7(BinlogStreamingChangeEventSource.java:182)
      	at io.debezium.connector.binlog.BinlogStreamingChangeEventSource.handleEvent(BinlogStreamingChangeEventSource.java:570)
      	... 6 more 

      Do you see the same behaviour using the latest released Debezium version?

      Yes, I do

      Do you have the connector logs, ideally from start till finish?

      I've put on previous question

      How to reproduce the issue using our tutorial deployment?

      Try to create connector to MariaDB system versioned table

              Unassigned Unassigned
              demsyiman Demsy Iman
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Created:
                Updated:
                Resolved: