Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-1645

Connector does not update its state correctly when processing compound ALTER statement

    Details

    • Steps to Reproduce:
      Hide

      First create table using:

      CREATE TABLE `table_a` (
      `id` INT(11) UNSIGNED NOT NULL AUTO_INCREMENT,
      `created` TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
      `last_update` TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
      `code` VARCHAR(20) NOT NULL default '',
      `user_id` INT(11) NOT NULL default '0',
      `value` DECIMAL(14,4) NOT NULL default '0.0000',
      `currency` VARCHAR(3) NOT NULL default '',
      `country` VARCHAR(2) NOT NULL default '',
      `city_id` INT(11) NULL default NULL,\n  PRIMARY KEY (`id`)\n);
      

      Then apply following DDL:

      ALTER TABLE `table_a`
      DROP COLUMN `country`,
      DROP COLUMN `currency`,
      DROP COLUMN `value`,
      DROP COLUMN `code`,
      CHANGE COLUMN `city_id` `output` LONGTEXT NOT NULL,
      RENAME TO `table_b`;
      

      Everything will be processed correctly from MySql server and Debezium connector side.

      But when we try to insert data:

      insert into table_b values(1, now(), now(), 1, 'test');
      

      connector will fail with following error:

      Invalid number of columns, expected '9' arrived '5' (io.debezium.connector.mysql.RecordMakers:349)
      The binlog event does not contain expected number of columns; the internal schema representation is probably out of sync with the real database schema, or the binlog contains events recorded with binlog_row_image other than FULL or the table in question is an NDB table
      

      If we break last ALTER statement into two

      ALTER TABLE `table_a`
      DROP COLUMN `country`,
      DROP COLUMN `currency`,
      DROP COLUMN `value`,
      DROP COLUMN `code`,
      CHANGE COLUMN `city_id` `output` LONGTEXT NOT NULL;
      ALTER TABLE `table_a` RENAME TO `table_b`;
      

      and insert data after that - everything is fine.

      Show
      First create table using: CREATE TABLE `table_a` ( `id` INT (11) UNSIGNED NOT NULL AUTO_INCREMENT , `created` TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP , `last_update` TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP , `code` VARCHAR (20) NOT NULL default '', `user_id` INT (11) NOT NULL default '0' , ` value ` DECIMAL (14,4) NOT NULL default '0.0000' , `currency` VARCHAR (3) NOT NULL default '', `country` VARCHAR (2) NOT NULL default '', `city_id` INT (11) NULL default NULL ,\n PRIMARY KEY (`id`)\n); Then apply following DDL: ALTER TABLE `table_a` DROP COLUMN `country`, DROP COLUMN `currency`, DROP COLUMN ` value `, DROP COLUMN `code`, CHANGE COLUMN `city_id` ` output ` LONGTEXT NOT NULL , RENAME TO `table_b`; Everything will be processed correctly from MySql server and Debezium connector side. But when we try to insert data: insert into table_b values (1, now(), now(), 1, 'test' ); connector will fail with following error: Invalid number of columns, expected '9' arrived '5' (io.debezium.connector.mysql.RecordMakers:349) The binlog event does not contain expected number of columns; the internal schema representation is probably out of sync with the real database schema, or the binlog contains events recorded with binlog_row_image other than FULL or the table in question is an NDB table If we break last ALTER statement into two ALTER TABLE `table_a` DROP COLUMN `country`, DROP COLUMN `currency`, DROP COLUMN ` value `, DROP COLUMN `code`, CHANGE COLUMN `city_id` ` output ` LONGTEXT NOT NULL ; ALTER TABLE `table_a` RENAME TO `table_b`; and insert data after that - everything is fine.

      Description

      MySql connector fails to correctly update its internal tables' representation when processing following ALTER statement:

      ALTER TABLE `table_a`
      DROP COLUMN `country`,
      DROP COLUMN `currency`,
      DROP COLUMN `value`,
      DROP COLUMN `code`,
      CHANGE COLUMN `city_id` `output` LONGTEXT NOT NULL,
      RENAME TO `table_b`;
      

        Gliffy Diagrams

          Attachments

            Activity

              People

              • Assignee:
                jpechanec Jiri Pechanec
                Reporter:
                rgibaiev Ruslan Gibaiev
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: