Details
-
Bug
-
Resolution: Done
-
Major
-
0.3.4
-
None
Description
Debezium seems to have problems when columns with type NVARCHAR appears in the binlog. NVARCHAR is equivalent to a VARCHAR CHARACTER SET utf8 (http://dev.mysql.com/doc/refman/5.7/en/charset-national.html).
It works fine when the table exists when starting Debezium, then for the initial snapshot it is parsed as a regular VARCHAR and everything is OK:
CREATE TABLE test (id INT PRIMARY KEY, someColumn NVARCHAR(100) NOT NULL);
2016-10-26 08:36:08,757] INFO CREATE DATABASE test (io.debezium.connector.mysql.SnapshotReader:569)
[2016-10-26 08:36:08,799] INFO USE test (io.debezium.connector.mysql.SnapshotReader:569)
[2016-10-26 08:36:08,836] INFO CREATE TABLE `test` (
`id` int(11) NOT NULL,
`someColumn` varchar(100) CHARACTER SET utf8 NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1 (io.debezium.connector.mysql.SnapshotReader:569)
[2016-10-26 08:36:08,876] INFO Step 7: releasing global read lock to enable MySQL writes (io.debezium.connector.mysql.SnapshotReader:325)
But, if I create a new table with Debezium running with a NVARCHAR, so that we get the CREATE TABLE command in the binlog stream, then I get this error:
CREATE TABLE test2 (id INT PRIMARY KEY, someColumn NVARCHAR(100) NOT NULL);
[2016-10-26 08:36:50,373] ERROR Error parsing DDL statement and updating tables: CREATE TABLE test2 (id INT PRIMARY KEY, someColumn NVARCHAR(100) NOT NULL) (io.debezium.connector.mysql.MySqlSchema:328) io.debezium.text.ParsingException: Expecting ')' at line 1, column 60 but found '(': someColumn NVARCHAR ===>> (100) NOT NULL) at io.debezium.text.TokenStream.consume(TokenStream.java:694) at io.debezium.connector.mysql.MySqlDdlParser.parseCreateDefinitionList(MySqlDdlParser.java:548) at io.debezium.connector.mysql.MySqlDdlParser.parseCreateTable(MySqlDdlParser.java:365) at io.debezium.connector.mysql.MySqlDdlParser.parseCreate(MySqlDdlParser.java:271) at io.debezium.connector.mysql.MySqlDdlParser.parseNextStatement(MySqlDdlParser.java:146) at io.debezium.relational.ddl.DdlParser.parse(DdlParser.java:285) at io.debezium.relational.ddl.DdlParser.parse(DdlParser.java:266) at io.debezium.connector.mysql.MySqlSchema.applyDdl(MySqlSchema.java:326) at io.debezium.connector.mysql.BinlogReader.handleQueryEvent(BinlogReader.java:390) at io.debezium.connector.mysql.BinlogReader.handleEvent(BinlogReader.java:295) at com.github.shyiko.mysql.binlog.BinaryLogClient.notifyEventListeners(BinaryLogClient.java:877) at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:735) at com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:451) at com.github.shyiko.mysql.binlog.BinaryLogClient$5.run(BinaryLogClient.java:632) at java.lang.Thread.run(Thread.java:745)
I assume this applies to the other synonyms for NVARCHAR listed here as well: http://dev.mysql.com/doc/refman/5.7/en/charset-national.html
