Status: Closed (View Workflow)
Affects Version/s: 1.3.0.Alpha1, 1.4.1.Final
Fix Version/s: 1.5.0.Beta2
Steps to Reproduce:
create table into mysql database
2. connect debezium
3. run attached file as insert statement
Watch debezium container CPU go 100% and no future events nor connector fail with error.create table into mysql database CREATE TABLE `events_blackhole` ( `id` int (11) unsigned NOT NULL AUTO_INCREMENT, `topic` varchar(100) NOT NULL, `message_key` varchar(255) DEFAULT NULL, `message_value` json NOT NULL, `message_headers` json DEFAULT NULL, PRIMARY KEY (`id`) ) ENGINE=InnoDB AUTO_INCREMENT=3 DEFAULT CHARSET=utf8; 2. connect debezium 3. run attached file as insert statement Watch debezium container CPU go 100% and no future events nor connector fail with error.
After upgrading debezium from 1.2 to 1.4 in production we noticed that in some cases debezium got infinitely stuck processing particular event. CPU at 100%... Restart or downgrade to 1.3 didn't solve it. After downloading 1.2 it continue as if normal.
After discussing on gitter chat we extracted debug level logs of it and also thread dump. From there it was clearly issue with thead that handles JSON parsing in binlog.
So afterwards did a mysql dump and found the exact event that caused it. Also reproduce it locally.