Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-1240

Debezium locking database durin bootstrap

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Major Major
    • None
    • 0.6.1
    • mysql-connector
    • Hide

      1. Create a DBZ distrubuted KC cluster
      2. submit a config for bootstrap
      3. While bootstrap is going on Add a new KC server to existing cluster
      4. This should trigger a rebalance of KC and we should see lock on the tables. (This is reproducible in large tables)

      Show
      1. Create a DBZ distrubuted KC cluster 2. submit a config for bootstrap 3. While bootstrap is going on Add a new KC server to existing cluster 4. This should trigger a rebalance of KC and we should see lock on the tables. (This is reproducible in large tables)

      We have a issue while running DBZ in distributed KC framework.

      Here is the scenario. if have 3 KC servers running in a distributed mode.

      KC1
      KC2
      KC3

      Now we submit a config for bootstrapping. and while bootstrap is still not complete. if i try to add another server say KC4 to existing cluster this will create a lock on the DB it does looks like all the connectors try to get lock.

      Lock will not be released untill i kill all KC process on all the servers. This causes drastic issues on database (mysql) as well.

      Here is the database process list during that time.

      | Id      | User         | Host               | db    | Command          | Time    | State                                                         | Info                                    | Rows_sent | Rows_examined |
| 7603394 | debezium     | 10.40.0.147:49502  | wepay | Query            |     337 | Sending data                                                  | SELECT * FROM `***`.`**** ` | 166504738 |             0 |
| 7603674 | debezium     | 10.40.255.52:54592 | NULL  | Query            |     186 | Waiting for table flush                                       | FLUSH TABLES WITH READ LOCK             |         0 |             0 |
| 7603804 | debezium     | 10.40.1.46:46880   | NULL  | Query            |     118 | Waiting for table flush                                       | FLUSH TABLES WITH READ LOCK             |         0 |             0 |
| 7603805 | debezium     | 10.40.1.46:46878   | NULL  | Sleep            |     118 |                                                               | NULL                                    |         1 |             0 |
| 7603814 | debezium     | 10.40.1.46:46934   | NULL  | Binlog Dump GTID |     118 | Master has sent all binlog to slave; waiting for more updates | NULL

              Unassigned Unassigned
              akshathpatkar Akshath Patkar (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated: