-
Sub-task
-
Resolution: Done
-
Major
-
None
-
None
We need a "catch-up reader" for DBZ to be able to handle table additions to an existing connector's filter.
This catch up reader will merge two separate sub-connectors (two binlog reading processes within a single connector with disparate, non-intersecting filters and potentially different binlog positions information) into a single unified connector.
To do this, it will determine which sub-connector is "lagging", and read only events filtered by that sub-connector till it reaches the offset of the "leading" sub-connector.
At that point, we are free to create a new binlog reading connector with the joined filter offset information of the two sub-connectors and the binlog position that now both sub-connectors have reached. Once this connector has been created, the two sub-connectors have effectively merged into a single connector.