The fix for FDP-757 introduced a performance regression with regards to the southbound database.

The patch made it so that changes to logical routers would no longer result in a full recompute of all logical flows. Instead, the logical flows relating to the inserted/updated/deleted router would be processed incrementally. This resulted in a performance improvement in northd since we were computing fewer logical flows.

As background, if the same logical flow is created for more than one datapath, then we create a datapath group for the set of datapaths that the logical flow refers to. This way, we only have to insert a single logical flow, and the datapath group helps us to ensure the logical flow applies to all datapath bindings in the datapath group. Previous incremental processing of logical flows didn't have to worry much about the underlying datapath groups because changes to the datapaths would result in a full recompute of the logical flows.

With the patch introduced for FDP-757, this is different. Now, datapath changes are what directly result in the new logical flow incremental processing. This means that every logical flow that gets incrementally processed will send with it a corresponding operation to insert or update logical datapath groups, since the logical flows will always have a change to their related datapaths. This effect is worse when multiple logical routers have been changed, since it means that every logical flow will send an insert/update for each changed logical router.

For this task, we want to separate the datapath group syncing from the logical flow syncing. This way, when logical routers change, we can start by incrementally processing the logical flows. As we process the logical flows, we can update their in-memory datapath bitmap. Once all logical flows have been processed, we can then iterate over the entire lflow table and update the southbound datapath groups. This will result in a much smaller set of messages to send to the southbound database to insert/update the datapath groups. After this, we can update the southbound logical flows to ensure their logical_dp_group columns refer to any inserted datapath groups properly.

This can either be done within the existing en_lflow engine node, or we could separate datapath group processing into a new engine node that takes the lflow_table from en_lflow as input.