Current, both state transfer protocols close the barrier, process the state request and re-open it. However, this is NOT done for state responses !
The following issue could happen:
- D requests the state from A
- B multicasts #11
- D receives the digest A:5, B:10, C:44 and (in parallel) B:11
Scenario #1:
- [NAKACK] D applies the digest (is 10 for B)
- [NAKACK] D receives B:11 (is 11 now for B)
- D delivers B:11 to the app, changes state
- D sets state, overwrites the change made by B:11
--> B:11 will not get retransmitted as the digest in NAKACK already has it: the change made by B:11 is lost !
Scenario #2:
- [NAKACK] D receives B:11 (is 11 now for B)
- [NAKACK] D applies the digest: resets the digest for B to #10
- D sets state
- D delivers B:11 to the app, changes state
--> B:11 will get retransmitted as the digest in NAKACK doesn't contain it
--> B:11 changes state again: B:11 is delivered twice !
So scenario #1 loses B:11 while scenario #2 delivers B:11 twice, which has to be prevented.
SOLUTION:
- Close BARRIER before state response is handled (similar to what we do with a state request)
- This way, either B:11 is handled and changes the digest in NAKACK and the app state before we set the digest in NAKACK and the app state, OR the digest is set in NAKACK and the app state is set from the state response and THEN B:11 is handled