Details
-
Bug
-
Resolution: Done
-
Major
-
10.0.0.Alpha3
-
None
Description
The code was refactored in ComposedSegmentedLoadWriteStore to be more non blocking friendly. Unfortunately due to how groupBy and flatMap interact it is possible for the bulkUpdate to never complete.
FlatMap by default sets a parallelism level of 128. this means it will request 128 groups from groupBy, but unfortunately if there are more than 128 groups, it will never complete as groupBy must publish all groups before a single one can complete. Thus any time we use flatMap after a groupBy we must either set the parallelism level to Integer.MAX_VALUE or to an explicit value if we know how many groups at max there will be.