-
Enhancement
-
Resolution: Done
-
Major
-
None
In the situation in which the result of the Map/Reduce task is large, returning it to the originator does not work, as the originator would run OOM. In order to drop this limitation, would be good to:
- have a result cache
- put all the M/R entries into this result cache
It should be possible to specify this cache to the Map/Reduce task to be executed (cache name), as users might reuse the cache and configure it externally.
The caller will be notified once the data Map/Reduce is finished. We'd need a way for the user to get all the keys in this cache for further processing.