Details
-
Feature Request
-
Resolution: Duplicate
-
Major
-
None
-
None
-
13
-
Undefined
-
NEW
-
NEW
Description
For datasets with 100k+ entities, a normal planning clone is too expensive.
In same cases, the default solution cloner can take 95% of the performance - and this always happens on the solver thread, so a higher moveThreadCount doesn't help.
To reproduce, open the vaccination scheduling example and pump it up to 300k+ appointments:
# Number of generated vaccination centers demo-data.vaccination-center-count=50 # Number of generated of total booths. Results in 320 000 appointments and 384 000 persons. demo-data.total-booth-count=2000
The intrinsic problem is the iteration of n=300k elements.
Proposal A)
Reuse the same best solution instance and iteratre over the n elements to sync them up. This does make it twice as faster, and heavily reduces GC stress, but still performance is 90% in the solution cloner
=> rejected
Proposal B) Record the winning steps.
- First, create a separate ScoreDirector for the best solution.
- Every time, there is a new winning step, Move.rebase() it on the best solution score director and put it in a queue.
- If the queue.size > n/2, then clear the queue and signal threshold=true
- Every time, there is a new best solution
- and the signal threshold is still false, then replay all the moves from the queue on the best solution and send out the best solution changed event
- if the signal is true, just create a PartitionChangeMove (better name needed) to copy the entire working state to the best solution. Rebase it too, of course, to replay it there. And send out the best solution changed event.
Proposal C) Like B), but put the entire replaying on a separate "best solution" thread (aka consumer thread of SolverManager) and implement skip-ahead. Pitfall: Solver doesn't do thread management normally (except for moveThreadCount threads, for which it uses a thread factory)
Attachments
Issue Links
- duplicates
-
PLANNER-818 Incremental solution cloning
- Open