Uploaded image for project: 'OptaPlanner'
  1. OptaPlanner
  2. PLANNER-2923

Benchmarker: Incorrect plotting of best score

    XMLWordPrintable

Details

    • Bug
    • Resolution: Unresolved
    • Major
    • None
    • 9.37.0.Final
    • optaplanner-benchmark
    • None
    • Hide

      1. Run GeneralOptaPlannerBenchmarkApp with the attached config and see the results in a browser (not sure if it matters, but tested on MacOS with both Firefox and Safari)

      Show
      1. Run GeneralOptaPlannerBenchmarkApp with the attached config and see the results in a browser (not sure if it matters, but tested on MacOS with both Firefox and Safari)
    • NEW
    • NEW

    Description

      using the attached benchmark config (I simply left the first solver from the default config and cut out the rest) and most up to date main branch the plotting of best score is glitchy.

      The results in the outputed csv files are correct, but only plotting of it looks strange. See the images below:

      Hard constraint best score plotting doesn't work at all. It is always zero regardless of config used and actual value of hard score. In more rare cases it also may happen for soft score that it is not plotted correctly (see the last image), but then it is usually from some point of time (the beginning is plotted fine and then we have flat line similarly like for hard score).

      both soft and hard score doesn't always start from zero point of time and sometimes it also does not finish with solver termination time (see the last image) - secondsSpentLimit (which is harder to reproduce, but possible with some more complicated configs). Usually when it exceeds the solver termination time it is flat line again, even before the termination

      I also attach packaged, ready to view benchmark results.

      And here the results I managed to reproduced only with some more complicated benchmarks (also attached the result in a separate package from 2023-04-23). The flat line may be observed for 2 solvers, which extends way beyond solvers termination time of 10 minutes. The rest 6 solvers are plotted correctly though. Actually according to scores it should not be flat at all even before the termination

      Attachments

        Activity

          People

            Unassigned Unassigned
            michal.dudkiewicz Michał Dudkiewicz (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated: