Uploaded image for project: 'OptaPlanner'
  1. OptaPlanner
  2. PLANNER-588

Benchmark report's standard deviation should take initScore into account and use score level labels if possible

XMLWordPrintable

    • NEW
    • NEW

      1) ScoreUtils.extractLevelDoubles() ignores Score.getInitScore(). Start reading from SolverBenchmarkResult#accumulateResults() which does:

              standardDeviationDoubles = StatisticUtils.determineStandardDeviationDoubles(singleBenchmarkResultList, averageScore, getSuccessCount());
      

      2) The string value of standard deviation doesn't always make sense, for example for the nurse rostering sprint benchmark:

      Standard Deviation 	sprint01 	sprint02
      0.0E0/1.0 	0hard/-59soft  0 	0hard/-61soft  0
      0.0E0/0.0E0 	0hard/-64soft  1 	0hard/-64soft  1
      

      What does that trailing 0 mean? 0.0E0/0.0E0

      3) Can we add the hard and soft level strings from ScoreDefinition.getLevelLabels() ?

      4) Better yet, can we calculate the standard deviation as a Score object somehow? Then we 'd get the levels formatting etc out-of-the-box? It's hard though as we probably need to use doubles to calculate the standard deviation (underflow, overlfow, rounding, ...).

            Unassigned Unassigned
            gdesmet@redhat.com Geoffrey De Smet (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated: