Things that are problematic in the ranking code (all classes under org.drools.planner.benchmark.core.ranking):

• score can be null, totalScore can be null, ... when there are failures
• org.drools.planner.benchmark.core.ranking.TotalRankSolverBenchmarkRankingWeightFactory: Usage of SolverBenchmark.getScoreList() because some SolverBenchmark might have extra problem inputFiles.

The question is, should the ranking first compare the failureCount? Yes, I think it should.
What code should do that? the DefaultPlannerBenchmark, by default, or the rankers itself?