With customPhase, users can plug in their own algorithms easily. But any parameters for such an algorithm would be hardcoded. Yet, they might still want to tweak their parameters and reuse the benchmarker to do that.

I propose adding a new method to the CustomPhaseCommand interface (which isn't part of the public api yet, so we can still break backwards compatibility by documenting it in the upgrade recipe) called setParameters():

public interface CustomPhaseCommand {

   void setParameters(Map<String, String> parameters);   

    void changeWorkingSolution(ScoreDirector scoreDirector);

}

Note: There parameters are always strings, parsing to ints etc is the setParameters's responsibility. That method is free to fail fast when needed, as it's called during bootstrap.