This has been happening to a few of the modules during every build for some time, but with the most recent Apple JDK (1.6.0_51) this now happens with every run of Surefire. It is extremely annoying, because the window opens on top of whatever active window I'm using, and any key press will open the help.

The fix is relatively simple, and supposedly doesn't affect other platforms: include "-Djava.awt.headless=true" in the Surefire configuration.