Currently some testsuite cases do fail with an error instead of a failure in Maven surefire terms. In JUnit tests the BeforeClass and AfterClass are called independet. One has to check preconditions in both, it's not correct to rely on state of BeforeClass in AfterClass. In any case a unit tests with a known assumption should fail with a failure at the earliest point.

UseCase where this came up:
I was trying to identify the issue where the LoggingSubSystem Configuration error was not showing up in the Stacktraces of the tests. For example this NPE was a follow up, but should have shown up earlier:

[ERROR] org.jboss.as.test.integration.domain.AdminOnlyModeTestCase  Time elapsed: 0 s  <<< ERROR!
  java.lang.NullPointerException
      at org.jboss.as.test.integration.domain.AdminOnlyModeTestCase.tearDownDomain(AdminOnlyModeTestCase.java:94)
      at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      at java.base/java.lang.reflect.Method.invoke(Method.java:566)
      at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)

Solution: In most cases it was possible to have Asserts (@Before) or tearDown(@After) methods. Sometimes it requires checking for a started server.
The behaviour to break shall be kept. If it's a NPE at the moment, there shall be not a if(x != null) statement but an Assert.assertNotNull.