TestNG doesn't like exceptions being thrown from @Factory methods, so when MultipleCacheManager.defaultFactory() finds a problem with the concrete factory() implementation or with the annotations it replaces the test instance with a TestFrameworkFailure.

Turns out that doesn't actually work: TestNG ignores the TestFrameworkFailure instance and runs the tests on a "default instance" of the test class. This is ok when the problem is the concrete class didn't override factory() and very likely the test author wanted a single test instance, but it's not ok for other errors like factory() being copy-pasted from super with the wrong class name.

We can make TestNG report our factory failures if we implement IInstanceInfo and mock all the test/configuration methods to throw our exception. We just need to make all configuration methods non-final so we can mock them.