When passing a cache loader instance to the cache loader configuration builder it's not the instance that's used, instead the class info is fetched from it and it is instantiated through reflection. This is not good, as it doesn't allow programmatic configuration of that instance and is counter intuitive.
There is a comment on the code that it needs to be fixed, but couldn't find any related jiras: https://github.com/infinispan/infinispan/blob/master/core/src/main/java/org/infinispan/configuration/cache/LoaderConfigurationBuilder.java#L51-54