In RHDG 7.1/7.2, a SingleFileStore is very fast as the keys are stored in the JVM heap and even the values are likely to be in memory, in the OS buffers. Users were able to add a SingleFileStore with passivation to their cache and they would see virtually no overhead from the store as long as the cache entries fit in the data container limits.

In RHDG 7.3/Infinispan 9.4 there is more overhead as we try to avoid blocking on the Netty thread, so we delegate cache operations to a worker thread when persistence is enabled. This means 2 thread changes for a cache get: one to run the operation and another to write the response (because the response must be written from the netty connection's event loop thread).

In RHDG 8.1/Infinispan 11.0 the cache operation is processed on the Netty thread again, but now the store operation is sent to a blocking thread, so there are 3 thread changes for a cache get: a) start the SingleFileStore operation on a blocking thread b) get back to a non-blocking thread for the rest of the operation and c) write the response.

In addition to the thread switching, there are also some extra allocations because the non-blocking code now uses CompletableFuture and RxJava3 Flowable everywhere.

Because SingleFileStore keeps the keys in heap, we don't need to block in order to check if a key exists or to remove a non-existent key. We can thus avoid the thread switching, and we can also avoid some allocations if the CompletableFutures are completed synchronously.