While InfiniSpan is very fast in-memory, once passivation is turned on, things get slow for me.
On my i5-2400 with a Crucial C300 SSD, it takes about 11 ms to passivate a 2964 B value. This comes down to a write speed of 250 to 300 KB/s. The disk can write up to 180 MB/s so it's not the bottleneck.
Clearly, passivation should be faster than that.
Profiling with YourKit shows most of the hot spots are centered around FileCacheStore, the JBoss Marshalling library and various bucket and ImmortalCacheEntry methods. Locking also seems to play a role. Here's a screenshot:
To make profiling easier for you, I've taken my test case and turned it into a small Maven project. Simply running mvn install should run the test, ready for profiling.
Of course, it could also be that my configuration is suboptimal. However, in my experiments, many configurations were unusable (hitting the file handle limit), which is not a good thing.
The project is available at http://johannburkard.de/resources/Johann/infinispan-performance.zip
I've also found one or two other things I think are more costly than they would need to be
wrapping the FileInputStream in a BufferedInputStream is unnecessary because you already have a buffer in byte buffer. I think I have seen this before in InfiniSpan, so you might want to check for occurrences of BufferedInputStream or BufferedOutputStream.
f.exists() is essentially a syscall. Just calling f.delete() is enough.
This could also be shortened to