Currently we use WeakHashMap, but should not, reasons outlined below. We could replace it with a LazyRemovalCache. Andrew's email refers to SecretKeys but this probably also applies to Ciphers.

Andrew Haley's email:

TL/DR: Please don't use WeakReferences, SoftReferences, etc. to cache
any data which might point to native memory. In particular, never do
this with instances of java.security.Key. Instead, implement either
some kind of ageing strategy or a fixed-size cache.
...
This is a warning to anybody who might cache crypto keys.

A customer has been having problems with the exhaustion of native
memory before the Java heap is full. It was fun trying to track down
the cause, but it's now happened several times to several customers,
and it's a serious problem for real-world usage in app servers.

PKCS#11 is a standard way to communicate between applications and
crypto libraries. There is a Java crypto provider which supports
PKCS#11. Some of our customers must use this provider in order to get
FIPS certification.

The problem is this:

A crypto key is a buffer in memory, allocated by the PKCS#11 native
library. It's accessed via a handle which is stored as an integer
field in a Java object. This Java object is a PhantomReference, so
when the garbage collector detects that a crypto key is no longer
reachable it is closed and the associated native memory is freed.

Modern garbage collectors don't much bother to process objects in the
old generation because it's not usually worthwhile. Thus, crypto keys
don't get recycled very quickly. They can pile up in the old
generation. This isn't a problem for the Java heap because the
objects containing the references to crypto keys are very small.
Unfortunately, the native side of a crypto key is much bigger, maybe
up to a thousand times bigger. So if we have 4000 stale crypto keys
in the heap that's not a problem, a few kbytes. But the native memory
may be a megabyte.

This problem is made even worse by Tomcat because it uses
SoftReferences to cache crypto keys. SoftReferences are processed
lazily, and maybe not at all until the Java heap runs out of memory.
Unfortunately it doesn't, but the machine runs out of native memory
instead.

We could solve this simply by making instances of PKCS#11 keys really
big Java objects by padding with dummy fields. Then, the GC would
collect them quickly. This does work but it seriously impacts
performance. Also, we could tweak the garbage collectors to clear out
stale references more enthusiastically, but this impacts performance
even more. There are some controls with the G1 collector which
process SoftReferences more aggressively and these help, but again at
the cost of performance.

Finally: the Shanandoah collector we're working on handles this
problem much better than the older collectors, but it's some
way off.