Deadlock: expiration blocks state transfer

We use replicated caches configured with expiration max-idle and file-store in a web application in WildFly. It's very risky to redeploy/restart because of this bug. Here's what happens.

1. StateProviderImpl.publishStoreEntries callls persistenceManager.publishEntries(segments, k -> !dataContainer.containsKey(k), ....
2. SingleFileStore.entryPublisher uses the above !dataContainer.containsKey(k) filter in a synchronized (entries) block.
3. AbstractInternalDataContainer.containsKey calls expirationManager.entryExpiredInMemory with a tremendous side effect by CompletionStages and lambdas (if there is an expired key!)
4. that eventually calls persistenceManager.deleteFromAllStores in DistCacheWriterInterceptor.visitRemoveCommand
5. that calls storeStatus.store.delete(segment, key) in PersistenceManagerImpl.deleteFromAllStores
6. that calls entries.remove(key) in SingleFileStore.delete
But entries is locked in step 2 in a different thread, so the future in step 3 never completes and joins.

The attached StateTransferExpiredFileStoreTest.java reproduces this bug.