Details
-
Bug
-
Resolution: Done
-
Major
-
9.4.23.Final
-
None
Description
While a store is unavailable, PersistenceManagerImpl caches a StoreUnavailableException instance and throws it for every persistence operation.
If the cache operation causing the StoreUnavailableException is blocking (cache.put(key, value) instead of cache.putAsync(key, value)), then it adds a suppressed TraceException to the StoreUnavailableException. Since the StoreUnavailableException is the same, it keeps collecting new suppressed exceptions as long as the store is unavailable.
[2021-10-05 13:06:59,256+0000] WARN [org.infinispan.statetransfer.StateConsumerImpl] (stateTransferExecutor-thread--p25-t48) ISPN000016: Problem Store org.infinispan.persistence.remote.RemoteStore@12ff78f6 is unavailable encountered when applying state for key 2fd32c62-b8b1-4142-ab74-fb6f8cae3b76!: org.infinispan.persistence.spi.StoreUnavailableException: Store org.infinispan.persistence.remote.RemoteStore@12ff78f6 is unavailable at org.infinispan.persistence.manager.PersistenceManagerImpl.pollStoreAvailability(PersistenceManagerImpl.java:204) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at org.wildfly.clustering.service.concurrent.ClassLoaderThreadFactory.lambda$newThread$0(ClassLoaderThreadFactory.java:47) at java.lang.Thread.run(Thread.java:748) Suppressed: org.infinispan.util.logging.TraceException at org.infinispan.interceptors.impl.SimpleAsyncInvocationStage.get(SimpleAsyncInvocationStage.java:41) at org.infinispan.interceptors.impl.AsyncInterceptorChainImpl.invoke(AsyncInterceptorChainImpl.java:250) at org.infinispan.cache.impl.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1915) at org.infinispan.cache.impl.CacheImpl.putIfAbsent(CacheImpl.java:1471) at org.infinispan.cache.impl.DecoratedCache.putIfAbsent(DecoratedCache.java:695) at org.infinispan.cache.impl.DecoratedCache.putIfAbsent(DecoratedCache.java:610) at org.infinispan.cache.impl.AbstractDelegatingCache.putIfAbsent(AbstractDelegatingCache.java:353) at org.infinispan.cache.impl.EncoderCache.putIfAbsent(EncoderCache.java:493) at org.infinispan.cache.impl.AbstractDelegatingCache.putIfAbsent(AbstractDelegatingCache.java:353) Suppressed: org.infinispan.util.logging.TraceException at org.infinispan.interceptors.impl.SimpleAsyncInvocationStage.get(SimpleAsyncInvocationStage.java:41) at org.infinispan.interceptors.impl.AsyncInterceptorChainImpl.invoke(AsyncInterceptorChainImpl.java:250) at org.infinispan.cache.impl.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1915) at org.infinispan.cache.impl.CacheImpl.putIfAbsent(CacheImpl.java:1471) at org.infinispan.cache.impl.DecoratedCache.putIfAbsent(DecoratedCache.java:695) at org.infinispan.cache.impl.DecoratedCache.putIfAbsent(DecoratedCache.java:610) at org.infinispan.cache.impl.AbstractDelegatingCache.putIfAbsent(AbstractDelegatingCache.java:353) at org.infinispan.cache.impl.EncoderCache.putIfAbsent(EncoderCache.java:493) at org.infinispan.cache.impl.AbstractDelegatingCache.putIfAbsent(AbstractDelegatingCache.java:353)
In older Infinispan versions, this also applies to state transfer inserting entries
Suppressed: org.infinispan.util.logging.TraceException at org.infinispan.interceptors.impl.SimpleAsyncInvocationStage.get(SimpleAsyncInvocationStage.java:41) at org.infinispan.interceptors.impl.AsyncInterceptorChainImpl.invoke(AsyncInterceptorChainImpl.java:250) at org.infinispan.statetransfer.StateConsumerImpl.doApplyState(StateConsumerImpl.java:650) at org.infinispan.statetransfer.StateConsumerImpl.applyChunk(StateConsumerImpl.java:616) at org.infinispan.statetransfer.StateConsumerImpl.lambda$applyState$1(StateConsumerImpl.java:572)
This slows things down as the entire collection of suppressed exceptions is logged at least once,
in InvocationContextInterceptor, then maybe once more in the application or in StateConsumerImpl.