-
Bug
-
Resolution: Done
-
Major
-
7.13.0.GA
-
False
-
None
-
False
-
-
-
-
-
-
CR1
-
-
-
---
-
---
-
2022 Week 41-43 (from Oct 10), 2022 Week 44-46 (from Oct 31)
rhpam performance downgrades significantly after migrating to 7.12.1.
The following is the thread dump analysis:
----------------------
"default task-817" #295624 prio=5 os_prio=0 cpu=5281.12ms elapsed=587.71s tid=0x0000564f06af8000 nid=0x4b66e waiting for monitor entry [0x00007fa4818c9000]
java.lang.Thread.State: BLOCKED (on object monitor)
at org.kie.server.services.jbpm.cluster.InfinispanClusterAwareService.addData(InfinispanClusterAwareService.java:179)
- waiting to lock <0x000000040cfab2e8> (a org.kie.server.services.jbpm.cluster.InfinispanClusterAwareService)
at org.kie.server.services.jbpm.cluster.ClusteredJobFailOverListener.afterJobScheduled(ClusteredJobFailOverListener.java:71)
at org.jbpm.executor.impl.event.ExecutorEventSupportImpl.lambda$fireAfterJobScheduled$3(ExecutorEventSupportImpl.java:50)
at org.jbpm.executor.impl.event.ExecutorEventSupportImpl$$Lambda$3333/0x00000008026fc440.accept(Unknown Source)
at org.drools.core.event.AbstractEventSupport.notifyAllListeners(AbstractEventSupport.java:60)
at org.jbpm.executor.impl.event.ExecutorEventSupportImpl.fireAfterJobScheduled(ExecutorEventSupportImpl.java:50)
at org.jbpm.executor.impl.ExecutorImpl.scheduleRequest(ExecutorImpl.java:419)
at org.jbpm.executor.impl.ExecutorImpl.scheduleRequest(ExecutorImpl.java:319)
at org.jbpm.executor.impl.ExecutorServiceImpl.scheduleRequest(ExecutorServiceImpl.java:139)
at org.jbpm.workflow.instance.node.AsyncEventNodeInstance.internalTrigger(AsyncEventNodeInstance.java:63)
....
----------------------
The thread owner of the lock:
----------------------
"Thread-81 (ActiveMQ-client-global-threads)" #244950 daemon prio=5 os_prio=0 cpu=81160.20ms elapsed=7642.90s tid=0x0000564f0a484000 nid=0x3ece5 waiting on condition [0x00007fa47aa7d000]
java.lang.Thread.State: TIMED_WAITING (parking)
at jdk.internal.misc.Unsafe.park(java.base@11.0.14.1/Native Method)
- parking to wait for <0x00000005e58606e8> (a java.util.concurrent.CompletableFuture$Signaller)
at java.util.concurrent.locks.LockSupport.parkNanos(java.base@11.0.14.1/LockSupport.java:234)
at java.util.concurrent.CompletableFuture$Signaller.block(java.base@11.0.14.1/CompletableFuture.java:1798)
at java.util.concurrent.ForkJoinPool.managedBlock(java.base@11.0.14.1/ForkJoinPool.java:3128)
at java.util.concurrent.CompletableFuture.timedGet(java.base@11.0.14.1/CompletableFuture.java:1868)
at java.util.concurrent.CompletableFuture.get(java.base@11.0.14.1/CompletableFuture.java:2021)
at org.infinispan.util.concurrent.CompletableFutures.await(CompletableFutures.java:125)
at org.infinispan.interceptors.impl.SimpleAsyncInvocationStage.get(SimpleAsyncInvocationStage.java:36)
at org.infinispan.interceptors.impl.AsyncInterceptorChainImpl.invoke(AsyncInterceptorChainImpl.java:246)
at org.infinispan.cache.impl.InvocationHelper.doInvoke(InvocationHelper.java:298)
at org.infinispan.cache.impl.InvocationHelper.executeCommandWithInjectedTx(InvocationHelper.java:203)
at org.infinispan.cache.impl.InvocationHelper.invoke(InvocationHelper.java:101)
at org.infinispan.cache.impl.InvocationHelper.invoke(InvocationHelper.java:84)
at org.infinispan.cache.impl.CacheImpl.computeIfAbsentInternal(CacheImpl.java:367)
at org.infinispan.cache.impl.CacheImpl.computeIfAbsent(CacheImpl.java:357)
at org.infinispan.cache.impl.CacheImpl.computeIfAbsent(CacheImpl.java:337)
at org.infinispan.cache.impl.AbstractDelegatingCache.computeIfAbsent(AbstractDelegatingCache.java:424)
at org.infinispan.cache.impl.EncoderCache.computeIfAbsent(EncoderCache.java:672)
at org.infinispan.cache.impl.AbstractDelegatingCache.computeIfAbsent(AbstractDelegatingCache.java:424)
at org.infinispan.Cache.computeIfAbsent(Cache.java:388)
at org.kie.server.services.jbpm.cluster.InfinispanClusterAwareService.addData(InfinispanClusterAwareService.java:180) - locked <0x000000040cfab2e8> (a org.kie.server.services.jbpm.cluster.InfinispanClusterAwareService)
at org.kie.server.services.jbpm.cluster.ClusteredJobFailOverListener.afterJobScheduled(ClusteredJobFailOverListener.java:71)
at org.jbpm.executor.impl.event.ExecutorEventSupportImpl.lambda$fireAfterJobScheduled$3(ExecutorEventSupportImpl.java:50)
at org.jbpm.executor.impl.event.ExecutorEventSupportImpl$$Lambda$3333/0x00000008026fc440.accept(Unknown Source)
at org.drools.core.event.AbstractEventSupport.notifyAllListeners(AbstractEventSupport.java:60)
at org.jbpm.executor.impl.event.ExecutorEventSupportImpl.fireAfterJobScheduled(ExecutorEventSupportImpl.java:50)
....
----------------------
We suspect that this is due to the changes made in 7.12.1 [1]
We notice that a couple of synchronized blocks are added in these changes.
- is cloned by
-
JBPM-10127 Performance downgrade possibly related to RHPAM-4069
- Pull Request Sent