Details
-
Bug
-
Resolution: Done
-
Major
-
7.23.0.Final, 7.30.0.Final
-
None
Description
In a load test that starts 40 thousand processes with a high timer to keep them active for several days(e.g. 7days). The server runs into an OutOfMemoryError before all instances are being created. The heap dump shows a ejb/jbpm timer and StatefulKnowledgeSession relevent objects for each created process instance remain on heap until timer gets fired.
This is very similar with RHPAM-2106, but the following system property which have been introduced by this JIRA does not help.
-Dorg.jbpm.ejb.timer.local.cache=false
Here is a histogram of customer's heap dump which was taken after forcible FullGC.
Class Name | Objects | Shallow Heap | Retained Heap
-----------------------------------------------------------------------------------------------------------------
| | |
java.util.HashMap | 381,289 | 18,301,872 | >= 1,073,054,072
java.util.HashMap$Node[] | 246,681 | 20,303,896 | >= 1,070,055,320
java.util.HashMap$Node | 1,090,475 | 34,895,200 | >= 1,061,942,088
java.util.Collections$SynchronizedMap | 1,283 | 41,056 | >= 941,503,424
org.jboss.as.ejb3.timerservice.TimerServiceImpl | 1 | 72 | >= 933,633,968
org.jboss.as.ejb3.timerservice.TimerImpl | 42,758 | 3,420,640 | >= 930,372,512
org.jbpm.services.ejb.timer.EjbTimerJob | 42,758 | 684,128 | >= 917,374,040
org.jbpm.persistence.timer.GlobalJpaTimerJobInstance | 42,758 | 2,052,384 | >= 916,689,920
org.drools.core.time.SelfRemovalJobContext | 42,758 | 1,026,192 | >= 904,033,352
org.jbpm.process.instance.timer.TimerManager$ProcessJobContext | 42,758 | 1,710,320 | >= 903,007,112
org.drools.core.impl.StatefulKnowledgeSessionImpl | 42,763 | 6,499,976 | >= 894,839,008
java.lang.Object[] | 467,965 | 15,955,216 | >= 777,768,176
java.util.ArrayList | 245,955 | 5,902,920 | >= 754,555,928
org.drools.core.SessionConfigurationImpl | 42,767 | 2,737,088 | >= 669,155,688
org.kie.internal.utils.ChainedProperties | 42,777 | 1,026,648 | >= 663,159,080
One object 'org.jboss.as.ejb3.timerservice.TimerServiceImpl' [1] has too many
object 'org.jboss.as.ejb3.timerservice.TimerImpl' by HashMap,
which contains object 'org.jbpm.persistence.timer.GlobalJpaTimerJobInstance',
which refers org.drools.core.impl.StatefulKnowledgeSessionImpl indirectly.
Class Name | Shallow Heap | Retained Heap | Percentage
------------------------------------------------------------------------------------------------------------------------------------------
org.jboss.as.ejb3.timerservice.TimerServiceImpl @ 0x8839f1a0 | 72 | 933,633,664 | 74.91%
|- java.util.Collections$SynchronizedMap @ 0x88bde118 | 32 | 932,003,040 | 74.78%
| |- java.util.HashMap @ 0x88bde138 | 48 | 932,002,968 | 74.78%
| | '- java.util.HashMap$Node[65536] @ 0xb281bf08 | 262,160 | 932,002,920 | 74.78%
| | |- java.util.HashMap$Node @ 0x9b15e798 | 32 | 130,752 | 0.01%
| | | |- java.util.HashMap$Node @ 0xb010f910 | 32 | 108,960 | 0.01%
| | | |- org.jboss.as.ejb3.timerservice.TimerImpl @ 0x9b15e7b8 | 80 | 21,760 | 0.00%
| | | | |- org.jbpm.services.ejb.timer.EjbTimerJob @ 0x9b15e808 | 16 | 21,456 | 0.00%
| | | | | '- org.jbpm.persistence.timer.GlobalJpaTimerJobInstance @ 0x9b15e818 | 48 | 21,440 | 0.00%
| | | | |- org.jboss.as.ejb3.timerservice.TimerHandleImpl @ 0x9b163bf0 | 32 | 152 | 0.00%
| | | | |- java.util.concurrent.Semaphore @ 0x9b163c88 | 16 | 48 | 0.00%
| | | | |- java.util.Date @ 0x9b15e8b0 | 24 | 24 | 0.00%
| | | | '- Total: 4 entries | | |
| | | '- Total: 2 entries | | |
| | |- java.util.HashMap$Node @ 0x94b23760 | 32 | 130,752 | 0.01%
| | |- java.util.HashMap$Node @ 0x9810ab28 | 32 | 130,752 | 0.01%
| | |- java.util.HashMap$Node @ 0xaad8d8f8 | 32 | 130,752 | 0.01%
...
Attachments
Issue Links
- is cloned by
-
-
- Closed
-
