-
Bug
-
Resolution: Done
-
Blocker
-
8.0.0.GA-CR2
-
None
-
False
-
None
-
False
-
-
-
-
-
-
-
-
Scenario: we have a 4 nodes cluster where we deploy a clustered application ( [^clusterbench-ee10.ear] ) containing some stateful EJB named RemoteStatefulSB:
java:global/clusterbench-ee10/clusterbench-ee10-ejb/RemoteStatefulSBImpl!org.jboss.test.clusterbench.ejb.stateful.RemoteStatefulSB java:app/clusterbench-ee10-ejb/RemoteStatefulSBImpl!org.jboss.test.clusterbench.ejb.stateful.RemoteStatefulSB java:module/RemoteStatefulSBImpl!org.jboss.test.clusterbench.ejb.stateful.RemoteStatefulSB java:jboss/exported/clusterbench-ee10/clusterbench-ee10-ejb/RemoteStatefulSBImpl!org.jboss.test.clusterbench.ejb.stateful.RemoteStatefulSB ejb:clusterbench-ee10/clusterbench-ee10-ejb/RemoteStatefulSBImpl!org.jboss.test.clusterbench.ejb.stateful.RemoteStatefulSB?stateful java:global/clusterbench-ee10/clusterbench-ee10-ejb/RemoteStatefulSBImpl java:app/clusterbench-ee10-ejb/RemoteStatefulSBImpl java:module/RemoteStatefulSBImpl
We have a remote EJB client application performing remote invocations on the cluster nodes, using the emote+http protocol:
remote+http://10.0.99.245:8080,remote+http://10.0.99.224:8080,remote+http://10.0.99.222:8080,remote+http://10.0.97.56:8080
The client applications creates 4000 sessions; invocations are repeated every 4 seconds;
Everything works fine if don't restart any node;
As soon as we shut-down and restart the EAP instance on the first node, EAP memory starts to grow:
EAP memory continues to grow:
until, eventually, the java process is killed by the OS:
$ sudo dmesg | tail -7 [12671.708228] [ 7229] 600 7229 88440 188 200704 0 0 gio [12671.709661] [ 8476] 600 8476 55691 129 81920 0 0 standalone.sh [12671.711250] [ 8627] 600 8627 1018253 723463 6549504 0 0 java [12671.712720] [ 9163] 0 9163 54274 18 69632 0 0 sleep [12671.714234] [ 9186] 600 9186 65958 141 155648 0 0 top [12671.715742] oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0,global_oom,task_memcg=/user.slice/user-600.slice/session-4.scope,task=java,pid=8627,uid=600 [12671.719219] Out of memory: Killed process 8627 (java) total-vm:4073012kB, anon-rss:2893852kB, file-rss:0kB, shmem-rss:0kB, UID:600 pgtables:6396kB oom_score_adj:0
We attached the memory dumps taken with e.g.
jmap -dump:live,format=b,file=$JMAP_FILE $MYPID
before the java process was oom killed: jmap-8627-4435.zip
- is caused by
-
REM3-404 Remoting connections closed during greetings exchange after HTTP upgrade are not properly cleaned after closed
- Closed
- is cloned by
-
JBEAP-26462 (7.4.z) OOM Error after node restart, in a 4 nodes cluster
- Closed
- is related to
-
WFLY-18700 java.lang.OutOfMemoryError: Direct buffer memory
- Closed
- relates to
-
UNDERTOW-2336 CVE-2024-1635 At Http upgrade to remoting, WriteTimeoutStreamSinkConduit leaks connections if RemotingConnection is closed by Remoting ServerConnectionOpenListener
- Closed
-
XNIO-427 ClosedChannelException when NioSocketConduit.handleReady invokes write listener after read listener closes connection
- Closed
- mentioned on