Loading...

Type: Bug
Resolution: Duplicate
Priority: Critical
Fix Version/s: None
Affects Version/s: 7.0.0.ER6
Component/s: Undertow
Labels:
None

CDW devel_ack:
CDW docs_ack:
CDW pm_ack:
CDW qa_ack:
CDW release:
Target Release:

7.0.0.GA
Steps to Reproduce:
Hide

How to run the test locally:

git clone git://git.app.eng.bos.redhat.com/jbossqe/eap-tests-hornetq.git cd eap-tests-hornetq/scripts/ git checkout refactoring_modules groovy -DEAP_VERSION=7.0.0.ER6 PrepareServers7.groovy export WORKSPACE=$PWD export JBOSS_HOME_1=$WORKSPACE/server1/jboss-eap export JBOSS_HOME_2=$WORKSPACE/server2/jboss-eap export JBOSS_HOME_3=$WORKSPACE/server3/jboss-eap export JBOSS_HOME_4=$WORKSPACE/server4/jboss-eap cd ../jboss-hornetq-testsuite/ mvn clean test -Dtest=CliReloadTest#reloadServerOOMTest -DfailIfNoTests=false -Deap=7x | tee log After the test is started: Start Visualvm and check memory grow of EAP server.
Show
How to run the test locally: git clone git: //git.app.eng.bos.redhat.com/jbossqe/eap-tests-hornetq.git cd eap-tests-hornetq/scripts/ git checkout refactoring_modules groovy -DEAP_VERSION=7.0.0.ER6 PrepareServers7.groovy export WORKSPACE=$PWD export JBOSS_HOME_1=$WORKSPACE/server1/jboss-eap export JBOSS_HOME_2=$WORKSPACE/server2/jboss-eap export JBOSS_HOME_3=$WORKSPACE/server3/jboss-eap export JBOSS_HOME_4=$WORKSPACE/server4/jboss-eap cd ../jboss-hornetq-testsuite/ mvn clean test -Dtest=CliReloadTest#reloadServerOOMTest -DfailIfNoTests= false -Deap=7x | tee log After the test is started: Start Visualvm and check memory grow of EAP server.

SFDC Cases Counter:
SFDC Cases Open:
SFDC Cases Links:

There is java.lang.OutOfMemoryError: Direct buffer memory if server is continuously reloaded. OOM was hit on Solaris 11 sparc64 with Oracle JDK 8 but other platforms may be affected as well.

There was different java.lang.OutOfMemoryError: GC overhead limit exceeded in the same scenario and it was caused by Artemis - jira ~~JBEAP-3635~~.
This may be caused by it but java.lang.OutOfMemoryError: Direct buffer memory is probably caused by another leak. By checking heap dumps there is increasing number of references to java.nio.DirectByteBuffer. Those seems to be allocated by Undertow. Heap dumps can be downloaded by:

scp jbossqa@10.40.4.81:~/tmp/heaps.zip . #password: jbossqa

Stack trace from OOM:

10:11:50,403 ERROR [io.undertow.request] (default I/O-253) UT005071: Undertow request failed HttpServerExchange{ GET / request {Connection=[Upgrade], httpPpgradeEndpoint=[http-acceptor], Upgrade=[activemq-remoting], Sec-ActiveMQRemoting-Key=[94inegbjaWkqtSG9Z7GzJw==], Host=[localhost]} response {Content-Length=[74], Content-Type=[text/html]}}: java.lang.OutOfMemoryError: Direct buffer memory
        at java.nio.Bits.reserveMemory(Bits.java:658) [rt.jar:1.8.0_71]
        at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:123) [rt.jar:1.8.0_71]
        at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:311) [rt.jar:1.8.0_71]
        at org.xnio.BufferAllocator$2.allocate(BufferAllocator.java:57) [xnio-api-3.3.4.Final-redhat-1.jar:3.3.4.Final-redhat-1]
        at org.xnio.BufferAllocator$2.allocate(BufferAllocator.java:55) [xnio-api-3.3.4.Final-redhat-1.jar:3.3.4.Final-redhat-1]
        at org.xnio.ByteBufferSlicePool.allocate(ByteBufferSlicePool.java:146) [xnio-api-3.3.4.Final-redhat-1.jar:3.3.4.Final-redhat-1]
        at io.undertow.server.XnioByteBufferPool.allocate(XnioByteBufferPool.java:45)
        at io.undertow.io.AsyncSenderImpl.send(AsyncSenderImpl.java:287)
        at io.undertow.io.AsyncSenderImpl.send(AsyncSenderImpl.java:271)
        at io.undertow.io.AsyncSenderImpl.send(AsyncSenderImpl.java:305)
        at io.undertow.server.handlers.error.SimpleErrorPageHandler$1.handleDefaultResponse(SimpleErrorPageHandler.java:70)
        at io.undertow.server.HttpServerExchange.endExchange(HttpServerExchange.java:1546)
        at io.undertow.server.Connectors.executeRootHandler(Connectors.java:218)
        at io.undertow.server.protocol.http.HttpReadListener.handleEventWithNoRunningRequest(HttpReadListener.java:233)
        at io.undertow.server.protocol.http.HttpReadListener.handleEvent(HttpReadListener.java:131)
        at io.undertow.server.protocol.http.HttpReadListener.handleEvent(HttpReadListener.java:57)
        at org.xnio.ChannelListeners.invokeChannelListener(ChannelListeners.java:92) [xnio-api-3.3.4.Final-redhat-1.jar:3.3.4.Final-redhat-1]
        at org.xnio.conduits.ReadReadyHandler$ChannelListenerHandler.readReady(ReadReadyHandler.java:66) [xnio-api-3.3.4.Final-redhat-1.jar:3.3.4.Final-redhat-1]
        at org.xnio.nio.NioSocketConduit.handleReady(NioSocketConduit.java:88) [xnio-nio-3.3.4.Final-redhat-1.jar:3.3.4.Final-redhat-1]
        at org.xnio.nio.WorkerThread.run(WorkerThread.java:559) [xnio-nio-3.3.4.Final-redhat-1.jar:3.3.4.Final-redhat-1]

Issue is under investigation. I'll provide more information.

duplicates

JBEAP-3635 Multiple :reload of server causes "java.lang.OutOfMemoryError: GC overhead limit exceeded"

Closed

relates to

JBEAP-3635 Multiple :reload of server causes "java.lang.OutOfMemoryError: GC overhead limit exceeded"

Closed

Details

Description

Attachments

Issue Links

Easy Agile Planning Poker

Activity

People

Dates