Uploaded image for project: 'WildFly'
  1. WildFly
  2. WFLY-3146

Undertow Memory Leak for JAX-RS requests

    Details

    • Steps to Reproduce:
      Hide

      Unzip the WildFly.8.0.0.Final distribution.
      Drop in the attached hello-service.war file into standalone-deployments.
      Add an https listener to the undertow service in standalone.xml:

                      <https-listener name="https" socket-binding="https" security-realm="SSLRealm"/>
      

      Add a new SSLRealm security domain to <security-realms> in standalone.xml:

                  <security-realm name="SSLRealm">
                      <server-identities>
                          <ssl>
                              <keystore path="ssl/server.keystore" relative-to="jboss.server.data.dir" keystore-password="abc123" alias="serverkey"/>
                          </ssl>
                      </server-identities>
                  </security-realm>
      

      Mkdir ...standalone/data/ssl and copy the attached server.keystore file into it.
      Start WildFly with standalone.bat
      Attach VisualVM (or your favourite profiler) to the WildFly process
      Run hello-client's a.b.c.HelloClient main method - this uses 10 threads to send parallel requests to the server.

      Show
      Unzip the WildFly.8.0.0.Final distribution. Drop in the attached hello-service.war file into standalone-deployments. Add an https listener to the undertow service in standalone.xml: <https-listener name= "https" socket-binding= "https" security-realm= "SSLRealm" /> Add a new SSLRealm security domain to <security-realms> in standalone.xml: <security-realm name= "SSLRealm" > <server-identities> <ssl> <keystore path= "ssl/server.keystore" relative-to= "jboss.server.data.dir" keystore-password= "abc123" alias= "serverkey" /> </ssl> </server-identities> </security-realm> Mkdir ...standalone/data/ssl and copy the attached server.keystore file into it. Start WildFly with standalone.bat Attach VisualVM (or your favourite profiler) to the WildFly process Run hello-client's a.b.c.HelloClient main method - this uses 10 threads to send parallel requests to the server.

      Description

      After half an hour or so of running the automated test suite for our application, the server hangs due to GC and many OutOfMemoryErrors appear in the WildFly server log.

      I've attached VisualVM and JProfiler, both of which demonstrate that many of the objects that are not being garbage collected are Undertow objects. Both tools show that the path to the GC root is via a Java nio SelectionKey, which is held by sun.nio.ch.WindowsSelectorImpl.

      I'll attach a screenshot of a JProfiler window to demonstrate, but VisualVM shows the same result (albeit presented differently).

      I have also found similar behaviour with a small, trivial example, as described in the Steps to Reproduce. It makes repeated calls to a simple RESTful call to a JAX-RS service. The example doesn't cause as fast a leak as our test suite, but it does demonstrate the problem.

      The example uses https for the call because it seems to leak memory faster than http - but http does also exhibit the problem.

      I have also tried patching WildFly 8.0.0.Final up to Undertow.1.0.2-SNAPSHOT and XNIO.3.3.0-SNAPSHOT (as of this morning), but the problem still occurs.

        Gliffy Diagrams

          Attachments

          1. hello-client.zip
            5 kB
          2. hello-service.zip
            4 kB
          3. hello-service-0.1-SNAPSHOT.war
            4 kB
          4. JProfiler GC Root.png
            JProfiler GC Root.png
            190 kB
          5. server.keystore
            2 kB
          6. standalone.xml
            20 kB
          7. VisualVM Memory.png
            VisualVM Memory.png
            57 kB

            Activity

              People

              • Assignee:
                swd847 Stuart Douglas
                Reporter:
                darrenjones Darren Jones
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: