Uploaded image for project: 'JBoss Enterprise Application Platform'
  1. JBoss Enterprise Application Platform
  2. JBEAP-328

Getting DirectBuffer OOM when sending fragmented binary message to websocket endpoint

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Critical Critical
    • 7.0.0.DR7
    • 7.0.0.DR4
    • Undertow
    • None
    • Hide

      1) Start EAP 7 (use default JAVA_OPTS defined in standalone.conf => JAVA_OPTS="-server -XX:+UseCompressedOops -server -XX:+UseCompressedOops -Xms64m -Xmx512m -Djava.net.preferIPv4Stack=true -Djboss.modules.system.pkgs=org.jboss.byteman -Djava.awt.headless=true"

      2) Deploy simple web application with websocket endpoint which just echo's messages, see [1]

      3) Install autobahn testsuite http://autobahn.ws/testsuite/installation.html#installation

      4) Run test case "9.4.1" (good case to easily reproduce the issue)

      [1]

      @ServerEndpoint("/echoBasicEndpoint")
public class EchoBasicEndpoint {

    Writer writer;
    OutputStream stream;

    @OnMessage
    public void handleMessage(final String message, Session session, boolean last) throws IOException {
        if (writer == null) {
            writer = session.getBasicRemote().getSendWriter();
        }
        writer.write(message);
        if (last) {
            writer.close();
            writer = null;
        }
    }

    @OnMessage
    public void handleMessage(final byte[] message, Session session, boolean last) throws IOException {
        if (stream == null) {
            stream = session.getBasicRemote().getSendStream();
        }
        stream.write(message);
        stream.flush();
        if (last) {
            stream.close();
            stream = null;
        }
    }

    @OnError
    public void onError(Throwable t) {
        System.err.println("onError event occured: " + t.getMessage());
        t.printStackTrace();
    }
      Show
      1) Start EAP 7 (use default JAVA_OPTS defined in standalone.conf => JAVA_OPTS="-server -XX:+UseCompressedOops -server -XX:+UseCompressedOops -Xms64m -Xmx512m -Djava.net.preferIPv4Stack=true -Djboss.modules.system.pkgs=org.jboss.byteman -Djava.awt.headless=true" 2) Deploy simple web application with websocket endpoint which just echo's messages, see [1] 3) Install autobahn testsuite http://autobahn.ws/testsuite/installation.html#installation 4) Run test case "9.4.1" (good case to easily reproduce the issue) [1] @ServerEndpoint( "/echoBasicEndpoint" ) public class EchoBasicEndpoint { Writer writer; OutputStream stream; @OnMessage public void handleMessage( final String message, Session session, boolean last) throws IOException { if (writer == null ) { writer = session.getBasicRemote().getSendWriter(); } writer.write(message); if (last) { writer.close(); writer = null ; } } @OnMessage public void handleMessage( final byte [] message, Session session, boolean last) throws IOException { if (stream == null ) { stream = session.getBasicRemote().getSendStream(); } stream.write(message); stream.flush(); if (last) { stream.close(); stream = null ; } } @OnError public void onError(Throwable t) { System .err.println( "onError event occured: " + t.getMessage()); t.printStackTrace(); }

      When sending fragmented binary message (message with message payload of length 4 * 2**20 (4M). Sent out in fragments of 64). The server throws java.lang.OutOfMemoryError: Direct buffer memory [1]

      The memory for direct buffer by default depends on the size set by -Xmx, which is in EAP 7.0.0.DR4 by default set to -Xmx512m. Increasing it just increases the time before the limit is hit (it is enough to send those messages multiple times to hit the limit again).

      I believe the issue is similar to the one for EAP 6.4: https://bugzilla.redhat.com/show_bug.cgi?id=1223708

      [1]

      15:10:55,463 ERROR [org.xnio.listener] (default I/O-1) XNIO001007: A channel event listener threw an exception: java.lang.OutOfMemoryError: Direct buffer memory
	at java.nio.Bits.reserveMemory(Bits.java:658)
	at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:123)
	at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:311)
	at org.xnio.BufferAllocator$2.allocate(BufferAllocator.java:57)
	at org.xnio.BufferAllocator$2.allocate(BufferAllocator.java:55)
	at org.xnio.ByteBufferSlicePool.allocate(ByteBufferSlicePool.java:143)
	at io.undertow.websockets.core.BufferedBinaryMessage$1.handleEvent(BufferedBinaryMessage.java:106)
	at io.undertow.websockets.core.BufferedBinaryMessage$1.handleEvent(BufferedBinaryMessage.java:97)
	at org.xnio.ChannelListeners.invokeChannelListener(ChannelListeners.java:92)
	at io.undertow.server.protocol.framed.AbstractFramedStreamSourceChannel$1.run(AbstractFramedStreamSourceChannel.java:264)
	at org.xnio.nio.WorkerThread.safeRun(WorkerThread.java:560)
	at org.xnio.nio.WorkerThread.run(WorkerThread.java:462)

        1. all-eap-fuzzing_client_spec.json
          0.3 kB
          Radim Hatlapatka
        2. websocket-endpoints.war
          8 kB
          Radim Hatlapatka

              sdouglas1@redhat.com Stuart Douglas (Inactive)
              rhatlapa@redhat.com Radim Hatlapatka (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Created:
                Updated:
                Resolved: