Uploaded image for project: 'RESTEasy'
  1. RESTEasy
  2. RESTEASY-2020

UnpooledHeapByteBuf objects built up in case of huge files returned concurrently with resteasy-netty4

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Major Major
    • None
    • 3.1.4.Final, 3.6.2.Final
    • Netty integration

      \We have implemented REST File Server using resteasy-netty4 library.
      We have provided 4 GB heap to the server.
      When we are trying to execute 60 concurrent file download REST calls of 250 MB size each, then we are getting OOM - GC Overheap limit exceeded.

      The heap dump shows byte[] held up by UnpooledHeapByteBuf objects in WriteAndFlushTask.
      These objects are created from ChunkOutputStream.flush() method with buffer.copy() and not getting easily cleanedup and causing this issue.

      We need to support atleast 500 concurrent users for file download API.
      Please provide or suggest a way to avoid built-up of UnpooledHeapByteBuf objects.
      Following is the JAX-RS resource code.

      			StreamingOutput output = new StreamingOutput() {
      				@Override
      				public void write(OutputStream out) throws IOException {
      					byte[] chunkData = new byte[1024]; /* 1 KB Buffer */
      					int loopCount = 0;
      					ByteBuffer buffer = filebuffer.asReadOnlyBuffer();
      					while (buffer.hasRemaining()) {
      						int remaining = chunkData.length;
      						if (buffer.remaining() < remaining)
      							remaining = buffer.remaining();
      						buffer.get(chunkData, 0, remaining);
      						out.write(chunkData, 0, remaining);
      						try {
      							if (loopCount == 0) {
      								Thread.sleep(50);
      								loopCount = 0;
      							}
      						} catch (InterruptedException e) {
      							// 
      						}
      						loopCount++;
      					}
      					buffer.clear();
      				}
      			};
      
      			ResponseBuilder response = Response.ok(output);
      			response.header("Content-Disposition", "attachment; packageName=" + packageFileName);
      			response.header("Content-Length", new File(packageFileName).length());
      			response.header("Content-Type", "application/octet-stream");
      			return response.build();
      

              rhn-support-asoldano Alessio Soldano
              rohan26may Rohan Kulkarni (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Created:
                Updated: