Uploaded image for project: 'JBoss Enterprise Application Platform'
  1. JBoss Enterprise Application Platform
  2. JBEAP-13272

[GSS](7.1.0) Once hitting the max-connections limit, new connections are not accepted any more and CLOSE_WAIT connections increase gradually

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Blocker Blocker
    • 7.1.0.CR3
    • 7.1.0.CR1, 7.1.0.CR2
    • Undertow
    • None
    • Hide
      1. Set max-connections attribute in listener and Start JBoss: 
        <http-listener name="default" socket-binding="http" redirect-socket="https" enable-http2="true" max-connections="10"/>
      2. Sent concurrent requests which can overflow the max-connections limit: 
        ab -c 100 -n 100 http://localhost:8080/

      You will see most requests will be failed with "The timeout specified has expired". Once the above command hit the max-connections limit, JBoss does not accept new connection any more even after all request processing are completed.

      • Try curl command to confirm you can not connect: 
        curl -v http://localhost:8080/
      • Check an output of netstat. CLOSE_WAIT will be increased one by one when you execute curl command and cancel it: 
        netstat -tunapl | grep 8080
      Show
      Set max-connections attribute in listener and Start JBoss: <http-listener name= " default " socket-binding= "http" redirect-socket= "https" enable-http2= " true " max-connections= "10" /> Sent concurrent requests which can overflow the max-connections limit: ab -c 100 -n 100 http: //localhost:8080/ You will see most requests will be failed with "The timeout specified has expired". Once the above command hit the max-connections limit, JBoss does not accept new connection any more even after all request processing are completed. Try curl command to confirm you can not connect: curl -v http: //localhost:8080/ Check an output of netstat . CLOSE_WAIT will be increased one by one when you execute curl command and cancel it: netstat -tunapl | grep 8080

      Once hitting the max-connections limit of listener, JBoss EAP does not accept new connections any more. Even after all request processing are completed, JBoss does not accept. So, any client can not connect to JBoss EAP.

      In addition, when a client send a new request and close the connection from client-side due to timeout or cancellation in such situation, CLOSE_WAIT connections remain until JBoss Java process is stopped.

      I think this issue is related to XNIO-276 and XNIO-279, which both fixes are merged in XNIO 3.4 branch but it's not merged (or partially merged) in XNIO 3.5 branch:

      • XNIO-276 - If QueuedNioTcpServer hits the max connection limit it gets stuck in an infinite loop
        This issue is fixed by the two commits 6a5d04e and 8c0af87 which were merged in 3.4 branch. However, 8c0af87 is not included in 3.5 branch.
      • XNIO-279 - QueuedNioTcpServer can go into an infinite loop if accept fails
        This issue is fixed by the commit 86d0f6c in 3.4 branch. However, this is not included in 3.5 branch.

      Note: JBEAP-6397 and JBEAP-8080 were raised to track the issues for 7.1.0. I think both are marked as resolved because 7.1.0 DR came with XNIO 3.4.x at that time. However, the latest 7.1.0 CR now comes with XNIO 3.5.x, so it regresses in the latest release.

              dlloyd@redhat.com David Lloyd
              rhn-support-mmiura Masafumi Miura
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Created:
                Updated:
                Resolved: