Uploaded image for project: 'JBoss Enterprise Application Platform'
  1. JBoss Enterprise Application Platform
  2. JBEAP-13272

[GSS](7.1.0) Once hitting the max-connections limit, new connections are not accepted any more and CLOSE_WAIT connections increase gradually

    XMLWordPrintable

Details

    • Bug
    • Resolution: Done
    • Blocker
    • 7.1.0.CR3
    • 7.1.0.CR1, 7.1.0.CR2
    • Undertow
    • None
    • Hide
      1. Set max-connections attribute in listener and Start JBoss: 
        <http-listener name="default" socket-binding="http" redirect-socket="https" enable-http2="true" max-connections="10"/>
      2. Sent concurrent requests which can overflow the max-connections limit: 
        ab -c 100 -n 100 http://localhost:8080/

      You will see most requests will be failed with "The timeout specified has expired". Once the above command hit the max-connections limit, JBoss does not accept new connection any more even after all request processing are completed.

      • Try curl command to confirm you can not connect: 
        curl -v http://localhost:8080/
      • Check an output of netstat. CLOSE_WAIT will be increased one by one when you execute curl command and cancel it: 
        netstat -tunapl | grep 8080
      Show
      Set max-connections attribute in listener and Start JBoss: <http-listener name= " default " socket-binding= "http" redirect-socket= "https" enable-http2= " true " max-connections= "10" /> Sent concurrent requests which can overflow the max-connections limit: ab -c 100 -n 100 http: //localhost:8080/ You will see most requests will be failed with "The timeout specified has expired". Once the above command hit the max-connections limit, JBoss does not accept new connection any more even after all request processing are completed. Try curl command to confirm you can not connect: curl -v http: //localhost:8080/ Check an output of netstat . CLOSE_WAIT will be increased one by one when you execute curl command and cancel it: netstat -tunapl | grep 8080

    Description

      Once hitting the max-connections limit of listener, JBoss EAP does not accept new connections any more. Even after all request processing are completed, JBoss does not accept. So, any client can not connect to JBoss EAP.

      In addition, when a client send a new request and close the connection from client-side due to timeout or cancellation in such situation, CLOSE_WAIT connections remain until JBoss Java process is stopped.

      I think this issue is related to XNIO-276 and XNIO-279, which both fixes are merged in XNIO 3.4 branch but it's not merged (or partially merged) in XNIO 3.5 branch:

      • XNIO-276 - If QueuedNioTcpServer hits the max connection limit it gets stuck in an infinite loop
        This issue is fixed by the two commits 6a5d04e and 8c0af87 which were merged in 3.4 branch. However, 8c0af87 is not included in 3.5 branch.
      • XNIO-279 - QueuedNioTcpServer can go into an infinite loop if accept fails
        This issue is fixed by the commit 86d0f6c in 3.4 branch. However, this is not included in 3.5 branch.

      Note: JBEAP-6397 and JBEAP-8080 were raised to track the issues for 7.1.0. I think both are marked as resolved because 7.1.0 DR came with XNIO 3.4.x at that time. However, the latest 7.1.0 CR now comes with XNIO 3.5.x, so it regresses in the latest release.

      Attachments

        Issue Links

          is cloned by

          Bug - A problem that impairs or prevents the functions of the product. WFLY-9397 Once hitting the max-connections limit, new connections are not accepted any more and CLOSE_WAIT connections increase gradually

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Closed
          is incorporated by

          Component Upgrade - A task tracking an update to a bundled component or revision of component upstream of the project. JBEAP-13285 Upgrade XNIO to 3.5.2.Final

          • Blocker - To be worked above all other priorities. This term is chosen when the severity of the issue is very high, has no workaround, or the effort for the change is comparatively low. Issues that may be very publicly visible and could generate significant media attention may also drive a higher priority.
          • Verified
          is related to

          Bug - A problem that impairs or prevents the functions of the product. XNIO-276 If QueuedNioTcpServer hits the max connection limit it gets stuck in an infinite loop

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Resolved

          Bug - A problem that impairs or prevents the functions of the product. XNIO-279 QueuedNioTcpServer can go into an infinite loop if accept fails

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Resolved

          Activity

            People

              dlloyd@redhat.com David Lloyd
              rhn-support-mmiura Masafumi Miura
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: