Uploaded image for project: 'TorqueBox'
  1. TorqueBox
  2. TORQUE-1126

Requests are hanging when multiple requests are made to different servers (Session Issues)

    Details

    • Type: Feature Request
    • Status: Resolved (View Workflow)
    • Priority: Critical
    • Resolution: Done
    • Affects Version/s: 2.3.0, 2.3.2
    • Fix Version/s: None
    • Component/s: None
    • Affects:
      Compatibility/Configuration

      Description

      We have our requests being load balanced between 4 different servers. When multiple requests are fired to the server, which happens with every request, there is a long wait time as torquebox handles the session sharing. The reason I believe this is related to TORQUEBOXs session handling is because we created a script to test removing the load balancer from the equation.

      https://gist.github.com/mikesnare/37b4984f60ccefca052a

      This script will fire requests simultaneously to multiple web servers. When we pass a Cookie that contains our session the requests get processed by a single server at a time, and the more servers in the pool the longer it takes.

      Example:
      $ ruby lb.rb
      Response from 2 is a 200
      Response from 2 is a 200
      Response from 2 is a 200
      Response from 2 is a 200
      Response from 2 is a 200
      Response from 2 is a 200
      Response from 2 is a 200
      Response from 1 is a 200
      Response from 1 is a 200
      Response from 1 is a 200
      Response from 1 is a 200
      Response from 1 is a 200
      Response from 1 is a 200
      Response from 3 is a 200
      Response from 3 is a 200
      Response from 3 is a 200
      Response from 3 is a 200
      Response from 3 is a 200
      Response from 3 is a 200
      Response from 3 is a 200
      0.440000 0.060000 0.500000 ( 15.844000)

      However, when we do not pass a Cookie with the session and each one is generated on the fly, the requests process quickly and do not block one another.

      Example:
      $ ruby lb.rb
      Response from 1 is a 200
      Response from 2 is a 200
      Response from 3 is a 200
      Response from 2 is a 200
      Response from 1 is a 200
      Response from 1 is a 200
      Response from 2 is a 200
      Response from 1 is a 200
      Response from 2 is a 200
      Response from 3 is a 200
      Response from 1 is a 200
      Response from 2 is a 200
      Response from 1 is a 200
      Response from 2 is a 200
      Response from 3 is a 200
      Response from 3 is a 200
      Response from 2 is a 200
      Response from 3 is a 200
      Response from 3 is a 200
      Response from 3 is a 200
      0.380000 0.040000 0.420000 ( 0.348000)

      Furthermore, regardless which way we do this, the actual time to process the request is very low. By watching our Rails log we are able to see each request being processed and returned in under 45ms. Even when it takes 15 seconds for all the requests to return.

      To reproduce this, set up a small application that uses Infinispan to handle session sharing in a cluster. Then run this script, with modifications for your server, and try out 1 hosts to 4 hosts. You will see as you add hosts the response time decreases. Particularly visible at 3+ hosts.

        Gliffy Diagrams

          Attachments

            Activity

              People

              • Assignee:
                benbrowning Ben Browning
                Reporter:
                altonymous Chris Altman
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: