Uploaded image for project: 'JBoss Core Services'
  1. JBoss Core Services
  2. JBCS-634

High CPU in mod_cluster with high httpd VirtualHost counts when restarting JBoss instances

XMLWordPrintable

    • 26
    • +
    • Hide

      -Set up standard JBCS/mod_cluster+EAP 7 config
      -Add many individual httpd vhosts. An easy way to generate:

      #!/bin/sh
# Number of vhosts to create
LOOP=10000
for ((i=1; i <= $LOOP; i++))
do
   echo "<VirtualHost *:80>" > host$i.conf
      echo  "   ServerName host"$i".com" >> host$i.conf
   echo "</VirtualHost>" >> host$i.conf
done
      Show
      -Set up standard JBCS/mod_cluster+EAP 7 config -Add many individual httpd vhosts. An easy way to generate: #!/bin/sh # Number of vhosts to create LOOP=10000 for ((i=1; i <= $LOOP; i++)) do echo "<VirtualHost *:80>" > host$i.conf echo " ServerName host" $i ".com" >> host$i.conf echo "</VirtualHost>" >> host$i.conf done

      If a configuration contains many VirtualHosts (100+), there is notable CPU spikes seen in httpd processes when JBoss instances stop/start and cause balancer changes. It looks like there is O(n^2) complexity here where n is the VirtualHost counts.

      add_balancers_workers loops for each VirtualHost and calls create_worker each time. Note that with CreateBalancers 2, the other vhosts always have a balancer once it's populatedi n the main_server, so this check never skips any work for the vhosts:

              if (!balancer && (creat_bal == CREAT_NONE ||
            (creat_bal == CREAT_ROOT && s!=main_server))) {
            s = s->next;
            continue;
        }

      So add_balancers_workers is O( n ). proxy_cluster_watchdog_func and proxy_cluster_child_init call update_workers_node/add_balancers_workers for each vhost as well for n^2 complexity.

              gzaronik@redhat.com George Zaronikas
              rhn-support-aogburn Aaron Ogburn
              Paul Lodge Paul Lodge
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

                Created:
                Updated:
                Resolved: