Uploaded image for project: 'JBoss Core Services'
  1. JBoss Core Services
  2. JBCS-1460

mod_cluster leaks memory with httpd 2.4.53+

    XMLWordPrintable

Details

    Description

      mod_cluster 1.3.16 with httpd 2.4.53 shows a slow build up of memory. This can also be observed in logs (note constant "Created: worker for..."):

      [Mon Apr 04 11:52:32.885887 2022] [:debug] [pid 27:tid 140335321683712] mod_proxy_cluster.c(240): Created: worker for http://172.29.0.4:8080
[Mon Apr 04 11:52:32.885897 2022] [proxy:debug] [pid 27:tid 140335321683712] proxy_util.c(2118): AH00924: worker  shared already initialized
[Mon Apr 04 11:52:32.885905 2022] [proxy:debug] [pid 27:tid 140335321683712] proxy_util.c(2183): AH00927: initializing worker  local
[Mon Apr 04 11:52:33.101789 2022] [:debug] [pid 27:tid 140334809990912] mod_proxy_cluster.c(240): Created: worker for http://172.29.0.3:8080
[Mon Apr 04 11:52:33.101816 2022] [proxy:debug] [pid 27:tid 140334809990912] proxy_util.c(2118): AH00924: worker  shared already initialized
[Mon Apr 04 11:52:33.101837 2022] [proxy:debug] [pid 27:tid 140334809990912] proxy_util.c(2183): AH00927: initializing worker  local
[Mon Apr 04 11:52:33.135431 2022] [:debug] [pid 27:tid 140334826776320] mod_proxy_cluster.c(240): Created: worker for http://172.29.0.5:8080
[Mon Apr 04 11:52:33.135459 2022] [proxy:debug] [pid 27:tid 140334826776320] proxy_util.c(2118): AH00924: worker  shared already initialized
[Mon Apr 04 11:52:33.135480 2022] [proxy:debug] [pid 27:tid 140334826776320] proxy_util.c(2183): AH00927: initializing worker  local
[Mon Apr 04 11:52:42.888737 2022] [:debug] [pid 27:tid 140334818383616] mod_proxy_cluster.c(240): Created: worker for http://172.29.0.4:8080
[Mon Apr 04 11:52:42.888743 2022] [proxy:debug] [pid 27:tid 140334818383616] proxy_util.c(2118): AH00924: worker  shared already initialized
[Mon Apr 04 11:52:42.888748 2022] [proxy:debug] [pid 27:tid 140334818383616] proxy_util.c(2183): AH00927: initializing worker  local
[Mon Apr 04 11:52:43.110318 2022] [:debug] [pid 27:tid 140334793205504] mod_proxy_cluster.c(240): Created: worker for http://172.29.0.3:8080
[Mon Apr 04 11:52:43.110366 2022] [proxy:debug] [pid 27:tid 140334793205504] proxy_util.c(2118): AH00924: worker  shared already initialized
[Mon Apr 04 11:52:43.110402 2022] [proxy:debug] [pid 27:tid 140334793205504] proxy_util.c(2183): AH00927: initializing worker  local
[Mon Apr 04 11:52:43.143600 2022] [:debug] [pid 26:tid 140335103604480] mod_proxy_cluster.c(240): Created: worker for http://172.29.0.5:8080
[Mon Apr 04 11:52:43.143661 2022] [proxy:debug] [pid 26:tid 140335103604480] proxy_util.c(2118): AH00924: worker  shared already initialized
[Mon Apr 04 11:52:43.143685 2022] [proxy:debug] [pid 26:tid 140335103604480] proxy_util.c(2183): AH00927: initializing worker  local
[Mon Apr 04 11:52:52.891695 2022] [:debug] [pid 27:tid 140336186615552] mod_proxy_cluster.c(240): Created: worker for http://172.29.0.4:8080
[Mon Apr 04 11:52:52.891704 2022] [proxy:debug] [pid 27:tid 140336186615552] proxy_util.c(2118): AH00924: worker  shared already initialized
[Mon Apr 04 11:52:52.891710 2022] [proxy:debug] [pid 27:tid 140336186615552] proxy_util.c(2183): AH00927: initializing worker  local
[Mon Apr 04 11:52:53.115061 2022] [:debug] [pid 27:tid 140335976019712] mod_proxy_cluster.c(240): Created: worker for http://172.29.0.3:8080
[Mon Apr 04 11:52:53.115077 2022] [proxy:debug] [pid 27:tid 140335976019712] proxy_util.c(2118): AH00924: worker  shared already initialized
[Mon Apr 04 11:52:53.115089 2022] [proxy:debug] [pid 27:tid 140335976019712] proxy_util.c(2183): AH00927: initializing worker  local
[Mon Apr 04 11:52:53.150788 2022] [:debug] [pid 27:tid 140336203417344] mod_proxy_cluster.c(240): Created: worker for http://172.29.0.5:8080
[Mon Apr 04 11:52:53.150807 2022] [proxy:debug] [pid 27:tid 140336203417344] proxy_util.c(2118): AH00924: worker  shared already initialized
[Mon Apr 04 11:52:53.150822 2022] [proxy:debug] [pid 27:tid 140336203417344] proxy_util.c(2183): AH00927: initializing worker  local
[Mon Apr 04 11:53:02.894837 2022] [:debug] [pid 26:tid 140335078426368] mod_proxy_cluster.c(240): Created: worker for http://172.29.0.4:8080
[Mon Apr 04 11:53:02.894847 2022] [proxy:debug] [pid 26:tid 140335078426368] proxy_util.c(2118): AH00924: worker  shared already initialized
[Mon Apr 04 11:53:02.894855 2022] [proxy:debug] [pid 26:tid 140335078426368] proxy_util.c(2183): AH00927: initializing worker  local
[Mon Apr 04 11:53:03.121473 2022] [:debug] [pid 27:tid 140335942448896] mod_proxy_cluster.c(240): Created: worker for http://172.29.0.3:8080
[Mon Apr 04 11:53:03.121504 2022] [proxy:debug] [pid 27:tid 140335942448896] proxy_util.c(2118): AH00924: worker  shared already initialized
[Mon Apr 04 11:53:03.121526 2022] [proxy:debug] [pid 27:tid 140335942448896] proxy_util.c(2183): AH00927: initializing worker  local
[Mon Apr 04 11:53:03.159392 2022] [:debug] [pid 27:tid 140335372039936] mod_proxy_cluster.c(240): Created: worker for http://172.29.0.5:8080
[Mon Apr 04 11:53:03.159422 2022] [proxy:debug] [pid 27:tid 140335372039936] proxy_util.c(2118): AH00924: worker  shared already initialized
[Mon Apr 04 11:53:03.159445 2022] [proxy:debug] [pid 27:tid 140335372039936] proxy_util.c(2183): AH00927: initializing worker  local

      This is due to httpd 2.4.53 adding field name_ex to worker and mod_cluster_proxy does not copy this field. Apache function ap_proxy_get_worker now uses name_ex to find worker. See: https://github.com/apache/httpd/blob/2.4.53/modules/proxy/proxy_util.c#L1790

      This seems to be solved by adding the following line in mod_proxy_cluster.c lines 316 - 352: strncpy(worker->s->name_ex, shared->name_ex, sizeof(worker->s->name_ex));

      There may be other concerns to consider from this change.

      Attachments

        Issue Links

          blocks

          Feature - Capability or a well-defined set of functionality that delivers business value. Features can include additions or changes to existing functionality. Features can easily span multiple teams, and potentially multiple releases. JBCS-1691 Release Notes for 2.4.57 SP3

          • Blocker - To be worked above all other priorities. This term is chosen when the severity of the issue is very high, has no workaround, or the effort for the change is comparatively low. Issues that may be very publicly visible and could generate significant media attention may also drive a higher priority.
          • Closed
          clones

          Bug - A problem that impairs or prevents the functions of the product. MODCLUSTER-761 mod_cluster leaks memory with httpd 2.4.53+

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Resolved
          links to

          Web Link RHBZ 2158845

          mentioned on

          GitLab Merge request - Rebase to upstream 1.3.18.Final tag. Resolves: rhbz#2158845

          GitLab Merge request - Rebase to upstream 1.3.18.Final tag. Resolves: rhbz#2158845

          Activity

            People

              rhn-engineering-jclere Jean-Frederic Clere
              rhn-support-aogburn Aaron Ogburn
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: