Uploaded image for project: 'WildFly'
  1. WildFly
  2. WFLY-12167

Memory leak in metrics in standalone-ha configuration

    XMLWordPrintable

Details

    • Hide

      1. start pristine WildFly 16 with -c=standalone-ha.xml
      2. send it a bit more than 10.000 requests to /metrics (equivalent to about 1 day of runtime while being scraped by 2 prometheus instances in default 15s intervals)
      3. observe increase heap memory usage and ultimately OOM

      Show
      1. start pristine WildFly 16 with -c=standalone-ha.xml 2. send it a bit more than 10.000 requests to /metrics (equivalent to about 1 day of runtime while being scraped by 2 prometheus instances in default 15s intervals) 3. observe increase heap memory usage and ultimately OOM

    Description

      When started in standalone HA configuration every request to the recently added metrics endpoint (<management-if>:9990/metrics) lead to an increase in memory consumption until the JVM is slowed down significantly by GC to a point where even the requests to /health fail within a reasonable timeout (2s) and untlimately lead to OOM.
      The same issue does not occur when WildFly is started in the default standalone configuration (non HA).
      I can provide a (compressed) heap dump if required.

      Attachments

        Issue Links

          is caused by

          Bug - A problem that impairs or prevents the functions of the product. MSC-245 ServiceContainerImpl.registry is leaking memory resources

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Resolved
          relates to

          Component Upgrade - A task tracking an update to a bundled component or revision of component upstream of the project. WFCORE-4509 Upgrade JBoss MSC to 1.4.7.Final

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Resolved

          Component Upgrade - A task tracking an update to a bundled component or revision of component upstream of the project. WFCORE-4663 Upgrade JBoss MSC to 1.4.11.Final

          • Blocker - To be worked above all other priorities. This term is chosen when the severity of the issue is very high, has no workaround, or the effort for the change is comparatively low. Issues that may be very publicly visible and could generate significant media attention may also drive a higher priority.
          • Closed

          Component Upgrade - A task tracking an update to a bundled component or revision of component upstream of the project. WFCORE-4660 Upgrade JBoss MSC to 1.4.10.Final

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Closed

          Activity

            People

              ropalka Richard Opalka
              bsx303 Bernd Stolle (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: