Uploaded image for project: 'Infinispan'
  1. Infinispan
  2. ISPN-12607

Metrics degrade cluster performance

    XMLWordPrintable

Details

    Description

      The `/metrics` endpoint exposes Infinispan stats so that they can be periodically scraped by monitoring tools such as Prometheus. However, these stats include calls to `size` which does not perform well as the number of entries in a cache increase. Consequently, if deploying DG in a k8 environment with Prometheus monitoring the DG cluster performance rapidly declines as the number of entries in a cache increases due to time /resources spent iterating the cache container.

      This problem is only exasperated when muiltiple caches exist with many entries exist.

      Attachments

        Issue Links

          clones

          Bug - A problem that impairs or prevents the functions of the product. JDG-4270 Metrics degrade cluster performance

          • Critical - To be worked on immediately following BLOCKER issues.
          • Closed
          is related to

          Enhancement - An enhancement to or refactoring of existing functionality that is not configurable by an end user (typically a change made by an internal team that affects users) ISPN-13709 Report approximate size when accurate size is disabled

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Code Review

          Enhancement - An enhancement to or refactoring of existing functionality that is not configurable by an end user (typically a change made by an internal team that affects users) ISPN-13378 Default to not calculating cache sizes in metrics

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Closed

          Enhancement - An enhancement to or refactoring of existing functionality that is not configurable by an end user (typically a change made by an internal team that affects users) ISPN-13421 Add approximate size metrics

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Closed
          relates to

          Bug - A problem that impairs or prevents the functions of the product. ISPN-12834 Metrics degrade cluster performance when RocksDB is used

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • New

          Activity

            People

              dberinde@redhat.com Dan Berindei (Inactive)
              remerson@redhat.com Ryan Emerson
              Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: