Uploaded image for project: 'Infinispan'
  1. Infinispan
  2. ISPN-12607

Metrics degrade cluster performance

XMLWordPrintable

      The `/metrics` endpoint exposes Infinispan stats so that they can be periodically scraped by monitoring tools such as Prometheus. However, these stats include calls to `size` which does not perform well as the number of entries in a cache increase. Consequently, if deploying DG in a k8 environment with Prometheus monitoring the DG cluster performance rapidly declines as the number of entries in a cache increases due to time /resources spent iterating the cache container.

      This problem is only exasperated when muiltiple caches exist with many entries exist.

        1. high-cpu-tdump_matric_6m_in_singlecache.out
          247 kB
        2. high-cpu-tdump_matrics1m_in_6caches.out
          703 kB
        3. high-cpu-tdump_vendor_6m_in_singlecache.out
          246 kB
        4. high-cpu-tdump_vendor1m_in_6caches.out
          779 kB

              dberinde@redhat.com Dan Berindei (Inactive)
              remerson@redhat.com Ryan Emerson
              Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

                Created:
                Updated:
                Resolved: