Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-1453

TargetDown alert expression is NOT correctly joining kube-state-metrics metric

    XMLWordPrintable

Details

    • Bug
    • Resolution: Done
    • Normal
    • None
    • 4.11.z, 4.10.z, 4.8.z
    • Monitoring
    • None
    • Moderate
    • False
    • Hide

      None

      Show
      None
    • NA

    Description

      Description of problem:

      TargetDown alert fired while it shouldn't.
      Prometheus endpoints are not always properly unregistered and the alert will therefore think that some Kube service endpoints are down

      Version-Release number of selected component (if applicable):

      The problem as always been there.

      How reproducible:

      Not reproducible.
      Most of the time Prometheus endpoints are properly unregistered.
      Aim here is to get the TargetDown Prometheus expression be more resilient; this can be tested on past metrics data in which the unregistration issue was encountered.

      Steps to Reproduce:

      N/A
      

      Actual results:

      TargetDown alert triggered while Kube service endpoints are all up & running.

      Expected results:

      TargetDown alert should not have been trigerred.

      Attachments

        1. prometheus-example-app.yaml
          1 kB
        2. targets_down.png
          targets_down.png
          127 kB
        3. targets.png
          targets.png
          115 kB

        Issue Links

          Activity

            People

              spasquie@redhat.com Simon Pasquier
              ngrauss.openshift Nicolas Grauss
              Junqi Zhao Junqi Zhao
              Nicolas Grauss, Tomas Havlicek
              Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: