Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-1453

TargetDown alert expression is NOT correctly joining kube-state-metrics metric

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Normal Normal
    • None
    • 4.11.z, 4.10.z, 4.8.z
    • Monitoring
    • None
    • Moderate
    • None
    • False
    • Hide

      None

      Show
      None
    • NA

      Description of problem:

      TargetDown alert fired while it shouldn't.
      Prometheus endpoints are not always properly unregistered and the alert will therefore think that some Kube service endpoints are down

      Version-Release number of selected component (if applicable):

      The problem as always been there.

      How reproducible:

      Not reproducible.
      Most of the time Prometheus endpoints are properly unregistered.
      Aim here is to get the TargetDown Prometheus expression be more resilient; this can be tested on past metrics data in which the unregistration issue was encountered.

      Steps to Reproduce:

      N/A
      

      Actual results:

      TargetDown alert triggered while Kube service endpoints are all up & running.

      Expected results:

      TargetDown alert should not have been trigerred.

        1. prometheus-example-app.yaml
          1 kB
        2. targets_down.png
          targets_down.png
          127 kB
        3. targets.png
          targets.png
          115 kB

            spasquie@redhat.com Simon Pasquier
            ngrauss.openshift Nicolas Grauss
            Junqi Zhao Junqi Zhao
            Nicolas Grauss, Tomas Havlicek (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            10 Start watching this issue

              Created:
              Updated:
              Resolved: