XMLWordPrintable

    • Icon: Feature Feature
    • Resolution: Unresolved
    • Icon: Normal Normal
    • None
    • None
    • PM Monitoring
    • None
    • False
    • None
    • False
    • Not Selected

      Our documentation suggests creating an alert after configuring scrape sample limits.

      That PrometheusRule object has two alerts configured within it [1]

       

      `ApproachingEnforcedSamplesLimit` 

      `TargetDown` 

       

      The `Targetdown` alert is designed to fire after the `ApproachingEnforcedSamplesLimit` because the target is dropped once the enforced sample limit is reached

       

      The TargetDown alert is creating false positives - its firing for reasons other than pods in the namespace have reached there enforced sample limit. 

      This has been discussed with the monitoring team and the suggestion was to raise this RFE to investigate the possible use of the following Prometheus feature flag  [2] in order to improve the quality of this alert 

      [1] - https://docs.openshift.com/container-platform/4.12/monitoring/configuring-the-monitoring-stack.html#creating-scrape-sample-alerts_configuring-the-monitoring-stack 

      [2] - https://prometheus.io/docs/prometheus/latest/feature_flags/#extra-scrape-metrics

              sstout@redhat.com Stephanie Stout
              rh-ee-rfloren Roger Florén
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Created:
                Updated: