-
Feature
-
Resolution: Unresolved
-
Normal
-
None
-
None
-
None
Our documentation suggests creating an alert after configuring scrape sample limits.
That PrometheusRule object has two alerts configured within it [1]
`ApproachingEnforcedSamplesLimit`
`TargetDown`
The `Targetdown` alert is designed to fire after the `ApproachingEnforcedSamplesLimit` because the target is dropped once the enforced sample limit is reached
The TargetDown alert is creating false positives - its firing for reasons other than pods in the namespace have reached there enforced sample limit.
This has been discussed with the monitoring team and the suggestion was to raise this RFE to investigate the possible use of the following Prometheus feature flag [2] in order to improve the quality of this alert
[2] - https://prometheus.io/docs/prometheus/latest/feature_flags/#extra-scrape-metrics