-
Bug
-
Resolution: Unresolved
-
Normal
-
None
-
4.12
-
None
-
Moderate
-
No
-
False
-
Description of problem:
Alert PrometheusMissingRuleEvaluations is firing intermittently only for "kube-apiserver.rules" group which is defined in "kube-apiserver-slos-extended" PrometheusRule (in openshift-kube-apiserver project). The alert gets resolved within 15 minutes of firing. Below warning is streamed repetitively every time before the alert fires: ~~~ 2024-02-09T13:34:48.214853072Z ts=2024-02-09T13:34:48.214Z caller=klog.go:108 level=warn component=k8s_client_runtime func=Warningf msg="github.com/prometheus/prometheus/discovery/kubernetes/kubernetes.go:543: watch of *v1.Pod ended with: very short watch: github.com/prometheus/prometheus/discovery/kubernetes/kubernetes.go:543: Unexpected watch close - watch lasted less than a second and no items received" ~~~
Version-Release number of selected component (if applicable):
RHOCP 4.12.45
How reproducible:
NA
Steps to Reproduce:
1. Not sure how to reproduce it 2. 3.
Actual results:
The alert PrometheusMissingRuleEvaluations intermittently for "kube-apiserver.rules" alert group.
Expected results:
The alert should not fire at all.
Additional info:
There are 304 prometheusrules defined in the cluster. There are no resource limits defined on prometheus pods. CPU utilization is around 3vCPU for 1 pod and 2vCPU for other pod. Interestingly, the doesn't stream for any other prometheusrule.