-
Bug
-
Resolution: Unresolved
-
Normal
-
None
-
4.16.0, 4.17.0, 4.18.0, 4.19.0
-
None
-
Quality / Stability / Reliability
-
False
-
-
None
-
Important
-
None
-
None
-
None
-
None
-
Sprint 280
-
1
-
None
-
None
-
None
-
None
-
None
-
None
-
None
Description of problem:
default cluster, there is AlertmanagerReceiversNotConfigured/Watchdog alert fired under openshift-monitoring project, login developer console, select openshift-monitoring project and go to "Observe - Alerts" tab, go to AlertmanagerReceiversNotConfigured details page, "No datapoints found." shows on the graph, see: https://drive.google.com/file/d/1F0xqF_p_MifT8q9QfejLgyTUwcZppFX3/view?usp=drive_link
no such issue for Watchdog alert, see: https://drive.google.com/file/d/1IoOGNbGiEmySTMLiLd4RVFLRMwoJDFDs/view?usp=drive_link
the Watchdog alert expr is vector(1), and AlertmanagerReceiversNotConfigured expr used record rule: cluster:alertmanager_integrations:max
$ oc -n openshift-monitoring get prometheusrules cluster-monitoring-operator-prometheus-rules -oyaml ... - name: general.rules rules: - alert: Watchdog annotations: description: | This is an alert meant to ensure that the entire alerting pipeline is functional. This alert is always firing, therefore it should always be firing in Alertmanager and always fire against a receiver. There are integrations with various notification mechanisms that send a notification when this alert is not firing. For example the "DeadMansSnitch" integration in PagerDuty. summary: An alert that should always be firing to certify that Alertmanager is working properly. expr: vector(1) labels: namespace: openshift-monitoring severity: none ... - alert: AlertmanagerReceiversNotConfigured annotations: description: Alerts are not configured to be sent to a notification system, meaning that you may not be notified in a timely fashion when important failures occur. Check the OpenShift documentation to learn how to configure notifications with Alertmanager. summary: Receivers (notification integrations) are not configured on Alertmanager expr: cluster:alertmanager_integrations:max == 0 for: 10m labels: namespace: openshift-monitoring severity: warning
cluster:alertmanager_integrations:max record rule details:
- expr: max(alertmanager_integrations{namespace="openshift-monitoring"})
record: cluster:alertmanager_integrations:max
Version-Release number of selected component (if applicable):
4.17.0-0.nightly-2024-07-20-191204
How reproducible:
for some alerts
Steps to Reproduce:
1. check alert details page for openshift-monitoring
Actual results:
AlertmanagerReceiversNotConfigured details page, "No datapoints found." shows on the graph
Expected results:
should show the data
Additional info:
