-
Bug
-
Resolution: Done
-
Normal
-
None
-
False
-
False
-
NEW
-
VERIFIED
-
Before this update, alerting rules for the Fluentd collector did not adhere to the {product-title} monitoring style guidelines. This update modifies those alerts to include the namespace label, resolving the issue.
-
-
Log Collection - Sprint 224
Hello,
The OpenShift Monitoring Team has published a set guidelines for
writing alerting rules in OpenShift, including a basic style guide.
You can find these here:
https://github.com/openshift/enhancements/blob/master/enhancements/monitoring/alerting-consistency.md
https://github.com/openshift/enhancements/blob/master/enhancements/monitoring/alerting-consistency.md#style-guide
A subset of these are now being enforced in OpenShift End-to-End
tests [1], with temporary exceptions for existing non-compliant rules.
This component was found to have the following issues:
- Alerts found to not include a namespace label:
- FluentDHighErrorRate
- FluentDVeryHighErrorRate
- FluentdNodeDown
Alerts SHOULD include a namespace label indicating the alert's source.
This requirement originally comes from our SRE team, as they use the
namespace label as the first means of routing alerts. Many alerts
already include a namespace label as a result of the PromQL
expressions used, others may require a static label.
Example of a change to PromQL to include a namespace label:
https://github.com/openshift/cluster-monitoring-operator/commit/52d1f05#diff-9024dcef0fd244c0267c46858da24fbd1f45633515fafae0f98781b20805ff1dL22-R22
Example of adding a static namespace label:
https://github.com/openshift/cluster-monitoring-operator/commit/52d1f05#diff-352702e71122d34a1be04c0588356cd8cb8a10df547f1c3c39fec18fa75b1593R304
If you have questions about how to best to modify your alerting rules
to include a namespace label, please reach out to the OpenShift
Monitoring Team in the #forum-monitoring channel on Slack, or on our
mailing list:
team-monitoring@redhat.com
Thank you!
Repo: openshift/cluster-logging-operator
- links to
- mentioned on