1. Proposed title of this feature request

Add custom labels to existing cluster alerts

2. What is the nature and description of the request?

Customer would like the ability to add labels to existing/default cluster alerts. Currently, the PrometheusRules which control these alerts are readonly and any changes are overwritten by the operator. This makes sense for stability and supportability, and RFEs requesting the option to edit these have been rejected (see: https://issues.redhat.com/browse/RFE-34)

This is reasonable, however we are requesting the possibility of only adding labels (not modifying default labels, nor making any other changes to the rules).

3. Why does the customer need this? (List the business requirements here)

For large scale deployments, with many large clusters, different components/systems are controlled by different technical teams. Allowing their alert systems to use granular labels will allow them to route alerts and issues to the correct team; i.e. componentA on cluster1 goes to one team, componentB on cluster1 goes to another, and componentA on cluster2 goes to yet a third team, etc.

4. List any affected packages or components.

Default PrometheusRules; monitoring operators