Goals

1. Enable OpenShift Administrators to create alerting rules based on logs.
2. Enable OpenShift Application owners to create alerting rules based on logs scoped only for applications they have access to.
3. Provide support for notifying on firing alerts in OpenShift Console

Non-Goals

1. Provide support for logs-based metrics that can be used in PrometheusRule custom resources for alerting.

Motivation

Since OpenShift 4.6, application owners can configure alerting rules based on metrics themselves as described in User Workload Monitoring (UWM) enhancement. The rules are defined as PrometheusRule resources and can be based on platform and/or application metrics.

To expand the alerting capabilities on logs as an observability signal, cluster admins and application owners should be able to configure alerting rules as described in the Loki Rules docs and in the Loki Operator Ruler upstream enhancement.

AlertingRule CRD fullfills the requirement to define alerting rules for Loki similar to PrometheusRule.

RulerConfig CRD fullfills the requirement to connect the Loki Ruler component to notify a list of Prometheus AlertManager hosts on firing alerts.

Alternatives

1. Use only the RecordingRule CRD to export logs as metrics first and rely on present cluster-monitoring/user-workload-monitoring alerting capabilities.

Acceptance Criteria

1. OpenShift Administrators receive notifications for application/infrastructure/audit logs-based alerts on the same OpenShift Console Alerts view as with metrics.
2. OpenShift Application owners receive notifications for application logs-based alerts on the same OpenShift Console Alerts view as with metrics.
3. OpenShift Administrators can route logs-based alerts seamless as with metrics-based alerts.

Risk and Assumptions

1. Assuming that the present OpenShift Console implementation for Alerts view is compatible to list and manage alerts from Alertmanager which originate from Loki.
2. Assuming that the present UWM tenancy model applies to the logs-based alerts.

Documentation Considerations

[rkratky] Document stories/procedures based on the three main goals.