-
Story
-
Resolution: Done
-
Critical
-
None
-
None
-
False
-
None
-
False
-
NEW
-
NEW
-
MON Sprint 227
Right now, MonitoringStack child components (Prometheus and Alertmanager) only watch for monitoring resources (monitors, rules and alertmanager configs) in their own namespace [1]. But new use cases (like HyperShift, see below) need a way to do cross-namespace monitoring. Leveraging the existing API of the Prometheus and Alertmanager CRDs, the MonitoringStack CRD could be extended like this:
apiVersion: monitoring.rhobs/v1alpha1
kind: MonitoringStack
metadata:
name: sample-monitoring-stack
spec:
...
resourceNamespaceSelector:
matchExpressions: [...]
matchLabels: {...}
The resourceNamespaceSelector field would be propagated to the Prometheus and Alertmanager resources:
apiVersion: monitoring.rhobs/v1 kind: Prometheus spec: ... podMonitorNamespaceSelector: <.spec.resourceNamespaceSelector from MonitoringStack> serviceMonitorNamespaceSelector: <.spec.resourceNamespaceSelector from MonitoringStack> prometheusRuleNamespaceSelector: <.spec.resourceNamespaceSelector from MonitoringStack>
apiVersion: monitoring.rhobs/v1 kind: Alertmanager spec: ... alertmanagerConfigNamespaceSelector: <.spec.resourceNamespaceSelector from MonitoringStack>
For HyperShift, the plan is to have one MonitoringStack resource that scrapes metrics from many hosted control planes (HCP), each running in a different namespace. Currently every HCP namespace has a "hypershift.openshift.io/hosted-control-plane" label [2] denoting that it's a namespace running control plane components.
apiVersion: monitoring.rhobs/v1alpha1
kind: MonitoringStack
metadata:
name: hcp-monitoring-stack
spec:
resourceSelector: {}
resourceNamespaceSelector:
matchExpressions:
- key: hypershift.openshift.io/hosted-control-plane
operator: Exists
[1] https://github.com/rhobs/observability-operator/blob/b6608492e3c4bc0838f931a59661a7b9fabdaf7e/pkg/controllers/monitoring/monitoring-stack/components.go#L138-L141
[2] https://github.com/openshift/hypershift/blob/5bc27175ad73d48dc2d2be213f0333e9529ed796/hypershift-operator/controllers/hostedcluster/hostedcluster_controller.go#L792-L800
