-
Bug
-
Resolution: Unresolved
-
Major
-
Pipelines 1.19.0, Pipelines 1.20.0, Pipelines 1.21.0
-
2
-
False
-
-
False
-
-
Description of problem:
When the OpenShift Pipelines Operator is installed in a namespace other than the default openshift-operators namespace (e.g., openshift-pipelines), the ServiceMonitor's namespaceSelector fails because it has a hardcoded reference to openshift-operators.
This causes Prometheus to attempt to scrape metrics from the wrong namespace, resulting in permission errors and triggering the PrometheusKubernetesListWatchFailures alert.
Error observed:
services is forbidden: User "system:serviceaccount:openshift-monitoring:prometheus-k8s" cannot list resource "services" in API group "" in the namespace "openshift-operators"
Root Cause:
In config/openshift/base/operator_servicemonitor.yaml:61-63:
namespaceSelector: matchNames: - openshift-operators
The namespaceSelector is hardcoded to openshift-operators. When the operator is installed in a different namespace (e.g., openshift-pipelines), the ServiceMonitor tries to find services in the wrong namespace.
Note: The RBAC resources (Role and RoleBinding) are working correctly. Only the ServiceMonitor's namespaceSelector needs to be fixed.
The operator already has a transformer function UpdateServiceMonitorTargetNamespace() at pkg/reconciler/openshift/common/transformer.go:174-186 that is used for
Pipelines-as-Code to dynamically update the ServiceMonitor namespace selector, but this transformer is NOT being applied to the operator's own ServiceMonitor.
Prerequisites (if any, like setup, operators/versions):
Steps to Reproduce
1. Install OpenShift Pipelines Operator in openshift-pipelines namespace instead of openshift-operators
2. Check Prometheus targets and logs
3. Observe errors about missing permissions to list services in openshift-operators namespace
4. Check for PrometheusKubernetesListWatchFailures alert
Workaround :
1) kubectl edit ServiceMonitor openshift-pipelines-operator-monitor -n openshift-pipelines
2) Update namespaceSelector.matchNames from openshift-operators to openshift-pipelines
Actual results:
ServiceMonitor's namespaceSelector continues to reference openshift-operators even when the operator is installed elsewhere, causing Prometheus scrape failures.
Expected results:
The ServiceMonitor's namespaceSelector.matchNames should dynamically reference the namespace where the operator is actually installed, not hardcode openshift-operators.
Reproducibility (Always/Intermittent/Only Once):
Always
Acceptance criteria:
Definition of Done:
Build Details:
Additional info (Such as Logs, Screenshots, etc):
*
- clones
-
SRVKP-10509 ServiceMonitor has hardcoded openshift-operators in namespaceSelector, causing Prometheus failures when operator is installed in different namespace
-
- Verified
-