Uploaded image for project: 'OpenShift Pipelines'
  1. OpenShift Pipelines
  2. SRVKP-10833

[QA] ServiceMonitor has hardcoded openshift-operators in namespaceSelector, causing Prometheus failures when operator is installed in different namespace

XMLWordPrintable

      Description of problem:

      When the OpenShift Pipelines Operator is installed in a namespace other than the default openshift-operators namespace (e.g., openshift-pipelines), the ServiceMonitor's namespaceSelector fails because it has a hardcoded reference to openshift-operators.

      This causes Prometheus to attempt to scrape metrics from the wrong namespace, resulting in permission errors and triggering the PrometheusKubernetesListWatchFailures alert.

      Error observed:

      services is forbidden: User "system:serviceaccount:openshift-monitoring:prometheus-k8s" cannot list resource "services" in API group "" in the namespace "openshift-operators"

       

      Root Cause:

       In config/openshift/base/operator_servicemonitor.yaml:61-63:

          namespaceSelector:
            matchNames:
              - openshift-operators

       The namespaceSelector is hardcoded to openshift-operators. When the operator is installed in a different namespace (e.g., openshift-pipelines), the ServiceMonitor tries to find services in the wrong namespace.

      Note: The RBAC resources (Role and RoleBinding) are working correctly. Only the ServiceMonitor's namespaceSelector needs to be fixed.

      The operator already has a transformer function UpdateServiceMonitorTargetNamespace() at pkg/reconciler/openshift/common/transformer.go:174-186 that is used for
        Pipelines-as-Code to dynamically update the ServiceMonitor namespace selector, but this transformer is NOT being applied to the operator's own ServiceMonitor.

      Prerequisites (if any, like setup, operators/versions):

      Steps to Reproduce

        1. Install OpenShift Pipelines Operator in openshift-pipelines namespace instead of openshift-operators
        2. Check Prometheus targets and logs
        3. Observe errors about missing permissions to list services in openshift-operators namespace
        4. Check for PrometheusKubernetesListWatchFailures alert

      Workaround :
        1) kubectl edit ServiceMonitor openshift-pipelines-operator-monitor -n openshift-pipelines
        2) Update namespaceSelector.matchNames from openshift-operators to openshift-pipelines

      Actual results:

      ServiceMonitor's namespaceSelector continues to reference openshift-operators even when the operator is installed elsewhere, causing Prometheus scrape failures.

      Expected results:

      The ServiceMonitor's namespaceSelector.matchNames should dynamically reference the namespace where the operator is actually installed, not hardcode openshift-operators.

      Reproducibility (Always/Intermittent/Only Once):

      Always

      Acceptance criteria: 

       

      Definition of Done:

      Build Details:

      Additional info (Such as Logs, Screenshots, etc):

       

       *

              jkhelil abdeljawed khelil
              rh-ee-abghosh Abhishek Ghosh
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated: