Uploaded image for project: 'Red Hat Advanced Cluster Management'
  1. Red Hat Advanced Cluster Management
  2. ACM-25366

Addon fails to deploy in QE env because adding the cluster monitoring label on the namespace is denied

XMLWordPrintable

    • None

      QE env:
      https://redhat-internal.slack.com/archives/C0804HR2D41/p1761055135334899

      Code involved:
      https://github.com/stolostron/multicluster-observability-operator/blob/edae0c3d498907fcbad5c72cf9803a7e545dd690/operators/endpointmetrics/controllers/observabilityendpoint/observabilityaddon_controller.go#L467

      logs:

      /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.19.7/pkg/internal/controller/controller.go:263
      sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2
      /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.19.7/pkg/internal/controller/controller.go:224
      2025-10-22T09:04:11.127Z INFO controllers.ObservabilityAddon Reconciling {"Request": "open-cluster-management-addon-observability/observability-metrics-allowlist"}
      2025-10-22T09:04:11.133Z INFO util unable to get CRD with ApiextensionsV1 Client, not found. {"CRD": "hostedclusters.hypershift.openshift.io"}
      2025-10-22T09:04:11.133Z INFO controllers.ObservabilityAddon Adding label: openshift.io/cluster-monitoring to namespace: open-cluster-management-addon-observability
      2025-10-22T09:04:11.146Z ERROR controllers.ObservabilityAddon Failed to update namespace for Endpoint Operator: open-cluster-management-addon-observability with the label: openshift.io/cluster-monitoring {"error": "admission webhook \"namespace-validation.managed.openshift.io\" denied the request: Denied. Err Managed OpenShift customers may only remove the following protected labels from Namespaces: ([openshift.io/cluster-monitoring])"}
      github.com/stolostron/multicluster-observability-operator/operators/endpointmetrics/controllers/observabilityendpoint.(*ObservabilityAddonReconciler).ensureOpenShiftMonitoringLabelAndRole
      /workspace/operators/endpointmetrics/controllers/observabilityendpoint/observabilityaddon_controller.go:469
      github.com/stolostron/multicluster-observability-operator/operators/endpointmetrics/controllers/observabilityendpoint.(*ObservabilityAddonReconciler).Reconcile
      /workspace/operators/endpointmetrics/controllers/observabilityendpoint/observabilityaddon_controller.go:105
      sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile
      /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.19.7/pkg/internal/controller/controller.go:116
      sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler
      /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.19.7/pkg/internal/controller/controller.go:303
      sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem
      /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.19.7/pkg/internal/controller/controller.go:263
      sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2
      /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.19.7/pkg/internal/controller/controller.go:224
      2025-10-22T09:04:11.146Z ERROR Reconciler error {"controller": "observabilityaddon-controller", "controllerGroup": "observability.open-cluster-management.io", "controllerKind": "ObservabilityAddon", "ObservabilityAddon": {"name":"observability-metrics-allowlist","namespace":"open-cluster-management-addon-observability"}, "namespace": "open-cluster-management-addon-observability", "name": "observability-metrics-allowlist", "reconcileID": "05f908d9-eefc-4e22-9672-f340612cf061", "error": "failed to ensure OpenShift monitoring label and role: admission webhook \"namespace-validation.managed.openshift.io\" denied the request: Denied. Err Managed OpenShift customers may only remove the following protected labels from Namespaces: ([openshift.io/cluster-monitoring])"}
      sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler
      /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.19.7/pkg/internal/controller/controller.go:316
      sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem
      /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.19.7/pkg/internal/controller/controller.go:263
      sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2
      /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.19.7/pkg/internal/controller/controller.go:224
      

              rh-ee-coquadro Coleen Iona Quadros
              rh-ee-tmange Thibault Mange
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated: