Uploaded image for project: 'Red Hat Advanced Cluster Management'
  1. Red Hat Advanced Cluster Management
  2. ACM-7471

MCE failing to uninstall due to submariner-addon finalizer preventing ManagedClusterSet from being removed

XMLWordPrintable

    • Submariner Sprint 2023-11
    • Important
    • No

      Description of problem:

      When the user tries to uninstall ACM, MCE will fail to uninstall due to the `cluster-manager` component failing to be removed:

      status:
        conditions:
          - lastTransitionTime: '2023-09-11T17:05:22Z'
            lastUpdateTime: '2023-09-11T17:05:22Z'
            reason: ComponentsUnavailable
            status: 'False'
            type: Available
          - lastTransitionTime: '2023-09-11T19:53:22Z'
            lastUpdateTime: '2023-09-11T19:53:22Z'
            message: ClusterManager cluster-manager still exists.
            reason: WaitingForResource
            status: 'False'
            type: Progressing

      When diving deeper into the issue, the `cluster-manager` component is failing due to a ManagedClusterSet CR remaining within the cluster:

      E0911 19:49:53.913881 1 base_controller.go:270] "ClusterManagerController" controller failed to sync "cluster-manager", err: Thera are still reaming CRDs: managedclustersets.cluster.open-cluster-management.io

      After checking the CR that is remaining within the cluster, we have determined that the issue is being caused by the `submariner-addon` finalizer preventing the resource from being deleted:

      apiVersion: cluster.open-cluster-management.io/v1beta2
      kind: ManagedClusterSet
      metadata:
        annotations:
          cluster.open-cluster-management.io/submariner-broker-ns: default-broker
        deletionTimestamp: '2023-09-12T13:54:22Z'
        resourceVersion: '332057'
        name: default
        uid: f9e5b8ee-351e-4f41-a5f6-3eb083b49257
        deletionGracePeriodSeconds: 0
        creationTimestamp: '2023-09-12T13:44:21Z'
        finalizers:
          - cluster.open-cluster-management.io/submariner-cleanup
      spec:
        clusterSelector:
          selectorType: ExclusiveClusterSetLabel
      status:
        conditions:
          - lastTransitionTime: '2023-09-12T13:44:26Z'
            message: 1 ManagedClusters selected
            reason: ClustersSelected
            status: 'False'
            type: ClusterSetEmpty 

      From discussions with skitt@redhat.com, the `submariner-addon` finalizer should not be added to the `default` ManagedClusterSet. By removing the finalizer, the uninstall is able to proceed and finish as expected. We need to ensure that the finalizer is not added by the `submariner-addon` to this ManagedClusterSet, since the addon is removed before the finalizer is able to be removed upon uninstall.

      Version-Release number of selected component (if applicable):

      ACM 2.9.0 / MCE 2.4.0

      How reproducible:

      Always

      Steps to Reproduce:

      1. Deploy ACM 2.9.0 build
      2. Uninstall the operator from the Installed Operator page on the OCP console.
      3. Wait until MCE is being uninstalled, MCE will progress to the stage where it is blocked from uninstall due to `cluster-manager`.
      4. Check the `default` ManagedClusterSet and see that the `submariner-addon` finalizer is set and preventing it from being removed.

      Actual results:

      ACM / MCE is prevented from being uninstalled from the cluster environment.

      Expected results:

      Uninstall proceeds without being stalled by ManagedClusterSet resource.

      Additional info:

              tpanteli Thomas Pantelis
              dbennett@redhat.com Disaiah Bennett
              Thuy Nguyen Thuy Nguyen
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

                Created:
                Updated:
                Resolved: