Uploaded image for project: 'Red Hat Advanced Cluster Management'
  1. Red Hat Advanced Cluster Management
  2. ACM-21001

managed cluster is detached by force but ACM agents are left over

XMLWordPrintable

    • Quality / Stability / Reliability
    • False
    • Hide

      None

      Show
      None
    • False
    • AI-56
    • Critical
    • Yes

      Description of problem:

      I'm trying to create seed image from a managed cluster with 2.14.0-DOWNSTREAM-2025-05-22-03-34-42 and found things act  differently  from last working build for me which is ACM 2.14.0-DOWNSTREAM-2025-05-20-15-53-07.

      1. the managed cluster,  a SNO, is turned down after the SNO is detached.  This breaks my automation that detaches a managed cluster then starts create a seed image from it. 
      2. After I powered on the SNO and trying to create image from it. the seedgenerator reports error"Rejected: Cluster must be detached from hub prior to seed generation" then when I check the pods on the SNO, all the ACM agents are still running  even the SNO is not managed anymore as shown below, the only managed cluster is vm00003 now. SNO vm00001 is not managed anymore. but the ACM agents pods are still running on it. This blocks creating seed image from it.
        # oc get managedcluster
        NAME      HUB ACCEPTED  MANAGED CLUSTER URLS                   JOINED  AVAILABLE  AGE
        local-cluster  true      https://api.acm-gigantic.rdu2.scalelab.redhat.com:6443  True   True    16h
        vm00003     true      https://api.vm00003.rdu2.scalelab.redhat.com:6443    True   True    16h
        ]#  oc --kubeconfig=/root/hv-vm/kc/vm00001/kubeconfig get pod -A
        NAMESPACE                                          NAME                                                              READY   STATUS             RESTARTS        AGE
        assisted-installer                                 assisted-installer-controller-qhr9z                               0/1     Completed          0               15h
        open-cluster-management-addon-observability        endpoint-observability-operator-76bf55db77-frlpm                  1/1     Running            1               14h
        open-cluster-management-addon-observability        metrics-collector-deployment-59d6649c5-pkfr9                      1/1     Running            1               14h
        open-cluster-management-agent-addon                cluster-proxy-proxy-agent-856d58756f-2fx5d                        3/3     Running            3               14h
        open-cluster-management-agent-addon                config-policy-controller-6675bf4c58-q9xzq                         1/1     Running            1               14h
        open-cluster-management-agent-addon                governance-policy-framework-6c866bd4fc-4f49s                      0/1     CrashLoopBackOff   8 (39s ago)     14h
        open-cluster-management-agent-addon                klusterlet-addon-search-8dc59fc7b-9877c                           1/1     Running            1               14h
        open-cluster-management-agent-addon                klusterlet-addon-workmgr-7bcd7b68c5-rkstf                         0/1     CrashLoopBackOff   7 (4m19s ago)   14h
        open-cluster-management-agent-addon                managed-serviceaccount-addon-agent-698d84d646-7hw89               0/1     CrashLoopBackOff   7 (4m17s ago)   14h
        open-cluster-management-agent                      klusterlet-56ff88d56f-2bd4l                                       1/1     Running            1               14h
        open-cluster-management-agent                      klusterlet-agent-66f75f7868-bkm9w                                 1/1     Running            2 (10m ago)     14h

      when I check the mce build changelog from https://gitlab.cee.redhat.com/acm-cicd/mce-bb2/-/raw/mce-2.9/snapshots/2025-05-21-15-22-03/changelog-2025-05-20-15-39-57-2025-05-21-15-22-03.out. seems  there is some change made for "fix crd work is force deleted issue" I'm not sure if the issue is related but  zyin@redhat.com  Could you please take a look first?

      Version-Release number of selected component (if applicable):

      How reproducible:

      Steps to Reproduce:

      1.  
      2.  
      3. ...

      Actual results:

      Expected results:

      Additional info:

              cchun@redhat.com Crystal Chun
              rhn-support-txue Ting Xue
              Vladislav Kolodny Vladislav Kolodny
              ACM QE Team
              Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

                Created:
                Updated:
                Resolved: