Loading...

XML

Word

Printable

Type: Bug
Resolution: Done
Priority: Critical
Fix Version/s: ACM 2.7.10
Affects Version/s: ACM 2.7.0
Component/s: Business Continuity, Server Foundation
Labels:
None

Blocked:
False
Blocked Reason:
None
Ready:
False
Intelligence Requested:
Market:

Severity:
Important

Regression:
No

SFDC Cases Links:
SFDC Cases Counter:
SFDC Cases Open:

Found in ACM: 2.8.1-DOWNSTREAM-2023-07-31-20-24-40 by the ODF team

ODF has a namespace called openshift-storage
After restoring a backup, this namespace gets deleted from the managed cluster

It seems that there is a managedserviceaccount involved, and somehow there was a manifestwork that showed that it included the openshift-storage namespace

The namespace got deleted because the AppliedManifestWork for it is orphaned. So it got evicted as shown in the AppliedManifestWork itself

apiVersion: work.open-cluster-management.io/v1
kind: AppliedManifestWork
metadata:
  creationTimestamp: "2023-08-01T11:02:39Z"
  deletionGracePeriodSeconds: 0
  deletionTimestamp: "2023-08-01T13:37:59Z"
  finalizers:
  - cluster.open-cluster-management.io/applied-manifest-work-cleanup
  generation: 2
  name: 9010a3f0bd7289444324d69b7aa45ec6014412f0f3719af587d819b4d0c42e40-addon-managed-serviceaccount-deploy-0
  resourceVersion: "3630732"
  uid: ae434785-5373-4c66-b502-f9cc02b316a0
spec:
  agentID: 0ce2d9d0-7b84-4004-95d4-f772de964e95
  hubHash: 9010a3f0bd7289444324d69b7aa45ec6014412f0f3719af587d819b4d0c42e40
  manifestWorkName: addon-managed-serviceaccount-deploy-0
status:
  appliedResources:
  - group: ""
    name: openshift-storage
    namespace: ""
    resource: namespaces
    uid: ab15dc0b-ebbd-43d8-97ca-c2a1eebf17e9
    version: v1
  evictionStartTime: "2023-08-01T12:37:59Z"

There are two managedserviceaccount for the new active cluster

oc get managedclusteraddon -A | grep serviceaccount
pbyregow-clu1   managed-serviceaccount        Unknown                
pbyregow-clu2   managed-serviceaccount        Unknown

Both showing as not available

oc -n pbyregow-clu2 get managedclusteraddon
NAME                          AVAILABLE   DEGRADED   PROGRESSING
application-manager           Unknown                
cert-policy-controller        Unknown                
cluster-proxy                 Unknown                
config-policy-controller      Unknown                
governance-policy-framework   Unknown                
iam-policy-controller         Unknown                
maintenance                   Unknown                
managed-serviceaccount        Unknown                
search-collector              Unknown                
tokenexchange                 Unknown                
volsync                       Unknown                
work-manager                  Unknown

apiVersion: addon.open-cluster-management.io/v1alpha1
  kind: ManagedClusterAddOn
  metadata:
    annotations:
      multicluster.openshift.io/mode: sync
    creationTimestamp: "2023-07-28T13:45:29Z"
    generation: 1
    name: maintenance
    namespace: pbyregow-clu2
    ownerReferences:
    - apiVersion: ramendr.openshift.io/v1alpha1
      kind: DRPolicy
      name: odr-policy-5m
      uid: d6c1d6d4-7f9c-46b9-9e0c-cc91040ea369
    resourceVersion: "6131799"
    uid: 6ed1316a-361b-4fa8-8620-bf2069ed5d5e
  spec:
    installNamespace: openshift-storage
  status:
    conditions:
    - lastTransitionTime: "2023-07-28T13:45:29Z"
      message: manifests of addon are applied successfully
      reason: AddonManifestApplied
      status: "True"
      type: ManifestApplied
    - lastTransitionTime: "2023-07-28T13:45:30Z"
      message: Registration of the addon agent is configured
      reason: RegistrationConfigured
      status: "True"
      type: RegistrationApplied
    - lastTransitionTime: "2023-07-28T13:45:30Z"
      message: client certificate rotated starting from 2023-07-29 10:15:30 +0000
        UTC to 2023-08-28 06:41:04 +0000 UTC
      reason: ClientCertificateUpdated
      status: "True"
      type: ClusterCertificateRotated
    - lastTransitionTime: "2023-08-09T12:22:30Z"
      message: Registration agent stopped updating its lease.






- apiVersion: addon.open-cluster-management.io/v1alpha1
  kind: ManagedClusterAddOn
  metadata:
    creationTimestamp: "2023-08-01T11:02:39Z"
    generation: 1
    labels:
      authentication.open-cluster-management.io/is-managed-serviceaccount: auto-import-account
    name: managed-serviceaccount
    namespace: pbyregow-clu2
    ownerReferences:
    - apiVersion: addon.open-cluster-management.io/v1alpha1
      blockOwnerDeletion: true
      controller: true
      kind: ClusterManagementAddOn
      name: managed-serviceaccount
      uid: 670264a4-6f67-4f03-81de-883062215e23
    resourceVersion: "6133662"
    uid: df6c8d46-08ae-4d06-bd1a-804e0788a685
  spec:
    installNamespace: openshift-storage
  status:
    addOnConfiguration: {}
    addOnMeta: {}
    conditions:
    - lastTransitionTime: "2023-08-09T12:22:48Z"
      message: manifests of addon are applied successfully
      reason: AddonManifestApplied
      status: "True"
      type: ManifestApplied
    - lastTransitionTime: "2023-08-01T11:02:39Z"
      message: Registration of the addon agent is configured
      reason: RegistrationConfigured
      status: "True"
      type: RegistrationApplied
    - lastTransitionTime: "2023-08-01T11:02:39Z"
      message: client certificate rotated starting from 2023-08-01 10:57:39 +0000
        UTC to 2023-08-28 06:41:04 +0000 UTC
      reason: ClientCertificateUpdated
      status: "True"
      type: ClusterCertificateRotated







- apiVersion: addon.open-cluster-management.io/v1alpha1
  kind: ManagedClusterAddOn
  metadata:
    annotations:
      multicluster.openshift.io/mode: sync
    creationTimestamp: "2023-07-28T13:34:30Z"
    generation: 1
    name: tokenexchange
    namespace: pbyregow-clu2
    ownerReferences:
    - apiVersion: multicluster.odf.openshift.io/v1alpha1
      kind: MirrorPeer
      name: mirrorpeer-sample
      uid: 163bee3c-5fe2-46cc-b7c2-75b24d68081a
    resourceVersion: "6131819"
    uid: 13087d88-ff57-4b6f-8dcf-1eb4b8fa3dcf
  spec:
    installNamespace: openshift-storage
  status:
    conditions:
    - lastTransitionTime: "2023-07-28T13:34:31Z"
      message: manifests of addon are applied successfully
      reason: AddonManifestApplied
      status: "True"
      type: ManifestApplied
    - lastTransitionTime: "2023-07-28T13:34:32Z"
      message: Registration of the addon agent is configured
      reason: RegistrationConfigured
      status: "True"
      type: RegistrationApplied
    - lastTransitionTime: "2023-07-28T13:34:32Z"
      message: client certificate rotated starting from 2023-07-29 10:09:32 +0000
        UTC to 2023-08-28 06:41:04 +0000 UTC
      reason: ClientCertificateUpdated
      status: "True"
      type: ClusterCertificateRotated
    - lastTransitionTime: "2023-08-09T12:22:30Z"
      message: Registration agent stopped updating its lease.

From the failed hub, here is the managed-serviceaccount ManifestWork for cluster1

oc get manifestwork -n pbyregow-clu1   addon-managed-serviceaccount-deploy-0 -o yaml     
apiVersion: work.open-cluster-management.io/v1
kind: ManifestWork
metadata:
  creationTimestamp: "2023-08-01T11:02:39Z"
  finalizers:
  - cluster.open-cluster-management.io/manifest-work-cleanup
  generation: 2
  labels:
    open-cluster-management.io/addon-name: managed-serviceaccount
  name: addon-managed-serviceaccount-deploy-0
  namespace: pbyregow-clu1
  ownerReferences:
  - apiVersion: addon.open-cluster-management.io/v1alpha1
    blockOwnerDeletion: true
    controller: true
    kind: ManagedClusterAddOn
    name: managed-serviceaccount
    uid: ec1082a7-abbe-4374-84d4-a29ef5d2fbd9
  resourceVersion: "6044862"
  uid: 3bc46a43-9c54-4a3d-a8b5-5dacb379dec2
spec:
  workload:
    manifests:
    - apiVersion: apps/v1
      kind: Deployment
      metadata:
        name: managed-serviceaccount-addon-agent
        namespace: open-cluster-management-agent-addon
      spec:
        replicas: 1
        selector:
          matchLabels:
            addon-agent: managed-serviceaccount
        strategy: {}
        template:
          metadata:
            labels:
              addon-agent: managed-serviceaccount
          spec:
            containers:
            - args:
              - --leader-elect=true
              - --cluster-name=pbyregow-clu1
              - --kubeconfig=/etc/hub/kubeconfig
              command:
              - /agent
              image: registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel8@sha256:db5a523a3cfbe4d3099766475fa094854cfa87337d32c23094ee1363baa5e3d3
              imagePullPolicy: IfNotPresent
              livenessProbe:
                httpGet:
                  path: /healthz
                  port: 8000
                initialDelaySeconds: 2
                periodSeconds: 10
              name: addon-agent
              resources: {}
              volumeMounts:
              - mountPath: /etc/hub/
                name: hub-kubeconfig
                readOnly: true
            serviceAccount: managed-serviceaccount
            volumes:
            - name: hub-kubeconfig
              secret:
                secretName: managed-serviceaccount-hub-kubeconfig
      status: {}
    - apiVersion: rbac.authorization.k8s.io/v1
      kind: Role
      metadata:
        name: open-cluster-management:managed-serviceaccount:addon-agent
        namespace: open-cluster-management-agent-addon
      rules:
      - apiGroups:
        - ""
        resources:
        - configmaps
        verbs:
        - get
        - create
        - update
        - patch
      - apiGroups:
        - ""
        resources:
        - serviceaccounts
        - serviceaccounts/token
        verbs:
        - get
        - watch
        - list
        - create
        - delete
      - apiGroups:
        - coordination.k8s.io
        resources:
        - leases
        verbs:
        - get
        - create
        - update
        - patch
      - apiGroups:
        - authentication.k8s.io
        resources:
        - tokenrequests
        verbs:
        - get
        - create
        - update
        - patch
    - apiVersion: rbac.authorization.k8s.io/v1
      kind: RoleBinding
      metadata:
        name: open-cluster-management:managed-serviceaccount:addon-agent
        namespace: open-cluster-management-agent-addon
      roleRef:
        apiGroup: rbac.authorization.k8s.io
        kind: Role
        name: open-cluster-management:managed-serviceaccount:addon-agent
      subjects:
      - kind: ServiceAccount
        name: managed-serviceaccount
        namespace: open-cluster-management-agent-addon
    - apiVersion: rbac.authorization.k8s.io/v1
      kind: ClusterRoleBinding
      metadata:
        name: open-cluster-management:managed-serviceaccount:addon-agent
      roleRef:
        apiGroup: rbac.authorization.k8s.io
        kind: ClusterRole
        name: open-cluster-management:managed-serviceaccount:addon-agent
      subjects:
      - kind: ServiceAccount
        name: managed-serviceaccount
        namespace: open-cluster-management-agent-addon
    - apiVersion: rbac.authorization.k8s.io/v1
      kind: ClusterRole
      metadata:
        name: open-cluster-management:managed-serviceaccount:addon-agent
      rules:
      - apiGroups:
        - authentication.k8s.io
        resources:
        - tokenreviews
        verbs:
        - create
    - apiVersion: v1
      kind: ServiceAccount
      metadata:
        name: managed-serviceaccount
        namespace: open-cluster-management-agent-addon
    - apiVersion: v1
      kind: Namespace
      metadata:
        name: open-cluster-management-agent-addon
      spec: {}
      status: {}
status:
  conditions:
  - lastTransitionTime: "2023-08-01T11:02:39Z"
    message: Apply manifest work complete
    observedGeneration: 2
    reason: AppliedManifestWorkComplete
    status: "True"
    type: Applied
  - lastTransitionTime: "2023-08-01T11:02:39Z"
    message: All resources are available
    observedGeneration: 2
    reason: ResourcesAvailable
    status: "True"
    type: Available
  resourceStatus:
    manifests:
    - conditions:
      - lastTransitionTime: "2023-08-01T11:02:39Z"
        message: Apply manifest complete
        reason: AppliedManifestComplete
        status: "True"
        type: Applied
      - lastTransitionTime: "2023-08-01T11:02:39Z"
        message: Resource is available
        reason: ResourceAvailable
        status: "True"
        type: Available
      - lastTransitionTime: "2023-08-01T11:02:39Z"
        message: ""
        reason: NoStatusFeedbackSynced
        status: "True"
        type: StatusFeedbackSynced
      resourceMeta:
        group: apps
        kind: Deployment
        name: managed-serviceaccount-addon-agent
        namespace: open-cluster-management-agent-addon
        ordinal: 0
        resource: deployments
        version: v1
      statusFeedback: {}
    - conditions:
      - lastTransitionTime: "2023-08-01T11:02:39Z"
        message: Apply manifest complete
        reason: AppliedManifestComplete
        status: "True"
        type: Applied
      - lastTransitionTime: "2023-08-01T11:02:39Z"
        message: Resource is available
        reason: ResourceAvailable
        status: "True"
        type: Available
      - lastTransitionTime: "2023-08-01T11:02:39Z"
        message: ""
        reason: NoStatusFeedbackSynced
        status: "True"
        type: StatusFeedbackSynced
      resourceMeta:
        group: rbac.authorization.k8s.io
        kind: Role
        name: open-cluster-management:managed-serviceaccount:addon-agent
        namespace: open-cluster-management-agent-addon
        ordinal: 1
        resource: roles
        version: v1
      statusFeedback: {}
    - conditions:
      - lastTransitionTime: "2023-08-01T11:02:39Z"
        message: Apply manifest complete
        reason: AppliedManifestComplete
        status: "True"
        type: Applied
      - lastTransitionTime: "2023-08-01T11:02:39Z"
        message: Resource is available
        reason: ResourceAvailable
        status: "True"
        type: Available
      - lastTransitionTime: "2023-08-01T11:02:39Z"
        message: ""
        reason: NoStatusFeedbackSynced
        status: "True"
        type: StatusFeedbackSynced
      resourceMeta:
        group: rbac.authorization.k8s.io
        kind: RoleBinding
        name: open-cluster-management:managed-serviceaccount:addon-agent
        namespace: open-cluster-management-agent-addon
        ordinal: 2
        resource: rolebindings
        version: v1
      statusFeedback: {}
    - conditions:
      - lastTransitionTime: "2023-08-01T11:02:39Z"
        message: Apply manifest complete
        reason: AppliedManifestComplete
        status: "True"
        type: Applied
      - lastTransitionTime: "2023-08-01T11:02:39Z"
        message: Resource is available
        reason: ResourceAvailable
        status: "True"
        type: Available
      - lastTransitionTime: "2023-08-01T11:02:39Z"
        message: ""
        reason: NoStatusFeedbackSynced
        status: "True"
        type: StatusFeedbackSynced
      resourceMeta:
        group: rbac.authorization.k8s.io
        kind: ClusterRoleBinding
        name: open-cluster-management:managed-serviceaccount:addon-agent
        namespace: ""
        ordinal: 3
        resource: clusterrolebindings
        version: v1
      statusFeedback: {}
    - conditions:
      - lastTransitionTime: "2023-08-01T11:02:39Z"
        message: Apply manifest complete
        reason: AppliedManifestComplete
        status: "True"
        type: Applied
      - lastTransitionTime: "2023-08-01T11:02:39Z"
        message: Resource is available
        reason: ResourceAvailable
        status: "True"
        type: Available
      - lastTransitionTime: "2023-08-01T11:02:39Z"
        message: ""
        reason: NoStatusFeedbackSynced
        status: "True"
        type: StatusFeedbackSynced
      resourceMeta:
        group: rbac.authorization.k8s.io
        kind: ClusterRole
        name: open-cluster-management:managed-serviceaccount:addon-agent
        namespace: ""
        ordinal: 4
        resource: clusterroles
        version: v1
      statusFeedback: {}
    - conditions:
      - lastTransitionTime: "2023-08-01T11:02:39Z"
        message: Apply manifest complete
        reason: AppliedManifestComplete
        status: "True"
        type: Applied
      - lastTransitionTime: "2023-08-01T11:02:39Z"
        message: Resource is available
        reason: ResourceAvailable
        status: "True"
        type: Available
      - lastTransitionTime: "2023-08-01T11:02:39Z"
        message: ""
        reason: NoStatusFeedbackSynced
        status: "True"
        type: StatusFeedbackSynced
      resourceMeta:
        group: ""
        kind: ServiceAccount
        name: managed-serviceaccount
        namespace: open-cluster-management-agent-addon
        ordinal: 5
        resource: serviceaccounts
        version: v1
      statusFeedback: {}
    - conditions:
      - lastTransitionTime: "2023-08-01T11:02:39Z"
        message: Apply manifest complete
        reason: AppliedManifestComplete
        status: "True"
        type: Applied
      - lastTransitionTime: "2023-08-01T11:02:39Z"
        message: Resource is available
        reason: ResourceAvailable
        status: "True"
        type: Available
      - lastTransitionTime: "2023-08-01T11:02:39Z"
        message: ""
        reason: NoStatusFeedbackSynced
        status: "True"
        type: StatusFeedbackSynced
      resourceMeta:
        group: ""
        kind: Namespace
        name: open-cluster-management-agent-addon
        namespace: ""
        ordinal: 6
        resource: namespaces
        version: v1
      statusFeedback: {}

The ManifestWork is for managed-serviceaccount. It does not mention the namespace openshift-storage but when the namespace was deleted, it was because the orphaned AppliedManifestWork.

apiVersion: work.open-cluster-management.io/v1
kind: AppliedManifestWork
metadata:
  creationTimestamp: "2023-08-01T11:02:39Z"
  deletionGracePeriodSeconds: 0
  deletionTimestamp: "2023-08-01T13:37:59Z"
  finalizers:
  - cluster.open-cluster-management.io/applied-manifest-work-cleanup
  generation: 2
  name: 9010a3f0bd7289444324d69b7aa45ec6014412f0f3719af587d819b4d0c42e40-addon-managed-serviceaccount-deploy-0
  resourceVersion: "3630732"
  uid: ae434785-5373-4c66-b502-f9cc02b316a0
spec:
  agentID: 0ce2d9d0-7b84-4004-95d4-f772de964e95
  hubHash: 9010a3f0bd7289444324d69b7aa45ec6014412f0f3719af587d819b4d0c42e40
  manifestWorkName: addon-managed-serviceaccount-deploy-0
status:
  appliedResources:
  - group: ""
    name: openshift-storage
    namespace: ""
    resource: namespaces
    uid: ab15dc0b-ebbd-43d8-97ca-c2a1eebf17e9
    version: v1
  evictionStartTime: "2023-08-01T12:37:59Z"

The ManifestWork that I put above is for cluster1. But for Cluster2, I do see the openshift-storage namespace referred to it in the manifest work. I will replace the above with the cluster2 that we care about.
Or I will just add it here so that we have two resources for the same but different for each cluster, which is weird
Cluster2 ManifestWork

oc get manifestwork -n pbyregow-clu2   addon-managed-serviceaccount-deploy-0 -o yaml                        
apiVersion: work.open-cluster-management.io/v1
kind: ManifestWork
metadata:
  creationTimestamp: "2023-08-01T11:02:39Z"
  finalizers:
  - cluster.open-cluster-management.io/manifest-work-cleanup
  generation: 2
  labels:
    open-cluster-management.io/addon-name: managed-serviceaccount
  name: addon-managed-serviceaccount-deploy-0
  namespace: pbyregow-clu2
  ownerReferences:
  - apiVersion: addon.open-cluster-management.io/v1alpha1
    blockOwnerDeletion: true
    controller: true
    kind: ManagedClusterAddOn
    name: managed-serviceaccount
    uid: df6c8d46-08ae-4d06-bd1a-804e0788a685
  resourceVersion: "6044869"
  uid: 0713ef3b-754d-4961-af60-7263599dd235
spec:
  workload:
    manifests:
    - apiVersion: apps/v1
      kind: Deployment
      metadata:
        name: managed-serviceaccount-addon-agent
        namespace: openshift-storage
      spec:
        replicas: 1
        selector:
          matchLabels:
            addon-agent: managed-serviceaccount
        strategy: {}
        template:
          metadata:
            labels:
              addon-agent: managed-serviceaccount
          spec:
            containers:
            - args:
              - --leader-elect=true
              - --cluster-name=pbyregow-clu2
              - --kubeconfig=/etc/hub/kubeconfig
              command:
              - /agent
              image: registry.redhat.io/multicluster-engine/managed-serviceaccount-rhel8@sha256:db5a523a3cfbe4d3099766475fa094854cfa87337d32c23094ee1363baa5e3d3
              imagePullPolicy: IfNotPresent
              livenessProbe:
                httpGet:
                  path: /healthz
                  port: 8000
                initialDelaySeconds: 2
                periodSeconds: 10
              name: addon-agent
              resources: {}
              volumeMounts:
              - mountPath: /etc/hub/
                name: hub-kubeconfig
                readOnly: true
            serviceAccount: managed-serviceaccount
            volumes:
            - name: hub-kubeconfig
              secret:
                secretName: managed-serviceaccount-hub-kubeconfig
      status: {}
    - apiVersion: rbac.authorization.k8s.io/v1
      kind: Role
      metadata:
        name: open-cluster-management:managed-serviceaccount:addon-agent
        namespace: openshift-storage
      rules:
      - apiGroups:
        - ""
        resources:
        - configmaps
        verbs:
        - get
        - create
        - update
        - patch
      - apiGroups:
        - ""
        resources:
        - serviceaccounts
        - serviceaccounts/token
        verbs:
        - get
        - watch
        - list
        - create
        - delete
      - apiGroups:
        - coordination.k8s.io
        resources:
        - leases
        verbs:
        - get
        - create
        - update
        - patch
      - apiGroups:
        - authentication.k8s.io
        resources:
        - tokenrequests
        verbs:
        - get
        - create
        - update
        - patch
    - apiVersion: rbac.authorization.k8s.io/v1
      kind: RoleBinding
      metadata:
        name: open-cluster-management:managed-serviceaccount:addon-agent
        namespace: openshift-storage
      roleRef:
        apiGroup: rbac.authorization.k8s.io
        kind: Role
        name: open-cluster-management:managed-serviceaccount:addon-agent
      subjects:
      - kind: ServiceAccount
        name: managed-serviceaccount
        namespace: openshift-storage
    - apiVersion: rbac.authorization.k8s.io/v1
      kind: ClusterRoleBinding
      metadata:
        name: open-cluster-management:managed-serviceaccount:addon-agent
      roleRef:
        apiGroup: rbac.authorization.k8s.io
        kind: ClusterRole
        name: open-cluster-management:managed-serviceaccount:addon-agent
      subjects:
      - kind: ServiceAccount
        name: managed-serviceaccount
        namespace: openshift-storage
    - apiVersion: rbac.authorization.k8s.io/v1
      kind: ClusterRole
      metadata:
        name: open-cluster-management:managed-serviceaccount:addon-agent
      rules:
      - apiGroups:
        - authentication.k8s.io
        resources:
        - tokenreviews
        verbs:
        - create
    - apiVersion: v1
      kind: ServiceAccount
      metadata:
        name: managed-serviceaccount
        namespace: openshift-storage
    - apiVersion: v1
      kind: Namespace
      metadata:
        name: openshift-storage
      spec: {}
      status: {}
status:
  conditions:
  - lastTransitionTime: "2023-08-01T11:02:39Z"
    message: Apply manifest work complete
    observedGeneration: 2
    reason: AppliedManifestWorkComplete
    status: "True"
    type: Applied
  - lastTransitionTime: "2023-08-01T11:02:39Z"
    message: All resources are available
    observedGeneration: 2
    reason: ResourcesAvailable
    status: "True"
    type: Available
  resourceStatus:
    manifests:
    - conditions:
      - lastTransitionTime: "2023-08-01T11:02:39Z"
        message: Apply manifest complete
        reason: AppliedManifestComplete
        status: "True"
        type: Applied
      - lastTransitionTime: "2023-08-01T11:02:39Z"
        message: Resource is available
        reason: ResourceAvailable
        status: "True"
        type: Available
      - lastTransitionTime: "2023-08-01T11:02:39Z"
        message: ""
        reason: NoStatusFeedbackSynced
        status: "True"
        type: StatusFeedbackSynced
      resourceMeta:
        group: apps
        kind: Deployment
        name: managed-serviceaccount-addon-agent
        namespace: openshift-storage
        ordinal: 0
        resource: deployments
        version: v1
      statusFeedback: {}
    - conditions:
      - lastTransitionTime: "2023-08-01T11:02:39Z"
        message: Apply manifest complete
        reason: AppliedManifestComplete
        status: "True"
        type: Applied
      - lastTransitionTime: "2023-08-01T11:02:39Z"
        message: Resource is available
        reason: ResourceAvailable
        status: "True"
        type: Available
      - lastTransitionTime: "2023-08-01T11:02:39Z"
        message: ""
        reason: NoStatusFeedbackSynced
        status: "True"
        type: StatusFeedbackSynced
      resourceMeta:
        group: rbac.authorization.k8s.io
        kind: Role
        name: open-cluster-management:managed-serviceaccount:addon-agent
        namespace: openshift-storage
        ordinal: 1
        resource: roles
        version: v1
      statusFeedback: {}
    - conditions:
      - lastTransitionTime: "2023-08-01T11:02:39Z"
        message: Apply manifest complete
        reason: AppliedManifestComplete
        status: "True"
        type: Applied
      - lastTransitionTime: "2023-08-01T11:02:39Z"
        message: Resource is available
        reason: ResourceAvailable
        status: "True"
        type: Available
      - lastTransitionTime: "2023-08-01T11:02:39Z"
        message: ""
        reason: NoStatusFeedbackSynced
        status: "True"
        type: StatusFeedbackSynced
      resourceMeta:
        group: rbac.authorization.k8s.io
        kind: RoleBinding
        name: open-cluster-management:managed-serviceaccount:addon-agent
        namespace: openshift-storage
        ordinal: 2
        resource: rolebindings
        version: v1
      statusFeedback: {}
    - conditions:
      - lastTransitionTime: "2023-08-01T11:02:39Z"
        message: Apply manifest complete
        reason: AppliedManifestComplete
        status: "True"
        type: Applied
      - lastTransitionTime: "2023-08-01T11:02:39Z"
        message: Resource is available
        reason: ResourceAvailable
        status: "True"
        type: Available
      - lastTransitionTime: "2023-08-01T11:02:39Z"
        message: ""
        reason: NoStatusFeedbackSynced
        status: "True"
        type: StatusFeedbackSynced
      resourceMeta:
        group: rbac.authorization.k8s.io
        kind: ClusterRoleBinding
        name: open-cluster-management:managed-serviceaccount:addon-agent
        namespace: ""
        ordinal: 3
        resource: clusterrolebindings
        version: v1
      statusFeedback: {}
    - conditions:
      - lastTransitionTime: "2023-08-01T11:02:39Z"
        message: Apply manifest complete
        reason: AppliedManifestComplete
        status: "True"
        type: Applied
      - lastTransitionTime: "2023-08-01T11:02:39Z"
        message: Resource is available
        reason: ResourceAvailable
        status: "True"
        type: Available
      - lastTransitionTime: "2023-08-01T11:02:39Z"
        message: ""
        reason: NoStatusFeedbackSynced
        status: "True"
        type: StatusFeedbackSynced
      resourceMeta:
        group: rbac.authorization.k8s.io
        kind: ClusterRole
        name: open-cluster-management:managed-serviceaccount:addon-agent
        namespace: ""
        ordinal: 4
        resource: clusterroles
        version: v1
      statusFeedback: {}
    - conditions:
      - lastTransitionTime: "2023-08-01T11:02:39Z"
        message: Apply manifest complete
        reason: AppliedManifestComplete
        status: "True"
        type: Applied
      - lastTransitionTime: "2023-08-01T11:02:39Z"
        message: Resource is available
        reason: ResourceAvailable
        status: "True"
        type: Available
      - lastTransitionTime: "2023-08-01T11:02:39Z"
        message: ""
        reason: NoStatusFeedbackSynced
        status: "True"
        type: StatusFeedbackSynced
      resourceMeta:
        group: ""
        kind: ServiceAccount
        name: managed-serviceaccount
        namespace: openshift-storage
        ordinal: 5
        resource: serviceaccounts
        version: v1
      statusFeedback: {}
    - conditions:
      - lastTransitionTime: "2023-08-01T11:02:39Z"
        message: Apply manifest complete
        reason: AppliedManifestComplete
        status: "True"
        type: Applied
      - lastTransitionTime: "2023-08-01T11:02:39Z"
        message: Resource is available
        reason: ResourceAvailable
        status: "True"
        type: Available
      - lastTransitionTime: "2023-08-01T11:02:39Z"
        message: ""
        reason: NoStatusFeedbackSynced
        status: "True"
        type: StatusFeedbackSynced
      resourceMeta:
        group: ""
        kind: Namespace
        name: openshift-storage
        namespace: ""
        ordinal: 6
        resource: namespaces
        version: v1
      statusFeedback: {}

For c2, the ManagedClusterAddon created had this installNamespace set to openshift-storage

The bug seems to be in the code that sets the installNamespace when creating the managedserviceaccount managedclusteraddon:
If it finds any addon it'll take the spec.installNamespace from it (each one it loops over will overwrite the previous)
To reproduce, it would require someone installing an addon that went to a non-default namespace, and then, it depends on the ordering - only if the last one it has in the list has the bad namespace would we see this

clones

ACM-6864 2.9 installNamespace is wrongly set when creating the managedserviceaccount managedclusteraddon

Closed

Assignee:: Jian Zhu

Reporter:: Sahar Ebrahimi

QA Contact:: Thuy Nguyen

Votes:: 0 Vote for this issue

Watchers:: 4 Start watching this issue

Created:: 2023/08/26 1:39 AM

Updated:: 2023/11/22 6:48 PM

Resolved:: 2023/11/22 6:48 PM

Details

Description

Attachments

Issue Links

Easy Agile Planning Poker

Activity

People

Dates