Uploaded image for project: 'OpenShift GitOps'
  1. OpenShift GitOps
  2. GITOPS-6367

ArgoCD fails to claim proper privileges to administrate the operartor CRDs

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Major Major
    • 1.18.0
    • 1.15.0, 1.16.0, 1.17.0, 1.18.0
    • ArgoCD
    • 8
    • False
    • Hide

      None

      Show
      None
    • False
    • Hide
      Before this update, the application controller lacked the necessary permissions to manage custom resources (CRs) introduced by new operators. With this update, the GitOps Operator watches the admin cluster role and immediately updates the application controller role with the necessary permissions. Now, the application controller will have the required permissions to manage custom resources in a managed namespace.
      Show
      Before this update, the application controller lacked the necessary permissions to manage custom resources (CRs) introduced by new operators. With this update, the GitOps Operator watches the admin cluster role and immediately updates the application controller role with the necessary permissions. Now, the application controller will have the required permissions to manage custom resources in a managed namespace.
    • GitOps Scarlet Sprint 17, GitOps Scarlet Sprint 18, GitOps Scarlet Sprint 19

      Description of Problem

      Customer is trying to deploy the `ibm-common-service operator` and `ibm-integration-platform-navigator` operators using ArgoCD application.
      With this application, they are trying deploy the 2 namespaces, catalogsources, ImageDigestMirrorSet and operator subscriptions and custom resources.
      Rollout of the resources largely work, except that ArgoCD fails to claim proper privileges to administrate the CRDs provided by the operators and we get a below errors :

      one or more objects failed to apply, reason: error when patching "/dev/shm/660316825": commonservices.operator.ibm.com "common-service" is forbidden: User "system:serviceaccount:openshift-gitops:openshift-gitops-argocd-application-controller" cannot patch resource "commonservices" in API group "operator.ibm.com" in the namespace "openshift-operators",platformnavigators.integration.ibm.com is forbidden: User "system:serviceaccount:openshift-gitops:openshift-gitops-argocd-application-controller"
      
      platformnavigators.integration.ibm.com is forbidden: User "system:serviceaccount:openshift-gitops:openshift-gitops-argocd-application-controller" cannot create resource "platformnavigators" in API group "integration.ibm.com" in the namespace "cp4i"
      

      The respective namespaces has a `argocd.argoproj.io/managed-by: openshift-gitops` label added.
      Still the respective `openshift-gitops-argocd-application-controller` role does not have permission for these respective resources in API group "operator.ibm.com" and "integration.ibm.com".

      Additional Info

      Customer has below hypothesis on this :

      As per customer, the `argocd.argoproj.io/managed-by` label is only honored on changes to namespace meta data, but not when new CRDs are introduced, which is causing permission issue.

      Customer is suspecting that ArgoCD does not recognize these CRDs and does not update the role, Once the namespace got updated with any change it recognize the CRDs and update the role to have a proper permissions on it.

      Customer presume that the ArgoCD component responsible for updating the namespace roles, is listening for events related to namespace updates in order to honor the argocd.argoproj.io/managed-by labels.
      They suspect that the same operator is either not listening for events related to new CRDs or that such events are improperly routed to this operator.

      ArgoCD/helm prioritizes namespace creation before most other resources, so naturally we can't expect that the initial argocd.argoproj.io/managed-by label causes permissions to be included for a CRD which hasn't been installed on the cluster yet. Instead we need to rely on ArgoCD discovering the discrepancy and adding the missing permissions at a later point in time.

      Workaround (If Possible)

      Customer confirmed that, As a workaround when they manually update/change namespace metadata, like simply updating namespace with just an annotation `foo=bar` after the CRDs successfully have been installed on the cluster.
      With this update to namespace, The respective role also gets updated with proper permission for the respective api resources and the api group.

              cbanavik Chetan Banavikalmutt
              rhn-support-dkarde Dipak Karde
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated:
                Resolved: