Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-16646

CCO fails to check if the root credential has sufficient permissions for cr/cloud-credential-operator-gcp-ro-creds in passthrough mode

XMLWordPrintable

      Description of problem:

      CCO fails to check if the root credential has sufficient permissions for cr/cloud-credential-operator-gcp-ro-creds in passthrough mode. 

      Steps to Reproduce:

      1. Create a GCP cluster with ClusterBot (4.14.0-0.nightly-2023-07-21-020713)
      
      2. Switch to passthrough mode:
      fxie@fxie-mac hive % oc edit cloudcredential
      
      3. Remove the status subresource of cr/cloud-credential-operator-gcp-ro-creds to trigger a reconcile:
      fxie@fxie-mac hive % oc edit credentialsrequest -n openshift-cloud-credential-operator cloud-credential-operator-gcp-ro-creds --subresource='status'
      
      4. Check CR status:
      fxie@fxie-mac hive % oc get credentialsrequest -n openshift-cloud-credential-operator cloud-credential-operator-gcp-ro-creds -o jsonpath='{.status}' | jq
      {
        "conditions": [
          {
            "lastProbeTime": "2023-07-21T10:30:29Z",
            "lastTransitionTime": "2023-07-21T10:30:29Z",
            "message": "failed to grant creds: error while validating permissions: error testing permissions: googleapi: Error 400: Permission advisorynotifications.notifications.get is not valid for this resource., badRequest",
            "reason": "CredentialsProvisionFailure",
            "status": "True",
            "type": "CredentialsProvisionFailure"
          }
        ],
        "lastSyncGeneration": 0,
        "provisioned": false
      } 
      
      5. Check CCO status:
      fxie@fxie-mac hive % oc get co cloud-credential -o jsonpath='{.status.conditions}' | jq 
      [
        {
          "lastTransitionTime": "2023-07-21T09:50:01Z",
          "message": "All is well",
          "reason": "AsExpected",
          "status": "True",
          "type": "Available"
        },
        {
          "lastTransitionTime": "2023-07-21T10:30:29Z",
          "message": "1 of 7 credentials requests are failing to sync.",
          "reason": "CredentialsFailing",
          "status": "True",
          "type": "Degraded"
        },
        {
          "lastTransitionTime": "2023-07-21T10:30:27Z",
          "message": "6 of 7 credentials requests provisioned, 1 reporting errors.",
          "reason": "Reconciling",
          "status": "True",
          "type": "Progressing"
        },
        {
          "lastTransitionTime": "2023-07-21T09:50:01Z",
          "message": "All is well",
          "reason": "AsExpected",
          "status": "True",
          "type": "Upgradeable"
        }
      ]
      
      
      fxie@fxie-mac hive % oc logs -f cloud-credential-operator-6c5df89fc4-kjmj2 -n openshift-cloud-credential-operator -c cloud-credential-operator | grep -i "level=error"
      time="2023-07-21T10:30:29Z" level=error msg="error syncing credentials: error while validating permissions: error testing permissions: googleapi: Error 400: Permission advisorynotifications.notifications.get is not valid for this resource., badRequest" controller=credreq cr=openshift-cloud-credential-operator/cloud-credential-operator-gcp-ro-creds secret=openshift-cloud-credential-operator/cloud-credential-operator-gcp-ro-creds
      time="2023-07-21T10:30:29Z" level=error msg="errored with condition: CredentialsProvisionFailure" controller=credreq cr=openshift-cloud-credential-operator/cloud-credential-operator-gcp-ro-creds secret=openshift-cloud-credential-operator/cloud-credential-operator-gcp-ro-creds
      time="2023-07-21T10:30:31Z" level=error msg="error syncing credentials: error while validating permissions: error testing permissions: googleapi: Error 400: Permission advisorynotifications.notifications.get is not valid for this resource., badRequest" controller=credreq cr=openshift-cloud-credential-operator/cloud-credential-operator-gcp-ro-creds secret=openshift-cloud-credential-operator/cloud-credential-operator-gcp-ro-creds
      time="2023-07-21T10:30:31Z" level=error msg="errored with condition: CredentialsProvisionFailure" controller=credreq cr=openshift-cloud-credential-operator/cloud-credential-operator-gcp-ro-creds secret=openshift-cloud-credential-operator/cloud-credential-operator-gcp-ro-creds
      ...

      Additional info:

      We encountered the same problem on another cluster installed into the openshift-qe gcp project. 

              jstuever@redhat.com Jeremiah Stuever
              fxierh Feilian Xie
              Jianping Shu Jianping Shu
              Jianping Shu, Lin Wang
              Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

                Created:
                Updated:
                Resolved: