Description of problem:
CCO fails to check if the root credential has sufficient permissions for cr/cloud-credential-operator-gcp-ro-creds in passthrough mode.
Steps to Reproduce:
1. Create a GCP cluster with ClusterBot (4.14.0-0.nightly-2023-07-21-020713) 2. Switch to passthrough mode: fxie@fxie-mac hive % oc edit cloudcredential 3. Remove the status subresource of cr/cloud-credential-operator-gcp-ro-creds to trigger a reconcile: fxie@fxie-mac hive % oc edit credentialsrequest -n openshift-cloud-credential-operator cloud-credential-operator-gcp-ro-creds --subresource='status' 4. Check CR status: fxie@fxie-mac hive % oc get credentialsrequest -n openshift-cloud-credential-operator cloud-credential-operator-gcp-ro-creds -o jsonpath='{.status}' | jq { "conditions": [ { "lastProbeTime": "2023-07-21T10:30:29Z", "lastTransitionTime": "2023-07-21T10:30:29Z", "message": "failed to grant creds: error while validating permissions: error testing permissions: googleapi: Error 400: Permission advisorynotifications.notifications.get is not valid for this resource., badRequest", "reason": "CredentialsProvisionFailure", "status": "True", "type": "CredentialsProvisionFailure" } ], "lastSyncGeneration": 0, "provisioned": false } 5. Check CCO status: fxie@fxie-mac hive % oc get co cloud-credential -o jsonpath='{.status.conditions}' | jq [ { "lastTransitionTime": "2023-07-21T09:50:01Z", "message": "All is well", "reason": "AsExpected", "status": "True", "type": "Available" }, { "lastTransitionTime": "2023-07-21T10:30:29Z", "message": "1 of 7 credentials requests are failing to sync.", "reason": "CredentialsFailing", "status": "True", "type": "Degraded" }, { "lastTransitionTime": "2023-07-21T10:30:27Z", "message": "6 of 7 credentials requests provisioned, 1 reporting errors.", "reason": "Reconciling", "status": "True", "type": "Progressing" }, { "lastTransitionTime": "2023-07-21T09:50:01Z", "message": "All is well", "reason": "AsExpected", "status": "True", "type": "Upgradeable" } ] fxie@fxie-mac hive % oc logs -f cloud-credential-operator-6c5df89fc4-kjmj2 -n openshift-cloud-credential-operator -c cloud-credential-operator | grep -i "level=error" time="2023-07-21T10:30:29Z" level=error msg="error syncing credentials: error while validating permissions: error testing permissions: googleapi: Error 400: Permission advisorynotifications.notifications.get is not valid for this resource., badRequest" controller=credreq cr=openshift-cloud-credential-operator/cloud-credential-operator-gcp-ro-creds secret=openshift-cloud-credential-operator/cloud-credential-operator-gcp-ro-creds time="2023-07-21T10:30:29Z" level=error msg="errored with condition: CredentialsProvisionFailure" controller=credreq cr=openshift-cloud-credential-operator/cloud-credential-operator-gcp-ro-creds secret=openshift-cloud-credential-operator/cloud-credential-operator-gcp-ro-creds time="2023-07-21T10:30:31Z" level=error msg="error syncing credentials: error while validating permissions: error testing permissions: googleapi: Error 400: Permission advisorynotifications.notifications.get is not valid for this resource., badRequest" controller=credreq cr=openshift-cloud-credential-operator/cloud-credential-operator-gcp-ro-creds secret=openshift-cloud-credential-operator/cloud-credential-operator-gcp-ro-creds time="2023-07-21T10:30:31Z" level=error msg="errored with condition: CredentialsProvisionFailure" controller=credreq cr=openshift-cloud-credential-operator/cloud-credential-operator-gcp-ro-creds secret=openshift-cloud-credential-operator/cloud-credential-operator-gcp-ro-creds ...
Additional info:
We encountered the same problem on another cluster installed into the openshift-qe gcp project.
- is duplicated by
-
OCPBUGS-16808 CCO fails to check if the root credential has sufficient permissions for cr/cloud-credential-operator-gcp-ro-creds in passthrough mode
- Closed
- is related to
-
OCPBUGS-36140 GCP cluster with CCO Passthrough mode failed to install due to CCO degraded
- Closed