Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-30412

CCO degrade when remove root credential for GCP cluster in Mint mode

XMLWordPrintable

    • Yes
    • Rejected
    • False
    • Hide

      None

      Show
      None
    • Hide
      * Previously, if the root credentials were removed from a Google Cloud Platform (GCP) cluster that was in mint mode, the Cloud Credential Operator (CCO) would go into a degraded state after approximately 1 hour. This issue means that CCO could not manage the credentials root secret for a component. With this update, mint mode supports custom roles so that removing root credentials from a GCP cluster does not cause the CCO to go into a degraded state. (link:https://issues.redhat.com/browse/OCPBUGS-30412[*OCPBUGS-30412*])
      Show
      * Previously, if the root credentials were removed from a Google Cloud Platform (GCP) cluster that was in mint mode, the Cloud Credential Operator (CCO) would go into a degraded state after approximately 1 hour. This issue means that CCO could not manage the credentials root secret for a component. With this update, mint mode supports custom roles so that removing root credentials from a GCP cluster does not cause the CCO to go into a degraded state. (link: https://issues.redhat.com/browse/OCPBUGS-30412 [* OCPBUGS-30412 *])
    • Bug Fix
    • Done

      This is a clone of issue OCPBUGS-28787. The following is the description of the original issue:
      —
      Description of problem:

      It was found when testing OCP-71263 and regression OCP-35770 for 4.15.
For GCP in Mint mode, the root credential can be removed after cluster installation.
But after removing the root credential, CCO became degrade.    

      Version-Release number of selected component (if applicable):

      4.15.0-0.nightly-2024-01-25-051548

4.15.0-rc.3

      How reproducible:

          
Always

      Steps to Reproduce:

          1.Install a GCP cluster with Mint mode

    2.After install, remove the root credential
jianpingshu@jshu-mac ~ % oc delete secret -n kube-system gcp-credentials
secret "gcp-credentials" deleted     

    3.Wait some time(about 1/2h to 1h), CCO became degrade 
    
jianpingshu@jshu-mac ~ % oc get co cloud-credential
NAME               VERSION       AVAILABLE   PROGRESSING   DEGRADED   SINCE   MESSAGE
cloud-credential   4.15.0-rc.3   True        True          True       6h45m   6 of 7 credentials requests are failing to sync.

jianpingshu@jshu-mac ~ % oc -n openshift-cloud-credential-operator get -o json credentialsrequests | jq -r '.items[] | select(tostring | contains("InfrastructureMismatch") | not) | .metadata.name as $n | .status.conditions // [{type: "NoConditions"}] | .[] | .type + "=" + .status + " " + $n + " " + .reason + ": " + .message' | sort
CredentialsProvisionFailure=False openshift-cloud-network-config-controller-gcp CredentialsProvisionSuccess: successfully granted credentials request
CredentialsProvisionFailure=True cloud-credential-operator-gcp-ro-creds CredentialsProvisionFailure: failed to grant creds: unable to fetch root cloud cred secret: Secret "gcp-credentials" not found
CredentialsProvisionFailure=True openshift-gcp-ccm CredentialsProvisionFailure: failed to grant creds: unable to fetch root cloud cred secret: Secret "gcp-credentials" not found
CredentialsProvisionFailure=True openshift-gcp-pd-csi-driver-operator CredentialsProvisionFailure: failed to grant creds: unable to fetch root cloud cred secret: Secret "gcp-credentials" not found
CredentialsProvisionFailure=True openshift-image-registry-gcs CredentialsProvisionFailure: failed to grant creds: unable to fetch root cloud cred secret: Secret "gcp-credentials" not found
CredentialsProvisionFailure=True openshift-ingress-gcp CredentialsProvisionFailure: failed to grant creds: unable to fetch root cloud cred secret: Secret "gcp-credentials" not found
CredentialsProvisionFailure=True openshift-machine-api-gcp CredentialsProvisionFailure: failed to grant creds: unable to fetch root cloud cred secret: Secret "gcp-credentials" not found

openshift-cloud-network-config-controller-gcp has no failure because it doesn't has customized role in 4.15.0.rc3

      Actual results:

       CCO became degrade

      Expected results:

       CCO not in degrade, just "upgradeable" condition updated with missing the root credential

      Additional info:

      Tested the same case on 4.14.10, no issue

       

            jstuever@redhat.com Jeremiah Stuever
            openshift-crt-jira-prow OpenShift Prow Bot
            Jianping Shu Jianping Shu
            Jeana Routh Jeana Routh
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated:
              Resolved: