Uploaded image for project: 'OCP Technical Release Team'
  1. OCP Technical Release Team
  2. TRT-1368

4.15 Nightly Payloads Failing on GCP Credentials Quota


    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Blocker Blocker
    • None
    • None
    • False
    • None
    • False

      Hit seemingly every job in the last payload:


      Credentials request shows:

      "conditions": [                                                                                                
              "lastProbeTime": "2023-11-26T11:20:40Z",                                                                   
              "lastTransitionTime": "2023-11-26T11:20:40Z",
              "message": "failed to grant creds: error syncing creds in mint-mode: error creating custom role: rpc error: code = ResourceExhausted desc = Maximum number of roles reached. Maximum is: 300\nerror details: retry in 24h0m1s",
              "reason": "CredentialsProvisionFailure",
              "status": "True",                             
              "type": "CredentialsProvisionFailure"         

      We've heard a new gcp account is live, but we're not sure if these are landing in it or not. Perhaps they are and a limit needs to be bumped?

      Additional info

      This issue shows up as a Cluster Version Operator component readiness regression due to failing the following tests:

      • [sig-cluster-lifecycle] Cluster completes upgrade
      • [sig-arch][Feature:ClusterUpgrade] Cluster should be upgradeable after finishing upgrade [Late][Suite:upgrade]
      • [sig-arch][Feature:ClusterUpgrade] Cluster should remain functional during upgrade [Disruptive] [Serial]

            rhn-engineering-dgoodwin Devan Goodwin
            rhn-engineering-dgoodwin Devan Goodwin
            0 Vote for this issue
            3 Start watching this issue