Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-48359

GCP fails to assign permissions to installer created SA

XMLWordPrintable

    • None
    • False
    • Hide

      None

      Show
      None
    • Hide

      * Previously, when a Google Cloud Platform (GCP) service account was created, the account would not always be immediately available. When the account was not available for updates, the installation program received failures when adding permissions to the account. According to link:https://cloud.google.com/iam/docs/retry-strategy[*Retry failed requests*], a service account might be created, but is not active for up to 60 seconds. With this release, the service account is updated on an exponential backoff to give the account enough time to update correctly. (link:https://issues.redhat.com/browse/OCPBUGS-48359[*OCPBUGS-48359*])
      ====
      Previously, when a GCP service account was created, the account would not always be available right away. When the account was not available for updates, the installer would receive failures when adding permissions to the account. According to Google, https://cloud.google.com/iam/docs/retry-strategy, a service account may be created but not active for up to 60 seconds. With this release, the service account
      will be updated on an exponential backoff to give the account enough time to update correctly.
      Show
      * Previously, when a Google Cloud Platform (GCP) service account was created, the account would not always be immediately available. When the account was not available for updates, the installation program received failures when adding permissions to the account. According to link: https://cloud.google.com/iam/docs/retry-strategy [*Retry failed requests*], a service account might be created, but is not active for up to 60 seconds. With this release, the service account is updated on an exponential backoff to give the account enough time to update correctly. (link: https://issues.redhat.com/browse/OCPBUGS-48359 [* OCPBUGS-48359 *]) ==== Previously, when a GCP service account was created, the account would not always be available right away. When the account was not available for updates, the installer would receive failures when adding permissions to the account. According to Google, https://cloud.google.com/iam/docs/retry-strategy, a service account may be created but not active for up to 60 seconds. With this release, the service account will be updated on an exponential backoff to give the account enough time to update correctly.
    • Bug Fix
    • Done

      This is a clone of issue OCPBUGS-48187. The following is the description of the original issue:
      —
      This is a clone of issue OCPBUGS-45280. The following is the description of the original issue:
      —
      Description of problem:

      DEBUG Creating ServiceAccount for control plane nodes 
DEBUG Service account created for XXXXX-gcp-r4ncs-m 
DEBUG Getting policy for openshift-dev-installer   
DEBUG adding roles/compute.instanceAdmin role, added serviceAccount:XXXXX-gcp-r4ncs-m@openshift-dev-installer.iam.gserviceaccount.com member 
DEBUG adding roles/compute.networkAdmin role, added serviceAccount:XXXXX-gcp-r4ncs-m@openshift-dev-installer.iam.gserviceaccount.com member 
DEBUG adding roles/compute.securityAdmin role, added serviceAccount:XXXXX-gcp-r4ncs-m@openshift-dev-installer.iam.gserviceaccount.com member 
DEBUG adding roles/storage.admin role, added serviceAccount:XXXXX-gcp-r4ncs-m@openshift-dev-installer.iam.gserviceaccount.com member 
ERROR failed to fetch Cluster: failed to generate asset "Cluster": failed to create cluster: failed during pre-provisioning: failed to add master roles: failed to set IAM policy, unexpected error: googleapi: Error 400: Service account XXXXX-gcp-r4ncs-m@openshift-dev-installer.iam.gserviceaccount.com does not exist., badRequest

It appears that the Service account was created correctly. The roles are assigned to the service account. It is possible that there needs to be a "wait for action to complete" on the server side to ensure that this will all be ok.

      Version-Release number of selected component (if applicable):

      How reproducible:

      Random. Appears to be a sync issue

      Steps to Reproduce:

          1. Run the installer for a normal GCP basic install
    2.
    3.

      Actual results:

          Installer fails saying that the Service Account that the installer created does not have the permissions to perform an action. Sometimes it takes numerous tries for this to happen (very intermittent).

      Expected results:

          Successful install

      Additional info:

              rh-ee-bbarbach Brent Barbachem
              openshift-crt-jira-prow OpenShift Prow Bot
              Jianli Wei Jianli Wei
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

                Created:
                Updated:
                Resolved: