Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-21745

Azure CCM unable to manage Load Balancer in Azure Managed Identity Installs

    XMLWordPrintable

Details

    • Bug
    • Resolution: Done-Errata
    • Critical
    • 4.15.0
    • 4.14.0, 4.15.0
    • Cloud Credential Operator
    • None
    • No
    • Proposed
    • False
    • Hide

      None

      Show
      None
    • Hide
      * Previously, an Azure Managed Identity role was omitted from the Cloud Controller Manager service account. As a result, the Cloud Controller Manager could not manage service type load balancers in environments deployed to existing vnets with a private publishing method. With this release, the missing role was added to the Cloud Credential Operator utility (`ccoctl`) and Azure Managed Identity installations into an existing vnet with private publishing is possible. (link:https://issues.redhat.com/browse/OCPBUGS-21745[*OCPBUGS-21745*])
      Show
      * Previously, an Azure Managed Identity role was omitted from the Cloud Controller Manager service account. As a result, the Cloud Controller Manager could not manage service type load balancers in environments deployed to existing vnets with a private publishing method. With this release, the missing role was added to the Cloud Credential Operator utility (`ccoctl`) and Azure Managed Identity installations into an existing vnet with private publishing is possible. (link: https://issues.redhat.com/browse/OCPBUGS-21745 [* OCPBUGS-21745 *])
    • Bug Fix
    • Done

    Description

      Description of problem:

      Upon installing 4.14.0-rc.6 in a cluster with private load balancer publishing and existing vnets Service type LoadBalancers lack permissions necessary to sync.

      Version-Release number of selected component (if applicable):

      4.14.0-rc.6

      How reproducible:

      Seemingly 100%

      Steps to Reproduce:

      1. Install w/ azure Managed Identity into an existing vnet with private LB publishing
2.
3.

      Actual results:

                      One or more other status conditions indicate a degraded state: LoadBalancerReady=False (SyncLoadBalancerFailed: The service-controller component is reporting SyncLoadBalancerFailed events like: Error syncing load balancer: failed to ensure load balancer: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 403, RawError: {"error":{"code":"AuthorizationFailed","message":"The client '194d5669-cb47-4199-a673-4b32a4a110be' with object id '194d5669-cb47-4199-a673-4b32a4a110be' does not have authorization to perform action 'Microsoft.Network/virtualNetworks/subnets/read' over scope '/subscriptions/14b86a40-8d8f-4e69-abaf-42cbb0b8a331/resourceGroups/net/providers/Microsoft.Network/virtualNetworks/rnd-we-net/subnets/paas1' or the scope is invalid. If access was recently granted, please refresh your credentials."}}

Operators dependent on Ingress are failing as well.
authentication                             4.14.0-rc.6   False       False         True       149m    OAuthServerRouteEndpointAccessibleControllerAvailable: Get https://oauth-openshift.apps.cnb10161.rnd.westeurope.example.com/healthz: dial tcp: lookup oauth-openshift.apps.cnb10161.rnd.westeurope.example.com on 10.224.0.10:53: no such host (this is likely result of malfunctioning DNS server)
console                                    4.14.0-rc.6   False       True          False      142m    DeploymentAvailable: 0 replicas available for console deployment...

       

      Expected results:

      Successful install

      Additional info:

      The client ID in the error correspond to “openshift-cloud-controller-manager-azure-cloud-credentials” which indeed when checking its Azure managed identity only has access to cluster RG and not the network RG.

Additionally, they note that this permission is granted to the MAPI roles just not the CCM roles.

       

      Attachments

        Issue Links

          blocks

          Bug - A problem that impairs or prevents the functions of the product. OCPBUGS-21926 Azure CCM unable to manage Load Balancer in Azure Managed Identity Installs

          • Critical - To be worked on immediately following BLOCKER issues.
          • Closed
          is cloned by

          Bug - A problem that impairs or prevents the functions of the product. OCPBUGS-21926 Azure CCM unable to manage Load Balancer in Azure Managed Identity Installs

          • Critical - To be worked on immediately following BLOCKER issues.
          • Closed
          relates to

          Feature - Capability or a well-defined set of functionality that delivers business value. Features can include additions or changes to existing functionality. Features can easily span multiple teams, and potentially multiple releases. OCPSTRAT-513 Azure managed identity with Azure AD workload identity for self-managed OpenShift

          • Blocker - To be worked above all other priorities. This term is chosen when the severity of the issue is very high, has no workaround, or the effort for the change is comparatively low. Issues that may be very publicly visible and could generate significant media attention may also drive a higher priority.
          • Closed
          links to

          GitHub openshift/cloud-credential-operator#607: OCPBUGS-21745: azure create-managed-identites to add cloud controller manager to network resource group

          Web Link RHEA-2023:7198 rpm

          Activity

            People

              jstuever@redhat.com Jeremiah Stuever
              rhn-support-sdodson Scott Dodson
              Mingxia Huang Mingxia Huang
              Jeana Routh Jeana Routh
              Votes:
              0 Vote for this issue
              Watchers:
              14 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: