Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-34332

Prompt “unable to sync storage configuration” after configure Managed Identity even using owner privileged role

    • Icon: Bug Bug
    • Resolution: Not a Bug
    • Icon: Critical Critical
    • None
    • premerge
    • Image Registry
    • Critical
    • No
    • False
    • Hide

      None

      Show
      None

      Description of problem:

          Create an azure cluster which enabled wokload identity using image built from https://github.com/openshift/cluster-image-registry-operator/pull/1020 , then configure openshift-image-registry/secret/installer-cloud-credentials to a new created managed identity with owner role(Or using the image registry existing managed identity which had configured appropriate permissions)

      Version-Release number of selected component (if applicable):

          https://github.com/openshift/cluster-image-registry-operator/pull/1020

      How reproducible:

         always 

      Steps to Reproduce:

          1.Create an azure cluster enabled workload identity
          2.Create a new managed identity from azure portal under this resource group, and save the client id
          3.Update the installer-cloud-credentials secret under openshift-image-registry with new azure client id and empty azure_federated_token_file data.
      $oc set data secret/installer-cloud-credentials --from-file=azure_federated_token_file=azure_federated_token_file --from-file=azure_client_id=azure_client_id -n openshift-image-registry
          4.Check the secret installer-cloud-credentials
      $oc get secret  installer-cloud-credentials -o jsonpath='{.data}' -n openshift-image-registry | jq -r
      {
        "azure_client_id": "MDdkNjI5YzQtZDcyOS00NTIzLTljY2EtNTJmMGVjMmZjMDYzCg==",
        "azure_federated_token_file": "",
        "azure_region": "ZWFzdHVz",
        "azure_subscription_id": "NTNiOGY1NTEtZjBmYy00YmVhLThjYmEtNmQxZmVmZDU0Yzhh",
        "azure_tenant_id": "NjA0N2M3ZTktYjJhZC00ODhkLWE1NGUtZGMzZjZiZTZhN2Vl"
      }
          5. Wait the registry pod back to running with the new credentials , push some images to image registry
      oc get pods -n openshift-image-registry -l docker-registry=default
      NAME                              READY   STATUS    RESTARTS   AGE
      image-registry-59489d45b5-6plzq   1/1     Running   0          45m
      image-registry-59489d45b5-tzb7g   1/1     Running   0          45m

      Actual results:

      registry operator reports "unable to sync storage configuration error", but could push/pull data to/from image registry.
      
      And couldn't update the storage account with the managed identity.
      
       %  oc get co image-registry
      NAME             VERSION                                                   AVAILABLE   PROGRESSING   DEGRADED   SINCE   MESSAGE
      image-registry   4.16.0-0.ci.test-2024-05-21-033626-ci-ln-5y2kwmk-latest   True        True          False      3h32m   Progressing: Unable to apply resources: unable to sync storage configuration: failed to get keys for the storage account imageregistrywxj5232lcgq: autorest/Client#Do: Preparing request failed: StatusCode=0 -- Original Error: ManagedIdentityCredential: ManagedIdentityCredential: Get "http://169.254.169.254/metadata/identity/oauth2/token?api-version=2018-02-01&client_id=07d629c4-d729-4523-9cca-52f0ec2fc063%0A&resource=https%3A%2F%2Fmanagement.azure.com%2F": dial tcp 169.254.169.254:80: connect: connection refused...
      xiuwang@default-route-openshift-image-registry /tmp % oc get builds
      NAME         TYPE     FROM          STATUS    STARTED          DURATION
      httpd-ex-1   Source   Git@6dd8df4   Running   23 seconds ago
      xiuwang@default-route-openshift-image-registry /tmp % oc get builds
      NAME         TYPE     FROM          STATUS     STARTED         DURATION
      httpd-ex-1   Source   Git@6dd8df4   Complete   2 minutes ago   28s
      xiuwang@default-route-openshift-image-registry /tmp % oc get pods
      NAME                        READY   STATUS      RESTARTS   AGE
      httpd-ex-1-build            0/1     Completed   0          2m43s
      httpd-ex-5dd7fb65c4-jvvnh   1/1     Running     0          2m16s

      Expected results:

          Shouldn't prompt errors and could update the storage account after configure managed identity.

      Additional info:

          Must-gather log https://drive.google.com/file/d/1SuIqVeFzKrfN5J6voOuzvtOxnn8UDyPh/view?usp=sharing 

              rh-ee-brcox Bryan Cox
              rh-ee-xiuwang XiuJuan Wang
              XiuJuan Wang XiuJuan Wang
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Created:
                Updated:
                Resolved: