Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-46483

[Azure disk/file csi driver]on ARO HCP the CPO reconcile CSO CSI Secrets incorrect

XMLWordPrintable

    • Important
    • None
    • Hypershift Sprint 264
    • 1
    • Proposed
    • False
    • Hide

      None

      Show
      None

      Description of problem:

      [Azure disk/file csi driver]on ARO HCP could not provision volume succeed   

      Version-Release number of selected component (if applicable):

      4.19.0-0.nightly-2024-12-13-083421    

      How reproducible:

      Always    

      Steps to Reproduce:

          1.Install AKS cluster on azure.
          2.Install hypershift operator on the AKS cluster.
          3.Use hypershift CLI create hosted cluster with the Client Certificate mode.
          4.Check the azure disk/file csi dirver work well on hosted cluster.

      Actual results:

          In step 4: the the azure disk/file csi dirver provision volume failed on hosted cluster
      
      # azure disk pvc provision failed
      $ oc describe pvc mypvc
      ...
        Normal   WaitForFirstConsumer  74m                    persistentvolume-controller                                                                                waiting for first consumer to be created before binding
        Normal   Provisioning          74m                    disk.csi.azure.com_azure-disk-csi-driver-controller-74d944bbcb-7zz89_2334468f-9d27-4bdd-a53c-27271ee60073  External provisioner is provisioning volume for claim "default/mypvc"
        Warning  ProvisioningFailed    74m                    disk.csi.azure.com_azure-disk-csi-driver-controller-74d944bbcb-7zz89_2334468f-9d27-4bdd-a53c-27271ee60073  failed to provision volume with StorageClass "managed-csi": rpc error: code = Unavailable desc = error reading from server: EOF
        Warning  ProvisioningFailed    71m                    disk.csi.azure.com_azure-disk-csi-driver-controller-74d944bbcb-7zz89_28ba5ad9-c4f8-4dc8-be40-c80c546b7ef8  failed to provision volume with StorageClass "managed-csi": rpc error: code = Unavailable desc = error reading from server: EOF
        Normal   Provisioning          71m                    disk.csi.azure.com_azure-disk-csi-driver-controller-74d944bbcb-7zz89_28ba5ad9-c4f8-4dc8-be40-c80c546b7ef8  External provisioner is provisioning volume for claim "default/mypvc"
      ...
      
      $ oc logs azure-disk-csi-driver-controller-74d944bbcb-7zz89 -c csi-driver
      W1216 08:07:04.282922       1 main.go:89] nodeid is empty
      I1216 08:07:04.290689       1 main.go:165] set up prometheus server on 127.0.0.1:8201
      I1216 08:07:04.291073       1 azuredisk.go:213]
      DRIVER INFORMATION:
      -------------------
      Build Date: "2024-12-13T02:45:35Z"
      Compiler: gc
      Driver Name: disk.csi.azure.com
      Driver Version: v1.29.11
      Git Commit: 4d21ae15d668d802ed5a35068b724f2e12f47d5c
      Go Version: go1.23.2 (Red Hat 1.23.2-1.el9) X:strictfipsruntime
      Platform: linux/amd64
      Topology Key: topology.disk.csi.azure.com/zone
      
      I1216 08:09:36.814776       1 utils.go:77] GRPC call: /csi.v1.Controller/CreateVolume
      I1216 08:09:36.814803       1 utils.go:78] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.disk.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.disk.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":1073741824},"name":"pvc-d6af3900-ec5b-4e09-83d6-d0e112b02316","parameters":{"csi.storage.k8s.io/pv/name":"pvc-d6af3900-ec5b-4e09-83d6-d0e112b02316","csi.storage.k8s.io/pvc/name":"mypvc","csi.storage.k8s.io/pvc/namespace":"default","skuname":"Premium_LRS"},"volume_capabilities":[{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":7}}]}
      I1216 08:09:36.815338       1 controllerserver.go:208] begin to create azure disk(pvc-d6af3900-ec5b-4e09-83d6-d0e112b02316) account type(Premium_LRS) rg(ci-op-zj9zc4gd-12c20-rg) location(centralus) size(1) diskZone() maxShares(0)
      panic: runtime error: invalid memory address or nil pointer dereference
      [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x190c61d]
      
      goroutine 153 [running]:
      sigs.k8s.io/cloud-provider-azure/pkg/provider.(*ManagedDiskController).CreateManagedDisk(0x0, {0x2265cf0, 0xc0001285a0}, 0xc0003f2640)
      	/go/src/github.com/openshift/azure-disk-csi-driver/vendor/sigs.k8s.io/cloud-provider-azure/pkg/provider/azure_managedDiskController.go:127 +0x39d
      sigs.k8s.io/azuredisk-csi-driver/pkg/azuredisk.(*Driver).CreateVolume(0xc000564540, {0x2265cf0, 0xc0001285a0}, 0xc000272460)
      	/go/src/github.com/openshift/azure-disk-csi-driver/pkg/azuredisk/controllerserver.go:297 +0x2c59
      github.com/container-storage-interface/spec/lib/go/csi._Controller_CreateVolume_Handler.func1({0x2265cf0?, 0xc0001285a0?}, {0x1e5a260?, 0xc000272460?})
      	/go/src/github.com/openshift/azure-disk-csi-driver/vendor/github.com/container-storage-interface/spec/lib/go/csi/csi.pb.go:6420 +0xcb
      sigs.k8s.io/azuredisk-csi-driver/pkg/csi-common.logGRPC({0x2265cf0, 0xc0001285a0}, {0x1e5a260, 0xc000272460}, 0xc00017cb80, 0xc00014ea68)
      	/go/src/github.com/openshift/azure-disk-csi-driver/pkg/csi-common/utils.go:80 +0x409
      github.com/container-storage-interface/spec/lib/go/csi._Controller_CreateVolume_Handler({0x1f3e440, 0xc000564540}, {0x2265cf0, 0xc0001285a0}, 0xc00029a700, 0x2084458)
      	/go/src/github.com/openshift/azure-disk-csi-driver/vendor/github.com/container-storage-interface/spec/lib/go/csi/csi.pb.go:6422 +0x143
      google.golang.org/grpc.(*Server).processUnaryRPC(0xc00059cc00, {0x2265cf0, 0xc000128510}, {0x2270d60, 0xc0004f5980}, 0xc000308480, 0xc000226a20, 0x31c8f80, 0x0)
      	/go/src/github.com/openshift/azure-disk-csi-driver/vendor/google.golang.org/grpc/server.go:1379 +0xdf8
      google.golang.org/grpc.(*Server).handleStream(0xc00059cc00, {0x2270d60, 0xc0004f5980}, 0xc000308480)
      	/go/src/github.com/openshift/azure-disk-csi-driver/vendor/google.golang.org/grpc/server.go:1790 +0xe8b
      google.golang.org/grpc.(*Server).serveStreams.func2.1()
      	/go/src/github.com/openshift/azure-disk-csi-driver/vendor/google.golang.org/grpc/server.go:1029 +0x7f
      created by google.golang.org/grpc.(*Server).serveStreams.func2 in goroutine 16
      	/go/src/github.com/openshift/azure-disk-csi-driver/vendor/google.golang.org/grpc/server.go:1040 +0x125
      
      # azure file pvc provision failed
      $ oc describe pvc mypvc
      Name:          mypvc
      Namespace:     openshift-cluster-csi-drivers
      StorageClass:  azurefile-csi
      Status:        Pending
      Volume:
      Labels:        <none>
      Annotations:   volume.beta.kubernetes.io/storage-provisioner: file.csi.azure.com
                     volume.kubernetes.io/storage-provisioner: file.csi.azure.com
      Finalizers:    [kubernetes.io/pvc-protection]
      Capacity:
      Access Modes:
      VolumeMode:    Filesystem
      Used By:       <none>
      Events:
        Type     Reason                Age                From                                                                                                      Message
        ----     ------                ----               ----                                                                                                      -------
        Normal   ExternalProvisioning  14s (x2 over 14s)  persistentvolume-controller                                                                               Waiting for a volume to be created either by the external provisioner 'file.csi.azure.com' or manually by the system administrator. If volume creation is delayed, please verify that the provisioner is running and correctly registered.
        Normal   Provisioning          7s (x4 over 14s)   file.csi.azure.com_azure-file-csi-driver-controller-879f56577-5hjn8_38c8218e-e52c-4248-ada7-268742afaac0  External provisioner is provisioning volume for claim "openshift-cluster-csi-drivers/mypvc"
        Warning  ProvisioningFailed    7s (x4 over 14s)   file.csi.azure.com_azure-file-csi-driver-controller-879f56577-5hjn8_38c8218e-e52c-4248-ada7-268742afaac0  failed to provision volume with StorageClass "azurefile-csi": rpc error: code = Internal desc = failed to ensure storage account: could not list storage accounts for account type Standard_LRS: StorageAccountClient is nil
      
      

      Expected results:

          In step 4: the the azure disk/file csi dirver should provision volume succeed on hosted cluster 

      Additional info:

          

              rh-ee-brcox Bryan Cox
              rhn-support-pewang Penghao Wang
              Penghao Wang Penghao Wang
              Penghao Wang
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Created:
                Updated: