-
Bug
-
Resolution: Unresolved
-
Undefined
-
4.14.z
-
None
-
None
-
False
-
-
None
-
Moderate
-
None
-
None
-
None
-
None
-
In Progress
-
Release Note Not Required
-
None
-
None
-
None
-
None
-
None
Description of problem:
When attempting to exercise the azure csi file driver storage class private endpoint options, we encounter errors in a workload identity cluster with minimal permissions, meaning there are permissions from the credentials request. Missing permissions are: - Microsoft.Network/privateDnsZones/read - Microsoft.Network/privateDnsZones/virtualNetworkLinks/write - Microsoft.Network/virtualNetworks/join/action - Microsoft.Network/privateEndpoints/write - Microsoft.Network/virtualNetworks/subnets/join/action - Microsoft.Storage/storageAccounts/PrivateEndpointConnectionsApproval/action - Microsoft.Network/privateEndpoints/privateDnsZoneGroups/write
Version-Release number of selected component (if applicable):
4.14.z +
How reproducible:
Every time
Steps to Reproduce:
1. Create an azure workload identity cluster with the current set of permissions outlined for azure file csi driver 2. Create a private dns zone in your network resource group (defined in the azure cloud config) named: "privatelink.file.core.windows.net" 3. Create a storage class with the driver option of "networkEndpointType" set to "privateEndpoint" 4. Attempt to create a pvc and use it in a pod, it fails.
Actual results:
{ "error": { "code": "AuthorizationFailed", "message": "The client '<client-id>' with object id '<object-id>' does not have authorization to perform action 'Microsoft.Network/privateDnsZones/read' over scope '/subscriptions/<subscription-id>/resourceGroups/bvesel/providers/Microsoft.Network/privateDnsZones/privatelink.file.core.windows.net' or the scope is invalid. If access was recently granted, please refresh your credentials." } } Following resolution of that, the next one appears: { "error": { "code": "AuthorizationFailed", "message": "The client '<client-id>' with object id '<object-id>' does not have authorization to perform action 'Microsoft.Network/privateDnsZones/virtualNetworkLinks/write' over scope '/subscriptions/<subscription-id>/resourceGroups/bvesel/providers/Microsoft.Network/privateDnsZones/privatelink.file.core.windows.net/virtualNetworkLinks/vnet-vnetlink' or the scope is invalid. If access was recently granted, please refresh your credentials." } } Following resolution of that, you need "message": "The client 'dd91be29-1d24-4338-87bf-04073e1c1f23' with object id 'dd91be29-1d24-4338-87bf-04073e1c1f23' has permission to perform action 'Microsoft.Network/privateDnsZones/virtualNetworkLinks/write' on scope '/subscriptions/fe16a035-e540-4ab7-80d9-373fa9a3d6ae/resourceGroups/bvesel/providers/Microsoft.Network/privateDnsZones/privatelink.file.core.windows.net/virtualNetworkLinks/vnet-vnetlink'; however, it does not have permission to perform action(s) 'Microsoft.Network/virtualNetworks/join/action' on the linked scope(s) '/subscriptions/<subscription-id>/resourceGroups/bvesel/providers/Microsoft.Network/virtualNetworks/vnet' (respectively) or the linked scope(s) are invalid." Almost there, missing Private endpoint permissions E0402 02:23:15.862384 1 utils.go:82] GRPC error: rpc error: code = Internal desc = failed to ensure storage account: create private endpoint for storage account(f8cfaf76d6fe34f629b7ae9), resourceGroup(bvesel): Retriable: false, RetryAfter: 0s, HTTPStatusCode: 403, RawError: {"error":{"code":"AuthorizationFailed","message":"The client 'dd91be29-1d24-4338-87bf-04073e1c1f23' with object id 'dd91be29-1d24-4338-87bf-04073e1c1f23' does not have authorization to perform action 'Microsoft.Network/privateEndpoints/write' over scope '/subscriptions/<subscription-id>/resourceGroups/bvesel/providers/Microsoft.Network/privateEndpoints/f8cfaf76d6fe34f629b7ae9-pvtendpoint' or the scope is invalid. If access was recently granted, please refresh your credentials."}} Following resolution of that, the next one appears: E0402 02:36:08.811567 1 utils.go:82] GRPC error: rpc error: code = Internal desc = failed to ensure storage account: create private endpoint for storage account(f00597ddf81894301a8d571), resourceGroup(bvesel): Retriable: false, RetryAfter: 0s, HTTPStatusCode: 403, RawError: {"error":{"code":"LinkedAuthorizationFailed","message":"The client 'dd91be29-1d24-4338-87bf-04073e1c1f23' with object id 'dd91be29-1d24-4338-87bf-04073e1c1f23' has permission to perform action 'Microsoft.Network/privateEndpoints/write' on scope '/subscriptions/<subscription>/resourceGroups/bvesel/providers/Microsoft.Network/privateEndpoints/f00597ddf81894301a8d571-pvtendpoint'; however, it does not have permission to perform action(s) 'Microsoft.Network/virtualNetworks/subnets/join/action' on the linked scope(s) '/subscriptions/<subscription>/resourceGroups/bvesel/providers/Microsoft.Network/virtualNetworks/vnet/subnets/worker' (respectively) or the linked scope(s) are invalid."}} Following resolution of that, the next one appears: E0402 02:43:28.895789 1 utils.go:82] GRPC error: rpc error: code = Internal desc = failed to ensure storage account: create private endpoint for storage account(fa9bb9d31287c4c668158cf), resourceGroup(bvesel): Retriable: false, RetryAfter: 0s, HTTPStatusCode: 403, RawError: {"error":{"code":"LinkedAuthorizationFailed","message":"The client 'dd91be29-1d24-4338-87bf-04073e1c1f23' with object id 'dd91be29-1d24-4338-87bf-04073e1c1f23' has permission to perform action 'Microsoft.Network/privateEndpoints/write' on scope '/subscriptions/<subscription>/resourceGroups/bvesel/providers/Microsoft.Network/privateEndpoints/fa9bb9d31287c4c668158cf-pvtendpoint'; however, it does not have permission to perform action(s) 'Microsoft.Storage/storageAccounts/PrivateEndpointConnectionsApproval/action' on the linked scope(s) '/subscriptions/<subscription>/resourceGroups/aro-hj8744wi/providers/Microsoft.Storage/storageAccounts/fa9bb9d31287c4c668158cf' (respectively) or the linked scope(s) are invalid."}} After resolving the above... E0402 03:07:45.741799 1 utils.go:82] GRPC error: rpc error: code = Internal desc = failed to ensure storage account: create private DNS zone group - privateEndpoint(f029ab610e7904847b7fa00-pvtendpoint), vNetName(vnet), resourceGroup(bvesel): Retriable: false, RetryAfter: 0s, HTTPStatusCode: 403, RawError: {"error":{"code":"AuthorizationFailed","message":"The client 'dd91be29-1d24-4338-87bf-04073e1c1f23' with object id 'dd91be29-1d24-4338-87bf-04073e1c1f23' does not have authorization to perform action 'Microsoft.Network/privateEndpoints/privateDnsZoneGroups/write' over scope '/subscriptions/<subscription>/resourceGroups/bvesel/providers/Microsoft.Network/privateEndpoints/f029ab610e7904847b7fa00-pvtendpoint/privateDnsZoneGroups/f029ab610e7904847b7fa00-dnszonegroup' or the scope is invalid. If access was recently granted, please refresh your credentials."}} And finally we're successful: $ oc get pvc NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE busybox-data-busybox-0 Bound pvc-8cc621a1-af25-4331-9180-687c3551f9c6 10Gi RWO file 3m40s
Expected results:
No failures, PVC is able to be created and attached successfully.
Additional info:
https://github.com/kubernetes-sigs/azurefile-csi-driver/blob/master/docs/driver-parameters.md --- # storageclass.yaml allowVolumeExpansion: true apiVersion: storage.k8s.io/v1 kind: StorageClass metadata: name: file parameters: skuname: Premium_LRS networkEndpointType: privateEndpoint provisioner: file.csi.azure.com reclaimPolicy: Delete volumeBindingMode: WaitForFirstConsumer --- --- # statefulset.yaml apiVersion: apps/v1 kind: StatefulSet metadata: name: busybox spec: selector: matchLabels: app: busybox updateStrategy: type: RollingUpdate replicas: 1 template: metadata: labels: app: busybox spec: containers: - name: busybox image: registry.hub.docker.com/library/busybox resources: limits: memory: 2Gi volumeMounts: - name: busybox-data mountPath: /usr/share/busybox volumeClaimTemplates: - metadata: name: busybox-data spec: accessModes: [ "ReadWriteOnce" ] storageClassName: "file" resources: requests: storage: 10Gi
- blocks
-
OCPBUGS-56253 Azure CSI File Missing Permissions in CredentialsRequest
-
- POST
-
- is cloned by
-
OCPBUGS-56253 Azure CSI File Missing Permissions in CredentialsRequest
-
- POST
-
- is related to
-
OCPBUGS-55182 Missing Azure permissions in azurefiles credentialsrequest
-
- Closed
-
- links to