-
Bug
-
Resolution: Unresolved
-
Undefined
-
None
-
4.17.z
-
None
-
None
-
False
-
Description of problem:
Unable to use AWS EFS CSI cross account feature in ROSA Classic
Version-Release number of selected component (if applicable):
ROSA Classic v4
How reproducible:
Always
Steps to Reproduce:
1. Following below doc to install AWS EFS CSI driver operator:
[0]https://cloud.redhat.com/experts/rosa/aws-efs
My IAM role for EFS CSI driver operator is hollytest-aws-efs-csi-operator.
Trust policy:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Federated": "arn:aws:iam::[AWS-ACCOUNT-A]:oidc-provider/oidc.os1.devshift.org/[]"
},
"Action": "sts:AssumeRoleWithWebIdentity",
"Condition": {
"StringEquals": {
"oidc.os1.devshift.org/[]:sub": [
"system:serviceaccount:openshift-cluster-csi-drivers:aws-efs-csi-driver-operator",
"system:serviceaccount:openshift-cluster-csi-drivers:aws-efs-csi-driver-controller-sa"
]
}
}
}
]
}
hollytest-rosa-efs-csi
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"elasticfilesystem:DescribeAccessPoints",
"elasticfilesystem:DescribeFileSystems",
"elasticfilesystem:DescribeMountTargets",
"elasticfilesystem:TagResource",
"ec2:DescribeAvailabilityZones"
],
"Resource": "*"
},
{
"Effect": "Allow",
"Action": [
"elasticfilesystem:CreateAccessPoint"
],
"Resource": "*",
"Condition": {
"StringLike": {
"aws:RequestTag/efs.csi.aws.com/cluster": "true"
}
}
},
{
"Effect": "Allow",
"Action": "elasticfilesystem:DeleteAccessPoint",
"Resource": "*",
"Condition": {
"StringEquals": {
"aws:ResourceTag/efs.csi.aws.com/cluster": "true"
}
}
}
]
}
my-cross-account-assume-policy:
{
"Version": "2012-10-17",
"Statement": {
"Effect": "Allow",
"Action": "sts:AssumeRole",
"Resource": "arn:aws:iam::[AWS-ACCOUNT-B]:role/holly-efs-acrossaccount-role"
}
}
2. Following below doc to enable AWS EFS across account feature:
[1]https://docs.openshift.com/container-platform/4.13/storage/container_storage_interface/persistent-storage-csi-aws-efs.html#persistent-storage-csi-efs-cross-account_persistent-storage-csi-aws-efs
Account B file system ID: fs-0ad92785ad729ee53
VPC peering done, security group and route table rules done.
CSI driver's controller service used IAM role in AWS account B named: holly-efs-acrossaccount-role.
Trust policy:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::[AWS-ACCOUNT-A]:root"
},
"Action": "sts:AssumeRole",
"Condition": {}
}
]
}
holly-cross-account-aasume-policy:
{
"Version": "2012-10-17",
"Statement": {
"Effect": "Allow",
"Action": "sts:AssumeRole",
"Resource": "arn:aws:iam::[AWS-ACCOUNT-B]:role/holly-efs-acrossaccount-role"
}
}
holly-efs-acrossaccount-driver-policy:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": [
"ec2:DescribeNetworkInterfaces",
"ec2:DescribeSubnets"
],
"Resource": "*"
},
{
"Sid": "VisualEditor1",
"Effect": "Allow",
"Action": [
"elasticfilesystem:DescribeMountTargets",
"elasticfilesystem:DeleteAccessPoint",
"elasticfilesystem:ClientMount",
"elasticfilesystem:DescribeAccessPoints",
"elasticfilesystem:ClientWrite",
"elasticfilesystem:ClientRootAccess",
"elasticfilesystem:DescribeFileSystems",
"elasticfilesystem:CreateAccessPoint"
],
"Resource": [
"arn:aws:elasticfilesystem:*:[AWS-ACCOUNT-B]:access-point/*",
"arn:aws:elasticfilesystem:*:[AWS-ACCOUNT-B]:file-system/*"
]
}
]
}
ManagedOpenshift-ControlPlane-Role attached with AmazonElasticFileSystemClientFullAccess policy.
Secret and rolebinding done.
3. Create test storageclass and test pvc:
kind: StorageClass apiVersion: storage.k8s.io/v1 metadata: name: efs-cross-account-mount-sc provisioner: efs.csi.aws.com mountOptions: - tls parameters: provisioningMode: efs-ap fileSystemId: fs-0ad92785ad729ee53 directoryPerms: "700" gidRangeStart: "1000" gidRangeEnd: "2000" basePath: "/account-a-data" csi.storage.k8s.io/provisioner-secret-name: my-efs-cross-account csi.storage.k8s.io/provisioner-secret-namespace: openshift-cluster-csi-drivers volumeBindingMode: Immediate
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: test
spec:
storageClassName: efs-cross-account-mount-sc
accessModes:
- ReadWriteMany
resources:
requests:
storage: 5Gi
Actual results:
PVC has below error:
Warning ProvisioningFailed 58m (x14 over 81m) efs.csi.aws.com_ip-10-0-25-177_28f23c19-4e8c-40dc-9b38-f6bd3b84b614 failed to provision volume with StorageClass "efs-cross-account-mount-sc": rpc error: code = Unauthenticated desc = Access Denied. Please ensure you have the right AWS permissions: Access denied Normal Provisioning 3m5s (x29 over 81m) efs.csi.aws.com_ip-10-0-25-177_28f23c19-4e8c-40dc-9b38-f6bd3b84b614 External provisioner is provisioning volume for claim "default/test" Normal ExternalProvisioning 93s (x330 over 81m) persistentvolume-controller Waiting for a volume to be created either by the external provisioner 'efs.csi.aws.com' or manually by the system administrator. If volume creation is delayed, please verify that the provisioner is running and correctly registered.
EFS CSI Driver controller pod repeated logs:
I1231 03:48:21.749330 1 metadata.go:65] getting MetadataService...
I1231 03:48:21.751289 1 metadata.go:70] retrieving metadata from EC2 metadata service
I1231 03:48:21.845411 1 controller.go:307] Using PV name for access point directory.
I1231 03:48:21.845431 1 controller.go:314] Using /account-a-data/pvc-92ad6300-2533-4eed-8f87-13593a57a7ab as the access point directory.
E1231 03:48:21.868512 1 driver.go:107] GRPC error: rpc error: code = Unauthenticated desc = Access Denied. Please ensure you have the right AWS permissions: Access denied
Expected results:
ROSA supports AWS EFS CSI across account feature.
Additional info:
