-
Bug
-
Resolution: Done-Errata
-
Major
-
4.14
-
Critical
-
No
-
Hypershift Sprint 241, Hypershift Sprint 242
-
2
-
Approved
-
False
-
Description of problem:
4.14 nightly HyperShift hosted cluster aws-pod-identity does not work. Pods are not injected env vars AWS_ROLE_ARN and AWS_WEB_IDENTITY_TOKEN_FILE.
In 4.13 HyperShift hosted cluster, it works well, see Additional info.
Version-Release number of selected component (if applicable):
4.14.0-0.nightly-2023-08-11-055332
How reproducible:
Always
Steps to Reproduce:
1. $ export KUBECONFIG=/path/to/hypershift-hosted-cluster/kubeconfig $ ogcv NAME VERSION AVAILABLE PROGRESSING SINCE STATUS version 4.14.0-0.nightly-2023-08-11-055332 True False 8h Cluster version is 4.14.0-0.nightly-2023-08-11-055332 $ oc get mutatingwebhookconfigurations --context admin NAME WEBHOOKS AGE aws-pod-identity 1 6h5m $ oc get --raw=/.well-known/openid-configuration | jq -r '.issuer' https://xxxx.s3.us-east-2.amazonaws.com/hypershift-xxxx 2. $ oc new-project xxia-proj $ oc create sa aws-provider serviceaccount/aws-provider created 3. $ ccoctl aws create-iam-roles --name=xxia --region=$REGION --credentials-requests-dir=credentialsrequest-dir-aws --identity-provider-arn=arn:aws:iam::xxxx:oidc-provider/xxxx.s3.us-east-2.amazonaws.com/hypershift-xxxx --output-dir=credrequests-ccoctl-output 2023/08/24 17:54:32 Role arn:aws:iam::xxxx:role/xxia-xxia-proj-aws-creds created 2023/08/24 17:54:32 Saved credentials configuration to: credrequests-ccoctl-output/manifests/xxia-proj-aws-creds-credentials.yaml 2023/08/24 17:54:32 Updated Role policy for Role xxia-xxia-proj-aws-creds 4. $ oc annotate sa/aws-provider eks.amazonaws.com/role-arn="arn:aws:iam::xxxx:role/xxia-xxia-proj-aws-creds" $ oc create deployment aws-cli --image=amazon/aws-cli --dry-run=client -o yaml -- sleep 360d | sed "/containers/i \ serviceAccountName: aws-provider" | oc create -f - deployment.apps/aws-cli created $ oc get po NAME READY STATUS RESTARTS AGE aws-cli-5c4f6d7d5b-g6d5v 1/1 Running 0 18s 5. $ oc rsh aws-cli-5c4f6d7d5b-g6d5v sh-4.2$ env | grep AWS sh-4.2$ ls /var/run/secrets/eks.amazonaws.com/serviceaccount/token ls: cannot access /var/run/secrets/eks.amazonaws.com/serviceaccount/token: No such file or directory sh-4.2$ exit command terminated with exit code 1
Actual results:
5. No AWS env vars.
Expected results:
5. Should have AWS env vars.
Additional info:
In 4.13 HyperShift hosted cluster, it works well:
1. $ ogcv NAME VERSION AVAILABLE PROGRESSING SINCE STATUS version 4.13.0-0.nightly-2023-08-11-101506 True False 10h Cluster version is 4.13.0-0.nightly-2023-08-11-101506 $ oc get --raw=/.well-known/openid-configuration | jq -r '.issuer' https://aos-xxxx.s3.us-east-2.amazonaws.com/xxxx $ oc get no NAME STATUS ROLES AGE VERSION ip-10-0-139-76.us-east-2.compute.internal Ready worker 10h v1.26.6+6bf3f75 ... $ REGION=us-east-2 2. $ oc new-project xxia-proj $ oc create sa aws-provider 3. $ ccoctl aws create-iam-roles --name=xxia-test --region=$REGION --credentials-requests-dir=credentialsrequest-dir-aws --identity-provider-arn=arn:aws:iam::xxxx:oidc-provider/aos-xxxx.s3.us-east-2.amazonaws.com/xxxx --output-dir=credrequests-ccoctl-output 2023/08/24 20:06:53 Role arn:aws:iam::xxxx:role/xxia-test-xxia-proj-aws-creds created 2023/08/24 20:06:53 Saved credentials configuration to: credrequests-ccoctl-output/manifests/xxia-proj-aws-creds-credentials.yaml 2023/08/24 20:06:53 Updated Role policy for Role xxia-test-xxia-proj-aws-creds 4. $ oc annotate sa/aws-provider eks.amazonaws.com/role-arn="arn:aws:iam::xxxx:role/xxia-test-xxia-proj-aws-creds" $ oc create deployment aws-cli --image=amazon/aws-cli --dry-run=client -o yaml -- sleep 360d | sed "/containers/i \ serviceAccountName: aws-provider" | oc create -f - $ oc get pod NAME READY STATUS RESTARTS AGE aws-cli-84875995cc-svszl 1/1 Running 0 16s 5. $ oc rsh aws-cli-84875995cc-svszl sh-4.2$ env | grep AWS AWS_ROLE_ARN=arn:aws:iam::xxxx:role/xxia-test-xxia-proj-aws-creds AWS_WEB_IDENTITY_TOKEN_FILE=/var/run/secrets/eks.amazonaws.com/serviceaccount/token AWS_DEFAULT_REGION=us-east-2 AWS_REGION=us-east-2