-
Bug
-
Resolution: Done-Errata
-
Critical
-
4.15.z
-
None
-
Critical
-
No
-
False
-
Description of problem:
In 4.15 cluster, after configuring external OIDC, kube-apiserver pod crashes with: run.go:74] "command failed" err="strict decoding error: unknown field \"jwt[0].claimMappings.uid\"". In 4.16 cluster, this issue is not reproduced.
Version-Release number of selected component (if applicable):
4.15.0-0.nightly-2024-02-28-013638
How reproducible:
Unsure for now.
Steps to Reproduce:
1. Install 4.15 fresh HCP env and configure external OIDC: $ HC_NAME=hypershift-ci-267403 MGMT_KUBECONFIG=/home/xxia/my/env/xxia-hs415-267403-4.15/kubeconfig HOSTED_KUBECONFIG=/home/xxia/my/env/xxia-hs415-267403-4.15/hypershift-ci-267403.kubeconfig AUDIENCE=76863fb1-xxxxxx ISSUER_URL=https://login.microsoftonline.com/xxxxxxxx/v2.0 CLIENT_ID=76863fb1-xxxxxx CLIENT_SECRET_VALUE="xxxxxxxx" CLIENT_SECRET_NAME=console-secret $ curl -sS "$ISSUER_URL/.well-known/openid-configuration" > microsoft-entra-id-oauthMetadata $ export KUBECONFIG=$HOSTED_KUBECONFIG $ oc create configmap tested-oauth-meta --from-file=oauthMetadata=microsoft-entra-id-oauthMetadata -n clusters --kubeconfig $MGMT_KUBECONFIG configmap/tested-oauth-meta created $ oc patch hc $HC_NAME -n clusters --kubeconfig $MGMT_KUBECONFIG --type=merge -p=" spec: configuration: authentication: oauthMetadata: name: tested-oauth-meta oidcProviders: - claimMappings: groups: claim: groups prefix: 'oidc-groups-test:' username: claim: email prefixPolicy: Prefix prefix: prefixString: 'oidc-user-test:' issuer: audiences: - $AUDIENCE issuerURL: $ISSUER_URL name: microsoft-entra-id oidcClients: - clientID: $CLIENT_ID clientSecret: name: $CLIENT_SECRET_NAME componentName: console componentNamespace: openshift-console type: OIDC " hostedcluster.hypershift.openshift.io/hypershift-ci-267403 patched $ oc create secret generic console-secret -n clusters --from-literal=clientSecret=$CLIENT_SECRET_VALUE --kubeconfig $MGMT_KUBECONFIG secret/console-secret created $ oc get authentication.config cluster -o yaml apiVersion: config.openshift.io/v1 kind: Authentication metadata: ... spec: oauthMetadata: name: tested-oauth-meta oidcProviders: - claimMappings: groups: claim: groups prefix: 'oidc-groups-test:' username: claim: email prefix: prefixString: 'oidc-user-test:' prefixPolicy: Prefix issuer: audiences: - 76863fb1-xxxxxx issuerCertificateAuthority: name: "" issuerURL: https://login.microsoftonline.com/xxxxxxxx/v2.0 name: microsoft-entra-id oidcClients: - clientID: 76863fb1-xxxxxx clientSecret: name: console-secret componentName: console componentNamespace: openshift-console serviceAccountIssuer: https://xxxxxxxx.s3.us-east-2.amazonaws.com/hypershift-ci-267403 type: OIDC status: oidcClients: - componentName: console componentNamespace: openshift-console conditions: - lastTransitionTime: "2024-02-28T09:20:08Z" message: "" reason: OIDCConfigAvailable status: "False" type: Degraded - lastTransitionTime: "2024-02-28T09:20:08Z" message: "" reason: OIDCConfigAvailable status: "False" type: Progressing - lastTransitionTime: "2024-02-28T09:20:08Z" message: "" reason: OIDCConfigAvailable status: "True" type: Available currentOIDCClients: - clientID: 76863fb1-xxxxxx issuerURL: https://login.microsoftonline.com/xxxxxxxx/v2.0 oidcProviderName: microsoft-entra-id $ oc get po -n clusters-$HC_NAME --kubeconfig $MGMT_KUBECONFIG --sort-by metadata.creationTimestamp NAME READY STATUS RESTARTS AGE ... openshift-apiserver-d665bdc58-7cfdg 3/3 Running 0 154m kube-controller-manager-577cf4566f-sgxz2 1/1 Running 0 154m openshift-apiserver-d665bdc58-52w9m 3/3 Running 0 154m kube-apiserver-74f569dfb5-7tnmn 4/5 CrashLoopBackOff 7 (2m47s ago) 15m $ oc logs --timestamps -n clusters-$HC_NAME --kubeconfig $MGMT_KUBECONFIG -c kube-apiserver kube-apiserver-74f569dfb5-7tnmn > ~/my/logs/kube-apiserver-74f569dfb5-7tnmn-CrashLoopBackOff-hcp415.log $ oc get cm auth-config -n clusters-$HC_NAME --kubeconfig $MGMT_KUBECONFIG -o jsonpath='{.data.auth\.json}' {"kind":"AuthenticationConfiguration","apiVersion":"apiserver.config.k8s.io/v1alpha1","jwt":[{"issuer":{"url":"https://login.microsoftonline.com/xxxxxxxx/v2.0","audiences":["76863fb1-xxxxxx"],"audienceMatchPolicy":"MatchAny"},"claimMappings":{"username":{"claim":"email","prefix":"oidc-user-test:"},"groups":{"claim":"groups","prefix":"oidc-groups-test:"},"uid":{}}}]} $ oc get cm auth-config -n clusters-$HC_NAME --kubeconfig $MGMT_KUBECONFIG -o jsonpath='{.data.auth\.json}' | jq | ~/auto/json2yaml.sh --- kind: AuthenticationConfiguration apiVersion: apiserver.config.k8s.io/v1alpha1 jwt: - issuer: url: https://login.microsoftonline.com/xxxxxxxx/v2.0 audiences: - 76863fb1-xxxxxx audienceMatchPolicy: MatchAny claimMappings: username: claim: email prefix: 'oidc-user-test:' groups: claim: groups prefix: 'oidc-groups-test:' uid: {} $ vi ~/my/logs/kube-apiserver-74f569dfb5-7tnmn-CrashLoopBackOff-hcp415.log ... 2024-02-28T09:32:06.077307893Z I0228 09:32:06.077298 1 options.go:220] external host was not specified, using 172.20.0.1 2024-02-28T09:32:06.077977888Z I0228 09:32:06.077952 1 server.go:189] Version: v1.28.6+6216ea1 2024-02-28T09:32:06.077977888Z I0228 09:32:06.077971 1 server.go:191] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" 2024-02-28T09:32:06.078556862Z I0228 09:32:06.078543 1 dynamic_serving_content.go:113] "Loaded a new cert/key pair" name="serving-cert::/etc/kubernetes/certs/server/tls.crt::/etc/kubernetes/certs/server/tls.key" 2024-02-28T09:32:06.408308750Z I0228 09:32:06.408274 1 dynamic_cafile_content.go:119] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/certs/client-ca/ca.crt" 2024-02-28T09:32:06.408487434Z E0228 09:32:06.408467 1 run.go:74] "command failed" err="strict decoding error: unknown field \"jwt[0].claimMappings.uid\""
Actual results:
As shown in above "Description of problem"
Expected results:
4.16 does not have the issue. 4.15 should have no such problem.
Additional info:
- links to
-
RHSA-2024:1210 OpenShift Container Platform 4.15.z security update