-
Bug
-
Resolution: Done-Errata
-
Major
-
4.15.0, 4.16.0
-
Critical
-
No
-
Rejected
-
False
-
-
Release Note Not Required
-
In Progress
Description of problem:
HCP does not honor the oauthMetadata field of hc.spec.configuration.authentication, making console crash and oc login fail.
Version-Release number of selected component (if applicable):
HyperShift management cluster: 4.16.0-0.nightly-2024-01-29-233218 HyperShift hosted cluster: 4.16.0-0.nightly-2024-01-29-233218
How reproducible:
Always
Steps to Reproduce:
1. Install HCP env. Export KUBECONFIG: $ export KUBECONFIG=/path/to/hosted-cluster/kubeconfig 2. Create keycloak applications. Then get the route: $ KEYCLOAK_HOST=https://$(oc get -n keycloak route keycloak --template='{{ .spec.host }}') $ echo $KEYCLOAK_HOST https://keycloak-keycloak.apps.hypershift-ci-18556.xxx $ curl -sSk "$KEYCLOAK_HOST/realms/master/.well-known/openid-configuration" > oauthMetadata $ cat oauthMetadata {"issuer":"https://keycloak-keycloak.apps.hypershift-ci-18556.xxx/realms/master" $ oc create configmap oauth-meta --from-file ./oauthMetadata -n clusters --kubeconfig /path/to/management-cluster/kubeconfig ... 3. Set hc.spec.configuration.authentication: $ CLIENT_ID=openshift-test-aud $ oc patch hc hypershift-ci-18556 -n clusters --kubeconfig /path/to/management-cluster/kubeconfig --type=merge -p=" spec: configuration: authentication: oauthMetadata: name: oauth-meta oidcProviders: - claimMappings: ... issuer: audiences: - $CLIENT_ID issuerCertificateAuthority: name: keycloak-oidc-ca issuerURL: $KEYCLOAK_HOST/realms/master name: keycloak-oidc-test type: OIDC " Check KAS indeed already picks up the setting: $ oc logs -c kube-apiserver kube-apiserver-5c976d59f5-zbrwh -n clusters-hypershift-ci-18556 --kubeconfig /path/to/management-cluster/kubeconfig | grep "oidc-" ... I0130 08:07:24.266247 1 flags.go:64] FLAG: --oidc-ca-file="/etc/kubernetes/certs/oidc-ca/ca.crt" I0130 08:07:24.266251 1 flags.go:64] FLAG: --oidc-client-id="openshift-test-aud" ... I0130 08:07:24.266261 1 flags.go:64] FLAG: --oidc-issuer-url="https://keycloak-keycloak.apps.hypershift-ci-18556.xxx/realms/master" ... Wait about 15 mins. 4. Check COs and check oc login. Both show the same error: $ oc get co | grep -v 'True.*False.*False' NAME VERSION AVAILABLE PROGRESSING DEGRADED SINCE MESSAGE console 4.16.0-0.nightly-2024-01-29-233218 True True False 4h57m SyncLoopRefreshProgressing: Working toward version 4.16.0-0.nightly-2024-01-29-233218, 1 replicas available $ oc get po -n openshift-console NAME READY STATUS RESTARTS AGE console-547cf6bdbb-l8z9q 1/1 Running 0 4h55m console-54f88749d7-cv7ht 0/1 CrashLoopBackOff 9 (3m18s ago) 14m console-54f88749d7-t7x96 0/1 CrashLoopBackOff 9 (3m32s ago) 14m $ oc logs console-547cf6bdbb-l8z9q -n openshift-console I0130 03:23:36.788951 1 metrics.go:156] usage.Metrics: Update console users metrics: 0 kubeadmin, 0 cluster-admins, 0 developers, 0 unknown/errors (took 406.059196ms) E0130 06:48:32.745179 1 asynccache.go:43] failed a caching attempt: request to OAuth issuer endpoint https://:0/oauth/token failed: Head "https://:0": dial tcp :0: connect: connection refused E0130 06:53:32.757881 1 asynccache.go:43] failed a caching attempt: request to OAuth issuer endpoint https://:0/oauth/token failed: Head "https://:0": dial tcp :0: connect: connection refused ... $ oc login --exec-plugin=oc-oidc --client-id=openshift-test-aud --extra-scopes=email,profile --callback-port=8080 error: oidc authenticator error: oidc discovery error: Get "https://:0/.well-known/openid-configuration": dial tcp :0: connect: connection refused error: oidc authenticator error: oidc discovery error: Get "https://:0/.well-known/openid-configuration": dial tcp :0: connect: connection refused Unable to connect to the server: getting credentials: exec: executable oc failed with exit code 1 5. Check root cause, the configured oauthMetadata is not picked up well: $ curl -k https://a6e149f24f8xxxxxx.elb.ap-east-1.amazonaws.com:6443/.well-known/oauth-authorization-server { "issuer": "https://:0", "authorization_endpoint": "https://:0/oauth/authorize", "token_endpoint": "https://:0/oauth/token", ... }
Actual results:
As above steps 4 and 5, the configured oauthMetadata is not picked up well, causing console and oc login hit the error.
Expected results:
The configured oauthMetadata is picked up well. No error.
Additional info:
For oc, if I manually use `oc config set-credentials oidc --exec-api-version=client.authentication.k8s.io/v1 --exec-command=oc --exec-arg=get-token --exec-arg="--issuer-url=$KEYCLOAK_HOST/realms/master" ...` instead of using `oc login --exec-plugin=oc-oidc ...`, oc authentication works well. This means my configuration is correct. $ oc whoami Please visit the following URL in your browser: http://localhost:8080 oidc-user-test:xxia@redhat.com
- blocks
-
OCPBUGS-29025 HCP .well-known/oauth-authorization-server shows "https://:0" even OIDC oauthMetadata is set in hc.spec.configuration.authentication
- Closed
- is cloned by
-
OCPBUGS-29025 HCP .well-known/oauth-authorization-server shows "https://:0" even OIDC oauthMetadata is set in hc.spec.configuration.authentication
- Closed
- links to
-
RHEA-2024:0041 OpenShift Container Platform 4.16.z bug fix update