-
Bug
-
Resolution: Duplicate
-
Undefined
-
None
-
premerge
-
None
-
Quality / Stability / Reliability
-
False
-
-
None
-
Critical
-
None
-
None
-
None
-
None
-
None
-
None
-
None
-
None
-
None
-
None
-
None
-
None
Description of problem:
This is a bug found during pre-merge test of 4.18 epic AUTH-528 PRs and filed for better tracking per existing "OpenShift - Testing Before PR Merges - Left-Shift Testing" google doc workflow.
After OCP BYO external OIDC configuration is configured, OAuth resources (oauth-openshift and oauth-apiserver pods) and associated APIs and stuff are not removed, though they do not work any more and the BYO external OIDC mechanism works well (i.e. `oc login --exec-plugin=oc-oidc ...snipped...` and console login to the external OIDC both work well)
Version-Release number of selected component (if applicable):
The cluster-bot build that is built at 2024-11-13 07:38 CST (UTC+800): build openshift/cluster-authentication-operator#713,openshift/cluster-kube-apiserver-operator#1760
How reproducible:
Always
Steps to Reproduce:
1. Launch a TechPreviewNoUpgrade cluster with above cluster-bot build that is built at 2024-11-13 07:38 CST (UTC+800).
2. Before BYO external OIDC configuration, configure OAuth IDP and prepare some OAuth resources (users, oauthaccesstokens) for later check:
$ oc login -u testuser-10 -p ...snipped...
Login successful.
...
$ oc whoami -t
sha256~JmDE...snipped...
$ oc get user
NAME UID FULL NAME IDENTITIES
testuser-10 45186db0-c152-4f01-a3cb-c3a4e951f201 flexy-htpasswd-provider:testuser-10
$ oc get oauthaccesstoken
NAME USER NAME CLIENT NAME CREATED EXPIRES REDIRECT URI SCOPES
sha256~... testuser-10 openshift-challenging-client 3m37s 2024-11-14 02:32:03 +0000 UTC https://...snipped.../oauth/token/implicit user:full
3. Configure BYO external OIDC:
# "OPENSHIFT DEV" tenant
ISSUER_URL=https://login.microsoftonline.com/...snipped...a7ee/v2.0
CONSOLE_CLIENT_ID=...snipped...b778
CONSOLE_CLIENT_SECRET_VALUE='...snipped...Wb04'
CONSOLE_CLIENT_SECRET_NAME=console-secret
CLI_CLIENT_ID=...snipped...900e
AUDIENCE_1=$CONSOLE_CLIENT_ID
AUDIENCE_2=$CLI_CLIENT_ID
oc create secret generic $CONSOLE_CLIENT_SECRET_NAME --from-literal=clientSecret=$CONSOLE_CLIENT_SECRET_VALUE -n openshift-config
oc patch authentication.config/cluster --type=merge -p="
spec:
oidcProviders:
- claimMappings:
groups:
claim: groups
prefix: 'oidc-groups-test:'
username:
claim: email
prefixPolicy: Prefix
prefix:
prefixString: 'oidc-user-test:'
issuer:
audiences:
- $AUDIENCE_1
- $AUDIENCE_2
issuerURL: $ISSUER_URL
name: microsoft-entra-id
oidcClients:
- clientID: $CONSOLE_CLIENT_ID
clientSecret:
name: $CONSOLE_CLIENT_SECRET_NAME
componentName: console
componentNamespace: openshift-console
type: OIDC
webhookTokenAuthenticator: null
"
Wait for KAS pods and console pods to complete rotation.
4. Test `oc login --exec-plugin=oc-oidc ...snipped...` and console login to the external OIDC. Both work well.
$ oc login --exec-plugin=oc-oidc --issuer-url=$ISSUER_URL --client-id=$CLI_CLIENT_ID --extra-scopes=email,profile --callback-port=8080
Logged into "https://api...snipped...:6443" as "oidc-user-test:xxia@redhat.com" from an external oidc issuer.
You don't have any projects. Contact your system administrator to request a project.
5. OAuth does not work any more which is expected by design.
$ oc login -u testuser-10 -p ...snipped...
error: couldn't get https://api....snipped...:6443/.well-known/oauth-authorization-server: unexpected response status 404
$ oc login -u kubeadmin -p ...snipped...
error: couldn't get https://api....snipped...:6443/.well-known/oauth-authorization-server: unexpected response status 404
$ oc login --token sha256~Z4NE...snipped...
error: The token provided is invalid or expired.
6. Check OAuth APIs and stuff:
$ oc get apiservices | grep -e user -e oauth
v1.oauth.openshift.io openshift-oauth-apiserver/api True 162m
v1.user.openshift.io openshift-oauth-apiserver/api True 162m
$ oc get po -n openshift-authentication
NAME READY STATUS RESTARTS AGE
oauth-openshift-744d78dd75-7fsdl 1/1 Running 0 140m
oauth-openshift-744d78dd75-7pwpd 1/1 Running 0 141m
oauth-openshift-744d78dd75-b6ngk 1/1 Running 0 140m
$ oc get po -n openshift-oauth-apiserver
NAME READY STATUS RESTARTS AGE
apiserver-64f859977c-2mgqw 1/1 Running 0 150m
apiserver-64f859977c-48lkb 1/1 Running 0 152m
apiserver-64f859977c-hcqhs 1/1 Running 0 151m
$ oc get user
NAME UID FULL NAME IDENTITIES
testuser-10 45186db0-c152-4f01-a3cb-c3a4e951f201 flexy-htpasswd-provider:testuser-10
$ oc get oauthaccesstoken
NAME USER NAME CLIENT NAME CREATED EXPIRES REDIRECT URI SCOPES
sha256~... testuser-10 openshift-challenging-client 129m 2024-11-14 02:32:03 +0000 UTC https://...snipped.../oauth/token/implicit user:full
7. Visit https://...snipped.../oauth/token/request in browser. It still shows the old kube:admin and htpasswd IDPs! Click htpasswd IDP and input user/password, it still shows "Display token". Click "Display token", it still shows below content though below oc login command does not work any more:
```
Your API token is
sha256~TW9W...snipped...
Log in with this token
oc login --token=sha256~TW9W...snipped... --server=https://api....snipped...:6443
...
```
8. Check more stuffs:
"integratedOAuthMetadata" still shows:
$ oc get authentication.config/cluster -o yaml
...
spec:
oauthMetadata:
name: ""
oidcProviders:
- claimMappings:
...
status:
integratedOAuthMetadata:
name: oauth-openshift
oidcClients:
...
"oauth" still shows:
$ oc get co authentication -o yaml | grep oauth
resource: oauths
name: oauth-openshift
name: oauth-openshift
name: openshift-oauth-apiserver
- name: oauth-apiserver
- name: oauth-openshift
Actual results:
See above step 6 ~ 8.
Expected results:
Given step 5 shows OAuth does not work any more, step 6 should remove original OAuth pods and OAuth APIs (apiservices) and/or resources like users/oauthaccesstokens. And step 7 should not have https://...snipped.../oauth/token/request still responsive and still show old IDPs and still return oauth tokens. And step 8 should remove all "oauth" traces in the YAML output.
Additional info: