-
Bug
-
Resolution: Done
-
Critical
-
DO378 - RHBoQ2.13-en-1-20230228, DO313 - RHSSO7.6-en-1-20230314, DO288 - OCP4.12-en-3-20231031, DO288 - OCP4.10-en-4-20220915, DO280 - OCP4.12-en-2-20230707, DO280 - OCP4.10-en-2-20220811, DO188 - RHOSCP4.12-en-3-20231031, DO188 - RHOSCP4.10-en-1-20220923, DO180 - OCP4.12-en-2-20230825, AD221 - RHF7.10-en-6-20230613
-
DO378 - RHBoQ2.13-en-1-20230228, DO313 - RHSSO7.6-en-1-20230314, DO288 - OCP4.12-en-2-20231002, DO288 - OCP4.10-en-4-20220915, DO280 - OCP4.12-en-2-20230707, DO280 - OCP4.10-en-2-20220811, DO180 - OCP4.12-en-2-20230825, AD221 - RHF7.10-en-6-20230613
-
None
-
False
-
-
False
-
-
-
en-US (English)
Issue description
After the lab environment images are over 90 days old, the node certificate for the kube-apiserver must be renewed. In some classrooms, this renewal does not properly finish, leaving the cluster completely unusable. The symptoms of this issue are the kube-apiserver operator getting stuck upgrading or the authentication service throwing TLS errors.
Steps to reproduce:
Start a lab in one of the affected courses.
Workaround:
Force the cert renewal using the following command (ssh to utility VM from workstation and then run it on utility):
ssh lab@utility
oc get secret -A -o json | jq -r '.items[] | select(.metadata.annotations."auth.openshift.io/certificate-not-after" | .!=null and fromdateiso8601<='$( date --date='+1year' +%s )') | "-n \(.metadata.namespace) \(.metadata.name)"' | xargs -n3 oc patch secret -p='{"metadata": {"annotations": {"auth.openshift.io/certificate-not-after": null}}}'
Expected result:
Within 5 minutes, the API server will restart with the correctly renewed certificate, and the cluster will be healthy again. Use the wait.sh script on utility to verify this.