-
Bug
-
Resolution: Unresolved
-
Normal
-
None
-
4.16.0
Description of problem:
If deleted the route referenced tls secret, route resource would failed with "spec.tls.externalCertificate: Not found: "secrets \"myroute-tls\" not found"" as expected. But if we re-create the secret with the same name, the route will not monitor the existence of the secret again. It will keep showing 'ExternalCertificateValidationFailed' and never return to normal, which is less user-friendly.
Version-Release number of selected component (if applicable):
OCP 4.16.0-0.nightly-2024-05-16-092402
How reproducible:
Always
Steps to Reproduce:
1. Enable the "ExternalRouteCertificate" feature gate $ oc patch featuregate/cluster --type=merge -p '{"spec":{"featureSet":"TechPreviewNoUpgrade"}}' 2. Create a route object from the "hello-openshift" demo app $ oc new-project test $ oc new-app openshift/hello-openshift $ oc create route edge myroute --service hello-openshift 3. Create the TLS secret for route to reference $ oc create secret tls myroute-tls --cert=tls.crt --key=tls.key 4. Patch the route object with created secret $ oc patch route myroute --type=merge --patch='{"spec":{"tls":{"externalCertificate":{"name":"myroute-tls"}}}}' 5. Check if the route is ready, and serve the TLS certs $ oc get route NAME HOST/PORT PATH SERVICES PORT TERMINATION WILDCARD myroute myroute-test.apps.ptalgulk-bug.qe.devcluster.openshift.com hello-openshift 8080-tcp edge None $ curl --cacert ./ca.crt https://myroute-test.apps.ptalgulk-bug.qe.devcluster.openshift.com Hello OpenShift! 6. Delete the TLS secret. Check the route status $ oc delete secret myroute-tls secret "myroute-tls" deleted $ oc describe route myroute ... Requested Host: myroute-test.apps.ptalgulk-bug.qe.devcluster.openshift.com rejected by router default: (host router-default.apps.ptalgulk-bug.qe.devcluster.openshift.com)ExternalCertificateValidationFailed (57 minutes ago) spec.tls.externalCertificate: Not found: "secrets \"myroute-tls\" not found" ... $ oc get route myroute -o yaml ... status: ingress: - conditions: - lastTransitionTime: "2024-05-20T09:37:17Z" message: 'spec.tls.externalCertificate: Not found: "secrets \"myroute-tls\" not found"' reason: ExternalCertificateValidationFailed status: "False" type: Admitted ... 7. Re create the TLS secret manually $ oc create secret tls myroute-tls --cert=tls.crt --key=tls.key 8. Check the route status, it still failed with "ExternalCertificateValidationFailed" message after ~30m $ oc get route NAME HOST/PORT PATH SERVICES PORT TERMINATION WILDCARD myroute ExternalCertificateValidationFailed hello-openshift 8080-tcp edge None
Actual results:
route "myroute" keep failed with "ExternalCertificateValidationFailed"
Expected results:
route "myroute" should back to normal because the secret is re-created.
Additional info:
router pod logs: ... I0520 09:28:03.398627 1 monitor.go:76] starting informer I0520 09:28:03.399864 1 reflector.go:351] Caches populated for *v1.Secret from github.com/openshift/library-go/pkg/secret/monitor.go:79 I0520 09:28:03.498762 1 secret_monitor.go:122] secret informer started item key {test myroute-tls} I0520 09:28:03.498790 1 secret_monitor.go:135] secret handler added item key {test myroute-tls} I0520 09:28:03.498798 1 manager.go:80] secret manager registered route for key test/myroute with secret myroute-tls I0520 09:28:03.638120 1 router.go:669] "msg"="router reloaded" "logger"="template" "output"=" - Checking http://localhost:80 using PROXY protocol ...\n - Health check ok : 0 retry attempt(s).\n" I0520 09:33:00.162473 1 router.go:669] "msg"="router reloaded" "logger"="template" "output"=" - Checking http://localhost:80 using PROXY protocol ...\n - Health check ok : 0 retry attempt(s).\n" I0520 09:37:17.820979 1 secret_monitor.go:165] secret handler removed item key{test myroute-tls} I0520 09:37:17.821018 1 monitor.go:93] informer stopped I0520 09:37:17.821023 1 secret_monitor.go:174] secret informer stopped item key {test myroute-tls} I0520 09:37:17.821028 1 manager.go:108] secret manager unregistered route for key test/myroute E0520 09:37:17.845149 1 route_secret_manager.go:124] "msg"="skipping route due to invalid externalCertificate configuration" "error"="spec.tls.externalCertificate: Not found: \"secrets \\\"myroute-tls\\\" not found\"" "logger"="controller" "namespace"="test" "route"="myroute" E0520 09:37:17.845191 1 router_controller.go:273] spec.tls.externalCertificate: Not found: "secrets \"myroute-tls\" not found" E0520 09:37:17.862319 1 route_secret_manager.go:124] "msg"="skipping route due to invalid externalCertificate configuration" "error"="spec.tls.externalCertificate: Not found: \"secrets \\\"myroute-tls\\\" not found\"" "logger"="controller" "namespace"="test" "route"="myroute" E0520 09:37:17.862358 1 router_controller.go:273] spec.tls.externalCertificate: Not found: "secrets \"myroute-tls\" not found" ... It seems that the informer stopped once the secret was deleted and didn't start again when the secret was re-created.
Workaround:
Recreate the route when the secret is recreated.