Description of problem: In one upgrading environment  (2.7.8 --> 2.8.2 --> 2.9.0)，some automation test cases are failed, checked the failures. They deleted cert secrts

"observability-server-ca-certs"
"observability-client-ca-certs"
"observability-server-certs"
"observability-grafana-certs"
The expected result after deleting the above secrets:
1. these secrets are re-generated
2. some pods on the hub should be re-started, like observability-observatorium-api, observability-rbac-query-proxy
3. metrics collector pod should be re-started from the managed cluster
4. metrics collector could forward data to the hub successfully
But all the above 4 does not happen, and metrics collect log shows:
```
level=error caller=logger.go:55 ts=2023-10-08T06:16:07.929939272Z component=forwarder/worker msg="unable to forward results" err="response status code is 500 Internal Server Error, response body is x509: certificate signed by unknown authority (possibly because of \"crypto/rsa: verification error\" while trying to verify candidate authority certificate \"observability-client-ca-certificate\")\n"
```
 

Version-Release number of selected component (if applicable): 2.9 Train-7 DS image

How reproducible:

Steps to Reproduce:

1. upgrade hub from 2.7.8 --> 2.8.2 --> 2.9.0
2. delete cert secrets as the above list
3. these secrets are not re-generated
4. observability-observatorium-api and observability-rbac-query-proxy pods are not re-started on the hub
5. metrics collector pod is not re-started, and reports error

Actual results:

Expected results:

Additional info: