Description of problem:

When installing a 4.18 or 4.19 cluster in ROSA HCP, metrics scraping of the cluster-version-operator fails. It is failing because CVO requires bearer token based authentication of the /metrics endpoint now.

 [deads2k]Impact: on HCP it is now impossible to determine the working/failing state of HostedCluster operators using the monitoring stack.  There is no known scalable alternative.  https://github.com/openshift/cluster-version-operator/pull/1223, https://github.com/openshift/cluster-version-operator/pull/1222, and https://github.com/openshift/cluster-version-operator/pull/1215. Introduced this problem.

Version-Release number of selected component

4.20, 4.19.9 and later, and 4.18.23 and later.

How reproducible

Every time

Steps to Reproduce

1. Install the latest 4.18 or 4.19 cluster
2. Check the openshift-observability-operator stack looking for cluster_operator_up, cluster_version, or other CVO-served metrics.

Actual results

Target scraping failing with 401s.

Expected results

Successfully-scraped CVO metrics like cluster_operator_up.

Additional info

I tried to utilize some of the TLS certs in the HCP namespace, like metrics-client, but these fail primarily because CVO is requiring a service account token.