$ oc explain servicemonitor.spec.endpoints.bearerTokenFile
GROUP:      monitoring.coreos.com
KIND:       ServiceMonitor
VERSION:    v1

FIELD: bearerTokenFile <string>


DESCRIPTION:
    File to read bearer token for scraping the target.
    
    Deprecated: use `authorization` instead. 

It is used in this manifest.

Prometheus uses the token from the file to send the request to scrape CVO and CVO uses the token to do the authentication and authorization for the request.

We want to use servicemonitor.spec.endpoints.authorization instead.

A potential solution to groom:

When CVO starts the HTTP server for the metrics endpoint, it generates a token and store it as the value for the key token in the secret/cluster-version-operator-client in ns/openshift-monitoring. (It is not clear in the API doc which namespace it should be, either openshift-monitoring or the namesapce where the service monitor stays: openshift-cluster-version. We will ask the monitoring team about this.).

Then 

spec:
  endpoints:
  - authorization:
      type: Bearer
      credentials:
        key: token
        name: cluster-version-operator-client
    interval: 30s
    port: metrics
    scheme: https
    tlsConfig:
      caFile: /etc/prometheus/configmaps/serving-certs-ca-bundle/service-ca.crt
      serverName: cluster-version-operator.openshift-cluster-version.svc 

When it serves the request, it validates to token for authentication/authorization.

We can use JTW to generate and validate a token based on some bytes which can be a random bytes or taken from any existing secret in ns/openshift-cluster-version: e.g.,

oc get secret -n openshift-cluster-version cluster-version-operator-serving-cert -o yaml | yq '.data."tls.key"' 

If we want to go with JTW, we can reuse the code here.

JTW is efficient enough and does not rely on K8S API server. So with this card implemented, we do not have to implement "Client certificate scraping". There should be example components doing that already.

https://rhobs-handbook.netlify.app/products/openshiftmonitoring/collecting_metrics.md/#kube-rbac-proxy-sidecar

It is also mentioned that CVO could switch to github.com/openshift/library-go/pkg/controller/controllercmd and then we get the auth on the metrics endpoint for free.

Additional info

The auth protection is disabled on HyperShift OTA-1705.

When we implement this card, it should work on a HyperShift cluster as well. Or at least it does not break HyperShift clusters.