In HyperShift, the kube-scheduler for a HCP namespace is not configured in a way that allows for metrics to be scraped. This also means HyperShift does not lay down a Service/PodMonitor allowing the Management Cluster's observability stack to pick it up.

I attempted to manually do this by disabling reconciliation of the HCP, then exposing the port like the following:

         name: kube-scheduler
+        ports:
+        - containerPort: 10259
+          protocol: TCP 

Besides exposing the port, we would also need to validate that the TLS auth HyperShift configures will be accepted by the /metrics endpoint.

I created a PodMonitor like this, however TLS fails. I am unsure if we have a TLS cert to auth against what kube-scheduler trusts.

apiVersion: monitoring.rhobs/v1
kind: PodMonitor
metadata:
  name: kube-scheduler
  namespace: ocm-staging-2lu71ffnbooql8hb213jqr40rms7dsob-jb419
spec:
  namespaceSelector:
    matchNames:
      - ocm-staging-2lu71ffnbooql8hb213jqr40rms7dsob-jb419
  podMetricsEndpoints:
    - interval: 60s
      path: /metrics
      targetPort: 10259
      scheme: https
      tlsConfig:
        ca:
          configMap:
            key: ca.crt
            name: root-ca
        cert:
          secret:
            key: tls.crt
            name: metrics-client
        keySecret:
          key: tls.key
          name: metrics-client
        serverName: localhost
  selector:
    matchLabels:
      app: kube-scheduler
      hypershift.openshift.io/control-plane-component: kube-scheduler 

Importance:

ROSA would like to monitor kube-scheduler out of the box, and even if we went the route of laying down our own custom PodMonitor, it would take Deployment modifications to make that work.

I am happy to implement some of this if someone wants to provide a little context on how to do this outside of just updating the YAML.