Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-62868

HyperShift ServiceMonitor for cluster-version-operator metrics: revert client auth

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Major Major
    • 4.19.z
    • 4.18.z, 4.19.z
    • Cluster Version Operator
    • None
    • Quality / Stability / Reliability
    • False
    • Hide

      None

      Show
      None
    • None
    • None
    • None
    • None
    • None
    • None
    • In Progress
    • Bug Fix
    • Hide
      * Before this update, the Cluster Version Operator (CVO) in 4.19.9 and 4.18.23 started to require bearer token authentication in metrics requests. As a consequence, HyperShift and Hosted clusters were broken because the metrics scraper did not provide client authentication. With this release, the CVO does not require client authentication for metrics requests. As a result, access to CVO metrics scraping is recovered on HyperShift and Hosted clusters. (link:https://issues.redhat.com/browse/OCPBUGS-62868[OCPBUGS-62868])
      Show
      * Before this update, the Cluster Version Operator (CVO) in 4.19.9 and 4.18.23 started to require bearer token authentication in metrics requests. As a consequence, HyperShift and Hosted clusters were broken because the metrics scraper did not provide client authentication. With this release, the CVO does not require client authentication for metrics requests. As a result, access to CVO metrics scraping is recovered on HyperShift and Hosted clusters. (link: https://issues.redhat.com/browse/OCPBUGS-62868 [ OCPBUGS-62868 ])
    • None
    • None
    • None
    • None

      This is a clone of issue OCPBUGS-62867. The following is the description of the original issue:
      —
      This is a clone of issue OCPBUGS-62861. The following is the description of the original issue:
      —

      Description of problem:

      When installing a 4.18 or 4.19 cluster in ROSA HCP, metrics scraping of the cluster-version-operator fails. It is failing because CVO requires bearer token based authentication of the /metrics endpoint now.

       

       [deads2k]Impact: on HCP it is now impossible to determine the working/failing state of HostedCluster operators using the monitoring stack.  There is no known scalable alternative.  https://github.com/openshift/cluster-version-operator/pull/1223, https://github.com/openshift/cluster-version-operator/pull/1222, and https://github.com/openshift/cluster-version-operator/pull/1215. Introduced this problem.

      Version-Release number of selected component

      4.20, 4.19.9 and later, and 4.18.23 and later.

      How reproducible

      Every time

      Steps to Reproduce

      1. Install the latest 4.18 or 4.19 cluster
      2. Check the openshift-observability-operator stack looking for cluster_operator_up, cluster_version, or other CVO-served metrics.

      Actual results

      Target scraping failing with 401s.

      Expected results

      Successfully-scraped CVO metrics like cluster_operator_up.

      Additional info

      I tried to utilize some of the TLS certs in the HCP namespace, like metrics-client, but these fail primarily because CVO is requiring a service account token.

              trking W. Trevor King
              jbranham.openshift Josh Branham
              None
              None
              Jia Liu Jia Liu
              None
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Created:
                Updated:
                Resolved: