Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-62869

HyperShift ServiceMonitor for cluster-version-operator metrics: revert client auth

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Major Major
    • None
    • 4.18.z, 4.19.z
    • Cluster Version Operator
    • None
    • Quality / Stability / Reliability
    • False
    • Hide

      None

      Show
      None
    • None
    • None
    • None
    • None
    • None
    • None
    • In Progress
    • Bug Fix
    • Hide
      In 4.18.23, the cluster-version operator began requiring bearer token authentication in metrics requests. That broke HyperShift/Hosted clusters, were the metrics scraper currently provides no client authentication. With this release, the cluster-version operator returns to not requiring client authentication for metrics requests, recovering access to cluster-version operator metrics on HyperShift clusters.
      Show
      In 4.18.23, the cluster-version operator began requiring bearer token authentication in metrics requests. That broke HyperShift/Hosted clusters, were the metrics scraper currently provides no client authentication. With this release, the cluster-version operator returns to not requiring client authentication for metrics requests, recovering access to cluster-version operator metrics on HyperShift clusters.
    • None
    • None
    • None
    • None

      This is a clone of issue OCPBUGS-62868. The following is the description of the original issue:
      —
      This is a clone of issue OCPBUGS-62867. The following is the description of the original issue:
      —
      This is a clone of issue OCPBUGS-62861. The following is the description of the original issue:
      —

      Description of problem:

      When installing a 4.18 or 4.19 cluster in ROSA HCP, metrics scraping of the cluster-version-operator fails. It is failing because CVO requires bearer token based authentication of the /metrics endpoint now.

       

       [deads2k]Impact: on HCP it is now impossible to determine the working/failing state of HostedCluster operators using the monitoring stack.  There is no known scalable alternative.  https://github.com/openshift/cluster-version-operator/pull/1223, https://github.com/openshift/cluster-version-operator/pull/1222, and https://github.com/openshift/cluster-version-operator/pull/1215. Introduced this problem.

      Version-Release number of selected component

      4.20, 4.19.9 and later, and 4.18.23 and later.

      How reproducible

      Every time

      Steps to Reproduce

      1. Install the latest 4.18 or 4.19 cluster
      2. Check the openshift-observability-operator stack looking for cluster_operator_up, cluster_version, or other CVO-served metrics.

      Actual results

      Target scraping failing with 401s.

      Expected results

      Successfully-scraped CVO metrics like cluster_operator_up.

      Additional info

      I tried to utilize some of the TLS certs in the HCP namespace, like metrics-client, but these fail primarily because CVO is requiring a service account token.

              trking W. Trevor King
              jbranham.openshift Josh Branham
              None
              None
              Jia Liu Jia Liu
              None
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Created:
                Updated: