Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-60877

'oc adm inspect-alerts' and relatives should respect --certificate-authority

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Normal Normal
    • 4.20.0
    • 4.14, 4.19, 4.20
    • oc
    • None
    • Quality / Stability / Reliability
    • False
    • Hide

      None

      Show
      None
    • None
    • Low
    • None
    • None
    • None
    • In Progress
    • Release Note Not Required
    • None
    • None
    • None
    • None
    • None

      Description of problem

      Current oc adm inspect-alerts (and other consumers of that code, including oc adm upgrade recommend and oc adm upgrade status) do not respect --certificate-authority or --insecure-skip-tls-verify. This isn't a problem in production, because docs say:

      Warning
      The Ingress Operator generates a default certificate for an Ingress Controller to serve as a placeholder until you configure a custom default certificate. Do not use operator-generated default certificates in production clusters.

      But it does make testing in throw-away CI clusters difficult.

      Version-Release number of selected component

      All release since inspect-alert was created in 4.16.

      How reproducible

      Every time.

      Steps to Reproduce

      Create a cluster-bot or other CI cluster with a default, self-signed Ingress certificate.

      Collect certificates used by the Kube API server and ingress into a local file:

      $ oc -n openshift-kube-apiserver get -o json secrets | jq -r '.items[] | select(.type == "kubernetes.io/tls").data["tls.crt"] | @base64d' >ca.crt
      $ oc -n openshift-ingress get -o json secrets | jq -r '.items[] | select(.type == "kubernetes.io/tls").data["tls.crt"] | @base64d' >>ca.crt
      

      Get a token, because platform Thanos doesn't currently support cert-based auth:

      Important
      You can only use bearer token authentication to access the Thanos Querier API route.

      $ TOKEN="$(oc -n openshift-cluster-version create token default)"
      

      Try and access Thanos alerts:

      $ OC_ENABLE_CMD_INSPECT_ALERTS=true oc --token "${TOKEN}" --certificate-authority ca.crt adm inspect-alerts
      $ OC_ENABLE_CMD_UPGRADE_RECOMMEND=true OC_ENABLE_CMD_UPGRADE_RECOMMEND_PRECHECK=true oc --token "${TOKEN}" --certificate-authority ca.crt adm upgrade recommend
      $ OC_ENABLE_CMD_UPGRADE_STATUS=true oc --token "${TOKEN}" --certificate-authority ca.crt adm upgrade status
      

      Actual results

      All three subcommands will mention something like:

      error: failed to get alerts from Thanos: unable to get /api/v1/alerts from URI in the openshift-monitoring/thanos-querier Route: thanos-querier-openshift-monitoring.apps.ci-ln-kljt4pt-76ef8.aws-2.ci.openshift.org->Get "https://thanos-querier-openshift-monitoring.apps.ci-ln-kljt4pt-76ef8.aws-2.ci.openshift.org/api/v1/alerts": tls: failed to verify certificate: x509: certificate signed by unknown authority
      

      except for status, which needs to be mid-update to avoid failing earlier on The cluster is not updating..

      Expected results

      Commands that get far enough to attempt Thanos access should succeed in that access. For example:

      $ OC_ENABLE_CMD_INSPECT_ALERTS=true ./oc --token "${TOKEN}" --certificate-authority ca.crt adm inspect-alerts
      {"status":"success",...
      

              trking W. Trevor King
              trking W. Trevor King
              None
              None
              Ying Zhou Ying Zhou
              None
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Created:
                Updated: