Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-45042

The console target in monitoring shows down after BYO external oidc is configured in HCP or OCP

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Duplicate
    • Icon: Undefined Undefined
    • None
    • premerge
    • Management Console
    • None
    • Quality / Stability / Reliability
    • False
    • Hide

      None

      Show
      None
    • None
    • Critical
    • None
    • None
    • None
    • Rejected
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Description of problem:
      This is a bug found during pre-merge test of 4.18 epic AUTH-528 PRs and filed for better tracking per existing "OpenShift - Testing Before PR Merges - Left-Shift Testing" google doc workflow.

      The console target in monitoring shows down after OCP BYO external oidc is configured
      

      Version-Release number of selected component (if applicable):
      Cluster-bot build with below, which is built at 2024-11-26 09:55 CST (UTC+800)

      build 4.18,openshift/cluster-authentication-operator#713,openshift/cluster-authentication-operator#740,openshift/cluster-kube-apiserver-operator#1760,openshift/console-operator#940
          

      How reproducible:

      Always
          

      Steps to Reproduce:

      1. Launch a TechPreviewNoUpgrade standalone OCP cluster with above build.
      
      2. Login to console Administrator page, click left Observe -> Targets, click "Filter", all show "Up".
      
      Meantime, check from CLI, the target returns 200, good:
      $ oc rsh -n openshift-monitoring prometheus-k8s-0
      sh-5.1$ curl -I -k -H "Authorization: Bearer `cat /var/run/secrets/kubernetes.io/serviceaccount/token`" https://console.openshift-console.svc/metrics
      HTTP/1.1 200 OK
      
      3. Configure BYO external OIDC in this OCP cluster using keycloak. KAS and console pods can roll out successfully. oc login and console login to keycloak can succeed.
      
      4. In console Administrator home page, below is seen:
        TargetDown
        Nov 26, 2024, 2:45 PM
        100% of the console/console targets in openshift-console namespace have been unreachable for more than 15 minutes. This may be a symptom of network connectivity issues, down nodes, or failures within these components. Assess the health of the infrastructure and nodes running these targets and then contact support.
      
      Click left Observe -> Targets, click "Filter", the console target shows "Down". Click it, it shows:
      Scrape failed
      server returned HTTP status 401 Unauthorized
      
      Meantime, check from CLI, the target returns bad now:
      $ oc rsh -n openshift-monitoring prometheus-k8s-0
      sh-5.1$ curl -I -k -H "Authorization: Bearer `cat /var/run/secrets/kubernetes.io/serviceaccount/token`" https://console.openshift-console.svc/metrics
      HTTP/1.1 401 Unauthorized
      

      Actual results:

      Step 4, The console target shows Down exactly after OCP BYO external oidc is configured, as above details.

      Expected results:

      Should not become Down.

      Additional info:

          

              rh-ee-jonjacks Jon Jackson
              xxia-1 Xingxing Xia
              None
              None
              Yanping Zhang Yanping Zhang
              None
              Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

                Created:
                Updated:
                Resolved: