Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-76588

Kube API Server with two service account issuers during the first 24h of cluster's lifetime

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Undefined Undefined
    • None
    • 4.19.z
    • kube-apiserver
    • None
    • False
    • Hide

      None

      Show
      None
    • None
    • None
    • None
    • In Progress
    • Bug Fix
    • Do not setup default service account issuer if authentications.config.openshift.io resource with a name cluster has a .spec.serviceAccountIssuer present during install time.
    • None
    • None
    • None
    • None

      Description of problem:

      - OCP 4.19.23 was installed and short lived credentials using AWS STS Manual Mode [1] were configured. 
      - The cluster gets installed successfully and it's healthy, however the Kube API Server has 2 service account issuers being one the default with 24h expiration time (kubernetes.default.svc): that means that in the first 24h of the cluster's lifetime, they can't deploy any application because the token would return two audiences.

      [1] - https://github.com/openshift/cloud-credential-operator/blob/master/docs/sts.md

      Version-Release number of selected component (if applicable):

          4.19.23

      Steps to Reproduce:

          1. Deploy a cluster with custom service account issuer.
      
          2. During the first 24h, two issuers are listed in the Kube API Server (and this is expected as per source code [2] to ensure a smooth transition between issuers):
      
      $ oc get kubeapiserver cluster -oyaml | yq .status.serviceAccountIssuers
      [
        {
          "name": "<custom issuer>"
        },
        {
          "expirationTime": "2026-02-04T13:51:57Z",
          "name": "https://kubernetes.default.svc"
        }
      ]
      
          3. Requesting a token return two audiences:
      $ export JWT=$(oc create token test-sa -n <ns>)
      $ jq -R -r 'split(".") | .[1] | @base64d | fromjson ' <<< "$JWT"
      {
        "aud": [
          "https://<custom issuer>",
          "https://kubernetes.default.svc"
        ]
      
          4. No application can be deployed during the first 24 hours of the cluster's life.
      
          5. Specifying the --audience option in the oc create token avoids the issue, but other custom applications can not specify the audience when requesting the token, therefore failing until the default issuer expires.
           

       [2] - https://github.com/openshift/cluster-kube-apiserver-operator/blob/release-4.19/pkg/operator/serviceaccountissuercontroller/serviceaccountissuer_controller.go

      Actual results:

          Default issuer is present during the cluster's first 24 hours even with a custom issuer defined.

      Expected results:

          When installing the cluster with a custom service account issuer, the default issuer should not be present at all.

              fkrepins@redhat.com Filip Krepinsky
              rhn-support-palmeida Patrick Almeida
              Ke Wang Ke Wang
              None
              Votes:
              1 Vote for this issue
              Watchers:
              7 Start watching this issue

                Created:
                Updated: