Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-15711

Seeing abnormal message in kubeapiserver.operator on SNO after upgrade

XMLWordPrintable

    • Moderate
    • Yes
    • False
    • Hide

      None

      Show
      None

      Description of problem:

      Regarding upgrade SNO clusters given kube-apiserver implements startup-monitor mechanism, when we upgrade 4.10 to 4.11, after completion of upgrade we see abnormal message for lastFailedRevisionErrors in KAS operator. We did not see any cluster operators degraded or abnormal operators status. We should not get these lastFailedRevisionErrors message in KAS operator when there is no issue on cluster after upgrade.

      Version-Release number of selected component (if applicable):

      4.11.44-x86_64 on upi-on-baremetal with packet-sno_bip-etcd_encryption-realtime_kernel-ci profile

      How reproducible:

      Always

      Steps to Reproduce:

      1. Install 4.10 sno cluster and upgrade to 4.11(upgrade path -4.10.63-x86_64 - > 4.11.44-x86_64)
       2. After completion of upgrade.
      3. Run oc get kubeapiserver.operator cluster -o jsonpath='{.status.nodeStatuses[?(@.lastFailedRevisionErrors)]}' 

      Actual results:

      {"currentRevision":9,"lastFailedCount":1,"lastFailedReason":"InstallerFailed","lastFailedRevision":3,"lastFailedRevisionErrors":["installer: ,\n CertSecretNames: ([]string) (len=10 cap=16) {\n  (string) (len=17) \"aggregator-client\",\n  (string) (len=30) \"localhost-serving-cert-certkey\",\n  (string) (len=31) \"service-network-serving-certkey\",\n  (string) (len=37) \"external-loadbalancer-serving-certkey\",\n  (string) (len=37) \"internal-loadbalancer-serving-certkey\",\n  (string) (len=33) \"bound-service-account-signing-key\",\n  (string) (len=40) \"control-plane-node-admin-client-cert-key\",\n  (string) (len=31) \"check-endpoints-client-cert-key\",\n  (string) (len=14) \"kubelet-client\",\n  (string) (len=16) \"node-kubeconfigs\"\n },\n OptionalCertSecretNamePrefixes: ([]string) (len=11 cap=16) {\n  (string) (len=17) \"user-serving-cert\",\n  (string) (len=21) \"user-serving-cert-000\",\n  (string) (len=21) \"user-serving-cert-001\",\n  (string) (len=21) \"user-serving-cert-002\",\n  (string) (len=21) \"user-serving-cert-003\",\n  (string) (len=21) \"user-serving-cert-004\",\n  (string) (len=21) \"user-serving-cert-005\",\n  (string) (len=21) \"user-serving-cert-006\",\n  (string) (len=21) \"user-serving-cert-007\",\n  (string) (len=21) \"user-serving-cert-008\",\n  (string) (len=21) \"user-serving-cert-009\"\n },\n CertConfigMapNamePrefixes: ([]string) (len=4 cap=4) {\n  (string) (len=20) \"aggregator-client-ca\",\n  (string) (len=9) \"client-ca\",\n  (string) (len=29) \"control-plane-node-kubeconfig\",\n  (string) (len=26) \"check-endpoints-kubeconfig\"\n },\n OptionalCertConfigMapNamePrefixes: ([]string) (len=1 cap=1) {\n  (string) (len=17) \"trusted-ca-bundle\"\n },\n CertDir: (string) (len=57) \"/etc/kubernetes/static-pod-resources/kube-apiserver-certs\",\n ResourceDir: (string) (len=36) \"/etc/kubernetes/static-pod-resources\",\n PodManifestDir: (string) (len=25) \"/etc/kubernetes/manifests\",\n Timeout: (time.Duration) 2m0s,\n StaticPodManifestsLockFile: (string) \"\",\n PodMutationFns: ([]installerpod.PodMutationFunc) \u003cnil\u003e,\n KubeletVersion: (string) \"\"\n})\nF0703 05:23:22.869315       1 cmd.go:106] Get \"https://172.30.0.1:443/api/v1/namespaces/openshift-kube-apiserver/pods?labelSelector=app%3Dinstaller\": net/http: TLS handshake timeout\n"],"lastFailedTime":"2023-07-03T05:24:37Z","nodeName":"master-00.rgangwar-39789.qe.devcluster.openshift.com"

      Expected results:

      Should be empty not see this error

      Additional info:

      https://drive.google.com/file/d/1O2qw0EksdB-NUSjmyjAh2wGb7p1S_FEi/view?usp=sharing

            Unassigned Unassigned
            rhn-support-rgangwar Rahul Gangwar
            Rahul Gangwar Rahul Gangwar
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: