Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-55024

1 of 3 kube-apiservers doesn't reach revision in time in Console job

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Undefined Undefined
    • None
    • 4.19
    • Management Console
    • Quality / Stability / Reliability
    • False
    • Hide

      None

      Show
      None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Description of problem:

      : operator conditions kube-apiserver

    {Operator progressing (NodeInstaller): NodeInstallerProgressing: 1 node is at revision 5; 2 nodes are at revision 7  Operator progressing (NodeInstaller): NodeInstallerProgressing: 1 node is at revision 5; 2 nodes are at revision 7}

      Version-Release number of selected component (if applicable):

      4.19 / e2e-aws-techpreview / periodics-e2e-aws

      How reproducible:

      3-5% regression from 100% reported by sippy (component readiness)

      Steps to Reproduce:

          1. Go to sippy and component readiness
    2. Pick red cube in the kube-apiserver row / column
    3. Select failing test cases

      Actual results:

          oc -n openshift-kube-apiserver get pods -ojson | jq '.items[] | "\(.metadata.name) [ \(.metadata.labels.revision) ]"' | rg -v null 
"kube-apiserver-ip-10-0-34-150.us-west-2.compute.internal [ 9 ]"
"kube-apiserver-ip-10-0-43-223.us-west-2.compute.internal [ 9 ]"
"kube-apiserver-ip-10-0-71-241.us-west-2.compute.internal [ 7 ]"

      Expected results:

           oc -n openshift-kube-apiserver get pods -ojson | jq '.items[] | "\(.metadata.name) [ \(.metadata.labels.revision) ]"' | rg -v null 
"kube-apiserver-ip-10-0-34-150.us-west-2.compute.internal [ 9 ]"
"kube-apiserver-ip-10-0-43-223.us-west-2.compute.internal [ 9 ]"
"kube-apiserver-ip-10-0-71-241.us-west-2.compute.internal [ 9 ]"

      Additional info:

      The revision change happens eventually, but it looks like the gathering of data starts already earlier and reports the failing test case?

      Not sure why this happens now. If the roll-out takes longer or if a timeout decreased.

      - 2025-04-04 04:34:09 installer finishes writing the pod manifest to disk, no further logs
- 2025-04-04 04:34:09 kube-apiserver receives SIGTERM
- 2025-04-04 04:34:34 kube-apiserver process is shutting down, no further logs
- 2025-04-04 04:36:24 kubelet reports container started
- 2025-04-04 04:36:29 kubelet reports container ready
- 2025-04-04 04:37:11 openshift-kube-apiserver-operator reports 3 nodes are at revision 8

      Links:

              jhadvig@redhat.com Jakub Hadvig
              kostrows@redhat.com Krzysztof Ostrowski
              None
              None
              Ke Wang Ke Wang
              None
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Created:
                Updated: