Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-44196

kube-apiserver revision rollout causes master node down situation

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Major Major
    • None
    • 4.12.z
    • kube-apiserver
    • Important
    • None
    • False
    • Hide

      None

      Show
      None

      Description of problem:

      kube-apiserver revision rollout causes master node down situation
      When the revision rollout of the kube-apiserver is triggered, the spike in resource utilization on the master nodes is seen and it causes master nodes to become inaccesible.
      ~~~
      X triggers revision update for the API-servers -> rolling update triggered as outcome --> during the rolling update the are peaks of requests and WATCH events towards the remaining available replica and probably causing the resource consumption within the system-load as well ->*WE ASSUME*: the graceperiod of the shutdown sig is not enough caused by the "good amount" of requests and the peak which creates LOCK or "ungracefully" shutdown and prevent from the control-plane to recover + host is dead due the system-load peak which leaving the masters in "dead" state.
      ~~~

      Version-Release number of selected component (if applicable):

          

      How reproducible:

          Reproducible in customer environment

      Steps to Reproduce:

          1. Revision rollout of the kube-apiserver is triggered
          2. System load peaks on master nodes 
          3. The master nodes become inaccessible      

      Actual results:

          The revision rollout of kube-apiserver should complete without issues

      Expected results:

          Master nodes becoming inaccessible dueing the revision rollout

      Additional info:

          

              Unassigned Unassigned
              rhn-support-alosingh Alok Singh
              Ke Wang Ke Wang
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Created:
                Updated: