Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-12846

Multiple pod restarts as part of operator updates that include webhooks

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Won't Do
    • Icon: Normal Normal
    • None
    • 4.12
    • OLM / OperatorHub
    • Important
    • No
    • Rejected
    • False
    • Hide

      None

      Show
      None

      Description of problem:

      Multiple pod restarts as part of operator updates that include webhooks

      Version-Release number of selected component (if applicable):

       

      How reproducible:

       

      Steps to Reproduce:

      A new v7.2 operator pod starts and goes into creating
      A New v7.2 webhook appears which contains the same caBundle value as the v7.1 wehbook
      At this point requests are still going to the old v7.1 operator pod and working
      
      New v7.2 operator pod becomes ready
      Old v7.1 oprator is terminated
      Old 7.1 webhook is deleted
      At this point requests are going to the new v7.2 operator pod and working
      
      Within a few seconds the following occurs (all within a ms of each other so we don't know cause vs effect)
      
      A second new v7.2 operator pod is created. Looking at the deployment it appears this gets created because the pod spec olmcahash has been udpated
      The ibm-appconnect-operator-service-cert secret updated
      The v7.2 webhook updated with a new caBundle value
      At this point all requests fail. We suspect this is because Kube is because all the requests are now made using the new caBundle, but they are hitting the original v7.2 pod which still has the old caBundle thus it fails. 

      Actual results:

      there is a window where requests fail i.e. its not a blue/green upgrade.

      Expected results:

      Request should not fail and there should not any impact while upgrading

      Additional info:

      https://issues.redhat.com/browse/RFE-3796

              krizza@redhat.com Kevin Rizza
              rhn-support-hjaiswal Himanshu Jaiswal
              Kui Wang Kui Wang
              Votes:
              1 Vote for this issue
              Watchers:
              6 Start watching this issue

                Created:
                Updated:
                Resolved: