Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-50500

ovnkube-controller fails under load during control-plane only upgrade from 4.17 to 4.18 after successful upgrade from 4.16

XMLWordPrintable

    • Yes
    • Approved
    • True
    • Hide

      None

      Show
      None

      OpenShift control-plane only upgrade (formerly EUS-to-EUS upgrade) fails while updating networking cluster operator to 4.18.0-rc.7 while under load.

      DaemonSet "/openshift-ovn-kubernetes/ovnkube-node" rollout is not making progress - last change 2025-02-09T03:23:25Z

      4.16.30 -> 4.17.14 successful control-plane only upgrade.

      4.17.14 -> 4.18.0-rc.7 failed control-plane only upgrade.

      As documented, worker machine config pool is paused for the duration of the control-plane only upgrade.

      From ovnkube-controller log:

       

      DeltaFIFO Pop Process" ID:default/virt-launcher-vm-instancetype-cirros-test-0354-bx4qw,Depth:3895,Reason:slow event handlers blocking the queue
...
F0209 10:31:31.434864 2151931 ovnkube.go:137] failed to run ovnkube: [failed to start network controller: error in syncing cache for *v1.Pod informer, failed to start node network controller: error in syncing cache for *v1.Pod informer]

      The cluster does respond to commands within a reasonable time frame and is currently running 2000 VMs with an internal stress-ng simulated load.

       

              trozet@redhat.com Tim Rozet
              rhn-support-sbennert Sarah Bennert
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

                Created:
                Updated:
                Resolved: