Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-48609

ocp upgrade to 4.18 failing due to openstack-manila-csi-controllerplugin-pdb

XMLWordPrintable

    • +
    • Important
    • None
    • ShiftStack Sprint 265, ShiftStack Sprint 266
    • 2
    • Rejected
    • False
    • Hide

      None

      Show
      None
    • Release Note Not Required
    • In Progress

      This is a clone of issue OCPBUGS-48507. The following is the description of the original issue:
      —
      Description of problem:

      Upgrade to 4.18 is not working, because the machine-config update is stuck:

      $ oc get co/machine-config
NAME             VERSION                              AVAILABLE   PROGRESSING   DEGRADED   SINCE   MESSAGE
machine-config   4.17.0-0.nightly-2025-01-13-120007   True        True          True       133m    Unable to apply 4.18.0-rc.4: error during syncRequiredMachineConfigPools: [context deadline exceeded, MachineConfigPool master has not progressed to latest configuration: controller version mismatch for rendered-master-ef1c06aa9aeedcebfa50569c3aa9472a expected a964f19a214946f0e5f1197c545d3805393d0705 has 3594c4b2eb42d8c9e56a146baea52d9c147721b0: 0 (ready 0) out of 3 nodes are updating to latest configuration rendered-master-826ddf793cf0a677228234437446740f, retrying]

      The machine-config-controller shows the responsible for that:

      $ oc logs -n openshift-machine-config-operator                  machine-config-controller-69f59598f7-57lkv
[...]
I0116 13:54:16.605692       1 drain_controller.go:183] node ostest-xgjnz-master-0: Drain failed. Drain has been failing for more than 10 minutes. Waiting 5 minutes then retrying. Error message from drain: error when evicting pods/"openstack-manila-csi-controllerplugin-6754c7589f-dwjtm" -n "openshift-manila-csi-driver": This pod has more than one PodDisruptionBudget, which the eviction subresource does not support.

      There are 2 PDBs on the manila namespace:

      $ oc get pdb -n openshift-manila-csi-driver
NAME                                        MIN AVAILABLE   MAX UNAVAILABLE   ALLOWED DISRUPTIONS   AGE
manila-csi-driver-controller-pdb            N/A             1                 1                     80m
openstack-manila-csi-controllerplugin-pdb   N/A             1                 1                     134m

      So a workaround is to remove the pdb openstack-manila-csi-controllerplugin-pdb.

      Version-Release number of selected component (if applicable):

      From 4.17.0-0.nightly-2025-01-13-120007 to 4.18.0-rc.4
on top of RHOS-17.1-RHEL-9-20241030.n.1

      How reproducible:

      Always

      Steps to Reproduce:
      1. Install latest 4.17
      2. Update to 4.18, for example:

      oc adm upgrade --to-image=registry.ci.openshift.org/ocp/release:4.18.0-rc.4 --allow-explicit-upgrade --force

      Additional info: must-gather on private comment.

              mdemaced Maysa De Macedo Souza
              openshift-crt-jira-prow OpenShift Prow Bot
              Itay Matza Itay Matza
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

                Created:
                Updated: