Uploaded image for project: 'OpenShift Request For Enhancement'
  1. OpenShift Request For Enhancement
  2. RFE-6361

PodDisruptionBudget (PDB) causes Machine-Config-Operator (MCO) to be degraded during OCP4 upgrade

XMLWordPrintable

    • Icon: Feature Request Feature Request
    • Resolution: Unresolved
    • Icon: Undefined Undefined
    • None
    • None
    • None
    • False
    • None
    • False
    • Not Selected
    • Hide
      • Many of our customers experience this issue consistently during each and every upgrade, which makes the upgrade process extremely time-consuming and labor-intensive for them. This repetitive issue affects their overall operational efficiency and satisfaction.
      • The current workaround solutions are less than ideal. They involve either removing the PDB, which is not always feasible, or manually deleting the pods when the drain operation fails, which is labor-intensive and prone to human error.
      • The exact error can only be identified by examining the machine config daemon pod logs, making it difficult for customers to diagnose and resolve the issue on their own. Consequently, our customers raise support tickets for this problem each time it occurs, adding to their frustration and our support workload. 
      • I am raising this RFE based on https://issues.redhat.com/browse/OCPBUGS-38884 , as this is the nature of PDB that cannot be changed , the error should be visible clearly to the customer that PDB is causing this issue and the required steps that needs to be followed.
      Show
      Many of our customers experience this issue consistently during each and every upgrade, which makes the upgrade process extremely time-consuming and labor-intensive for them. This repetitive issue affects their overall operational efficiency and satisfaction. The current workaround solutions are less than ideal. They involve either removing the PDB, which is not always feasible, or manually deleting the pods when the drain operation fails, which is labor-intensive and prone to human error. The exact error can only be identified by examining the machine config daemon pod logs, making it difficult for customers to diagnose and resolve the issue on their own. Consequently, our customers raise support tickets for this problem each time it occurs, adding to their frustration and our support workload.  I am raising this RFE based on  https://issues.redhat.com/browse/OCPBUGS-38884 , as this is the nature of PDB that cannot be changed , the error should be visible clearly to the customer that PDB is causing this issue and the required steps that needs to be followed.

      1. Proposed title of this feature request PodDisruptionBudget (PDB) causes Machine-Config-Operator (MCO) to be degraded during OCP4 upgrade
      2. What is the nature and description of the request? To create a warning for PDBs before starting the upgrade
      3. Why does the customer need this? (List the business requirements here) 

      • We have many premium customers that have faced this issue as mentioned in https://access.redhat.com/solutions/4857671 . I will name some of these in private comments soon.
      • It does not matter if the pods are healthy or not , they will simply block the upgrade process
      $ oc logs -n openshift-machine-config-operator machine-config-controller-xxxxx -c machine-config-controller
...
I0220 04:14:18.029980   49566 update.go:89] error when evicting pod "test-1-xxxxx" (will retry after 5s): Cannot evict pod as it would violate the pod's disruption budget.
I0220 04:14:23.055546   49566 update.go:89] error when evicting pod "test-1-xxxxx" (will retry after 5s): Cannot evict pod as it would violate the pod's disruption budget.
I0220 04:14:28.073188   49566 update.go:89] error when evicting pod "test-1-xxxxx" (will retry after 5s): Cannot evict pod as it would violate the pod's disruption budget.

      4. List any affected packages or components.
      OpenShift Upgrade Process , Machine Config Daemon degraded

              rhn-support-mrussell Mark Russell
              rhn-support-vismishr Vishvranjan Mishra
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated: