Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-7881

4.12 upgrade: "default" ingress controller reports Degraded

XMLWordPrintable

    • No
    • 1
    • Sprint 238
    • 1
    • Rejected
    • False
    • Hide

      None

      Show
      None

      Description of problem:

      When upgrading a cluster from 4.11.0-0.nightly-2023-02-17-211917 to 4.12.0-0.nightly-2023-02-17-160406 the ingress operator becomes degraded and can not proceed and finish the upgrade properly 
      4.11.0-0.nightly-2023-02-17-211917 4.12.0-0.nightly-2023-02-17-160406 126_IPI on AWS & OVN & EFA & ENA

      https://mastern-jenkins-csb-openshift-qe.apps.ocp-c1.prod.psi.redhat.com/job/ocp-upgrade/job/upgrade-runner/30404/console

      Version-Release number of selected component (if applicable):

      4.12.0-0.nightly-2023-02-17-160406

      How reproducible:

      Unknown

      Steps to Reproduce:

      1. Create 126_IPI on AWS & OVN & EFA & ENA cluster with version 4.11.0-0.nightly-2023-02-17-211917
      2. Upgrade cluster to 4.12.0-0.nightly-2023-02-17-160406
      3. Error occurs 
      

      Actual results:

      Cluster upgrades with no issues and all operators are in a non degraded state

      Expected results:

      Ingress operator is degraded and unable to continue the rollout 

      Additional info:

      02-18 04:11:25.471  oc get clusteroperators: 
      02-18 04:11:25.471   NAME                                       VERSION                              AVAILABLE   PROGRESSING   DEGRADED   SINCE   MESSAGE
      02-18 04:11:25.471  authentication                             4.12.0-0.nightly-2023-02-17-160406   True        False         False      58m     
      02-18 04:11:25.471  baremetal                                  4.12.0-0.nightly-2023-02-17-160406   True        False         False      4h6m    
      02-18 04:11:25.471  cloud-controller-manager                   4.12.0-0.nightly-2023-02-17-160406   True        False         False      4h10m   
      02-18 04:11:25.471  cloud-credential                           4.12.0-0.nightly-2023-02-17-160406   True        False         False      4h10m   
      02-18 04:11:25.471  cluster-autoscaler                         4.12.0-0.nightly-2023-02-17-160406   True        False         False      4h6m    
      02-18 04:11:25.471  config-operator                            4.12.0-0.nightly-2023-02-17-160406   True        False         False      4h7m    
      02-18 04:11:25.471  console                                    4.12.0-0.nightly-2023-02-17-160406   True        False         False      59m     
      02-18 04:11:25.471  control-plane-machine-set                  4.12.0-0.nightly-2023-02-17-160406   True        False         False      125m    
      02-18 04:11:25.471  csi-snapshot-controller                    4.12.0-0.nightly-2023-02-17-160406   True        False         False      4h6m    
      02-18 04:11:25.471  dns                                        4.12.0-0.nightly-2023-02-17-160406   True        False         False      4h6m    
      02-18 04:11:25.471  etcd                                       4.12.0-0.nightly-2023-02-17-160406   True        False         False      4h5m    
      02-18 04:11:25.471  image-registry                             4.12.0-0.nightly-2023-02-17-160406   True        False         False      4h      
      02-18 04:11:25.471  ingress                                    4.12.0-0.nightly-2023-02-17-160406   True        True          False      78m     ingresscontroller "default" is progressing: IngressControllerProgressing: One or more status conditions indicate progressing: DeploymentRollingOut=True (DeploymentRollingOut: Waiting for router deployment rollout to finish: 1 of 2 updated replica(s) are available......
      02-18 04:11:25.471  insights                                   4.12.0-0.nightly-2023-02-17-160406   True        False         False      4h      
      02-18 04:11:25.472  kube-apiserver                             4.12.0-0.nightly-2023-02-17-160406   True        False         False      4h2m    
      02-18 04:11:25.472  kube-controller-manager                    4.12.0-0.nightly-2023-02-17-160406   True        False         False      4h4m    
      02-18 04:11:25.472  kube-scheduler                             4.12.0-0.nightly-2023-02-17-160406   True        False         False      4h3m    
      02-18 04:11:25.472  kube-storage-version-migrator              4.12.0-0.nightly-2023-02-17-160406   True        False         False      55m     
      02-18 04:11:25.472  machine-api                                4.12.0-0.nightly-2023-02-17-160406   True        False         False      4h3m    
      02-18 04:11:25.472  machine-approver                           4.12.0-0.nightly-2023-02-17-160406   True        False         False      4h6m    
      02-18 04:11:25.472  machine-config                             4.12.0-0.nightly-2023-02-17-160406   True        False         False      160m    
      02-18 04:11:25.472  marketplace                                4.12.0-0.nightly-2023-02-17-160406   True        False         False      4h6m    
      02-18 04:11:25.472  monitoring                                 4.12.0-0.nightly-2023-02-17-160406   True        False         False      3h58m   
      02-18 04:11:25.472  network                                    4.12.0-0.nightly-2023-02-17-160406   True        False         False      4h9m    
      02-18 04:11:25.472  node-tuning                                4.12.0-0.nightly-2023-02-17-160406   True        False         False      121m    
      02-18 04:11:25.472  openshift-apiserver                        4.12.0-0.nightly-2023-02-17-160406   True        False         False      4h1m    
      02-18 04:11:25.472  openshift-controller-manager               4.12.0-0.nightly-2023-02-17-160406   True        False         False      121m    
      02-18 04:11:25.472  openshift-samples                          4.12.0-0.nightly-2023-02-17-160406   True        False         False      124m    
      02-18 04:11:25.472  operator-lifecycle-manager                 4.12.0-0.nightly-2023-02-17-160406   True        False         False      4h6m    
      02-18 04:11:25.472  operator-lifecycle-manager-catalog         4.12.0-0.nightly-2023-02-17-160406   True        False         False      4h6m    
      02-18 04:11:25.472  operator-lifecycle-manager-packageserver   4.12.0-0.nightly-2023-02-17-160406   True        False         False      120m    
      02-18 04:11:25.472  service-ca                                 4.12.0-0.nightly-2023-02-17-160406   True        False         False      4h7m    
      02-18 04:11:25.472  storage                                    4.12.0-0.nightly-2023-02-17-160406   True        False         False      4h6m  

      Must gather information here: 
      https://mastern-jenkins-csb-openshift-qe.apps.ocp-c1.prod.psi.redhat.com/job/ocp-tools/job/collect-logs/8057/console

              mmasters1@redhat.com Miciah Masters
              prubenda Paige Patton
              Shudi Li Shudi Li
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

                Created:
                Updated:
                Resolved: