Uploaded image for project: 'OpenShift Virtualization'
  1. OpenShift Virtualization
  2. CNV-23246

[2151693] After upgrade to 4.10.7->4.11.0 hco.spec.workloadUpdateStrategy value is getting overwritten

XMLWordPrintable

    • CNV I/U Operators Sprint 229
    • High
    • No

      Description of problem:For EUS->EUS upgrade we are supposed to turn off hco.spec.workloadUpdateStrategy, so that workload updates can only happen after upgrading to 4.12. However, that is currently not happening. Post upgrade to 4.11.0, I see hco.spec.workloadUpdateStrategy set to be LiveMigrate and workloads are live migrating.

      Version-Release number of selected component (if applicable):
      4.11.0

      How reproducible:
      100%

      Steps to Reproduce:
      1. Before upgrade set hco.spec.workloadUpdateStrategy to []
      2. After upgrade check that hco.spec.workloadUpdateStrategy is set to LiveMigrate
      3.

      Actual results:
      Before upgrade:
      ==============
      (cnv-tests) [cnv-qe-jenkins@cnv-qe-01 cnv-tests]$ kubectl get hco kubevirt-hyperconverged -n openshift-cnv -o json | jq ".status.versions"
      [

      { "name": "operator", "version": "4.10.7" }

      ]
      (cnv-tests) [cnv-qe-jenkins@cnv-qe-01 cnv-tests]$
      (cnv-tests) [cnv-qe-jenkins@cnv-qe-01 cnv-tests]$ kubectl get vm -A
      NAMESPACE NAME AGE STATUS READY
      test-upgrade-namespace vm-for-product-upgrade-hos-1670437886-760028 15m Running True
      test-upgrade-namespace vm-for-product-upgrade-hos-1670437887-3594792 14m Running True
      test-upgrade-namespace vm-for-product-upgrade-nfs-1670437884-7562118 15m Running True
      test-upgrade-namespace vm-for-product-upgrade-ocs-1670437885-5164044 15m Running True
      (cnv-tests) [cnv-qe-jenkins@cnv-qe-01 cnv-tests]$
      (cnv-tests) [cnv-qe-jenkins@cnv-qe-01 cnv-tests]$ kubectl get hco kubevirt-hyperconverged -n openshift-cnv -o json | jq ".spec.workloadUpdateStrategy"
      {
      "batchEvictionInterval": "1m0s",
      "batchEvictionSize": 10,
      "workloadUpdateMethods": []
      }
      (cnv-tests) [cnv-qe-jenkins@cnv-qe-01 cnv-tests]$
      (cnv-tests) [cnv-qe-jenkins@cnv-qe-01 cnv-tests]$ kubectl get hco kubevirt-hyperconverged -n openshift-cnv -o json | jq ".status.conditions"
      [

      { "lastTransitionTime": "2022-12-06T18:37:49Z", "message": "Reconcile completed successfully", "observedGeneration": 3, "reason": "ReconcileCompleted", "status": "True", "type": "ReconcileComplete" }

      ,

      { "lastTransitionTime": "2022-12-07T18:51:50Z", "message": "Reconcile completed successfully", "observedGeneration": 3, "reason": "ReconcileCompleted", "status": "True", "type": "Available" }

      ,

      { "lastTransitionTime": "2022-12-07T18:51:50Z", "message": "Reconcile completed successfully", "observedGeneration": 3, "reason": "ReconcileCompleted", "status": "False", "type": "Progressing" }

      ,

      { "lastTransitionTime": "2022-12-07T18:51:02Z", "message": "Reconcile completed successfully", "observedGeneration": 3, "reason": "ReconcileCompleted", "status": "False", "type": "Degraded" }

      ,

      { "lastTransitionTime": "2022-12-07T18:51:50Z", "message": "Reconcile completed successfully", "observedGeneration": 3, "reason": "ReconcileCompleted", "status": "True", "type": "Upgradeable" }

      ]
      (cnv-tests) [cnv-qe-jenkins@cnv-qe-01 cnv-tests]$ kubectl get hco kubevirt-hyperconverged -n openshift-cnv -o json | jq ".spec.workloadUpdateStrategy"
      {
      "batchEvictionInterval": "1m0s",
      "batchEvictionSize": 10,
      "workloadUpdateMethods": []
      }
      (cnv-tests) [cnv-qe-jenkins@cnv-qe-01 cnv-tests]$
      Post ocp upgrade:
      (cnv-tests) [cnv-qe-jenkins@cnv-qe-01 cnv-tests]$ kubectl get catalogsource -n openshift-marketplace
      NAME DISPLAY TYPE PUBLISHER AGE
      certified-operators Certified Operators grpc Red Hat 26h
      community-operators Community Operators grpc Red Hat 26h
      hco-catalogsource OpenShift Virtualization Index Image grpc Red Hat 25h
      ocs-catalogsource OpenShift Container Storage grpc Red Hat 25h
      redhat-marketplace Red Hat Marketplace grpc Red Hat 26h
      redhat-operators Red Hat Operators grpc Red Hat 26h
      (cnv-tests) [cnv-qe-jenkins@cnv-qe-01 cnv-tests]$ kubectl patch operatorhub cluster --type merge -p '{"spec": {"disableAllDefaultSources": true}}'
      operatorhub.config.openshift.io/cluster patched
      (cnv-tests) [cnv-qe-jenkins@cnv-qe-01 cnv-tests]$ kubectl get catalogsource -n openshift-marketplace
      NAME DISPLAY TYPE PUBLISHER AGE
      hco-catalogsource OpenShift Virtualization Index Image grpc Red Hat 25h
      ocs-catalogsource OpenShift Container Storage grpc Red Hat 25h
      (cnv-tests) [cnv-qe-jenkins@cnv-qe-01 cnv-tests]$
      (cnv-tests) [cnv-qe-jenkins@cnv-qe-01 cnv-tests]$ kubectl get ip -A
      NAMESPACE NAME CSV APPROVAL APPROVED
      openshift-cnv install-lqt5n kubevirt-hyperconverged-operator.v4.11.0 Manual false
      openshift-cnv install-njk24 kubevirt-hyperconverged-operator.v4.10.7 Manual true
      openshift-local-storage install-s8jm4 local-storage-operator.4.10.0-202211041323 Automatic true
      openshift-local-storage install-v86lx local-storage-operator.4.11.0-202211072116 Automatic true
      openshift-nfd install-t95wm nfd.4.10.0-202211041323 Automatic true
      openshift-nfd install-xcqkn nfd.4.11.0-202211091549 Automatic true
      openshift-operators install-4xbvx servicemeshoperator.v2.3.0 Automatic true
      openshift-operators install-lkc4w jaeger-operator.v1.39.0-3 Automatic true
      openshift-operators install-pj9kw jaeger-operator.v1.39.0-3 Automatic true
      openshift-sriov-network-operator install-42c9t sriov-network-operator.4.10.0-202211180226 Automatic true
      openshift-sriov-network-operator install-lw5p9 sriov-network-operator.4.11.0-202211211407 Automatic true
      openshift-storage install-s9qfr ocs-operator.v4.10.9 Automatic true
      (cnv-tests) [cnv-qe-jenkins@cnv-qe-01 cnv-tests]$ kubectl patch installplan install-lqt5n --namespace='openshift-cnv' --type='merge' --patch='{"spec":{"approved":true}}'
      installplan.operators.coreos.com/install-lqt5n patched
      (cnv-tests) [cnv-qe-jenkins@cnv-qe-01 cnv-tests]$ kubectl get csv -n openshift-cnv
      NAME DISPLAY VERSION REPLACES PHASE
      jaeger-operator.v1.39.0-3 Red Hat OpenShift distributed tracing platform 1.39.0-3 jaeger-operator.v1.34.1-5 Succeeded
      kiali-operator.v1.57.3 Kiali Operator 1.57.3 kiali-operator.v1.48.3 Succeeded
      kubevirt-hyperconverged-operator.v4.10.7 OpenShift Virtualization 4.10.7 kubevirt-hyperconverged-operator.v4.10.6 Replacing
      kubevirt-hyperconverged-operator.v4.11.0 OpenShift Virtualization 4.11.0 kubevirt-hyperconverged-operator.v4.10.7 Installing
      servicemeshoperator.v2.3.0 Red Hat OpenShift Service Mesh 2.3.0-0 servicemeshoperator.v2.2.3 Succeeded
      (cnv-tests) [cnv-qe-jenkins@cnv-qe-01 cnv-tests]$ kubectl get csv -n openshift-cnv
      NAME DISPLAY VERSION REPLACES PHASE
      jaeger-operator.v1.39.0-3 Red Hat OpenShift distributed tracing platform 1.39.0-3 jaeger-operator.v1.34.1-5 Succeeded
      kiali-operator.v1.57.3 Kiali Operator 1.57.3 kiali-operator.v1.48.3 Succeeded
      kubevirt-hyperconverged-operator.v4.11.0 OpenShift Virtualization 4.11.0 kubevirt-hyperconverged-operator.v4.10.7 Succeeded
      servicemeshoperator.v2.3.0 Red Hat OpenShift Service Mesh 2.3.0-0 servicemeshoperator.v2.2.3 Succeeded
      (cnv-tests) [cnv-qe-jenkins@cnv-qe-01 cnv-tests]$ kubectl get hco kubevirt-hyperconverged -n openshift-cnv -o json | jq ".status.versions"
      [

      { "name": "operator", "version": "4.11.0" }

      ]
      (cnv-tests) [cnv-qe-jenkins@cnv-qe-01 cnv-tests]$ kubectl get hco kubevirt-hyperconverged -n openshift-cnv -o json | jq ".spec.workloadUpdateStrategy"
      {
      "batchEvictionInterval": "1m0s",
      "batchEvictionSize": 10,
      "workloadUpdateMethods": [
      "LiveMigrate"
      ]
      }
      (cnv-tests) [cnv-qe-jenkins@cnv-qe-01 cnv-tests]$
      Expected results:
      hco.spec.workloadUpdateStrategy should stay unaltered.

      Additional info:
      This impacts EUS->EUS upgrade. Without ability to pause workload updates till CNV is updated to 4.12, we can't achive EUS->EUS upgradability

              stirabos Simone Tiraboschi
              rhn-support-dbasunag Debarati Basu-Nag
              Debarati Basu-Nag Debarati Basu-Nag
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Created:
                Updated:
                Resolved: