-
Bug
-
Resolution: Done-Errata
-
Critical
-
None
-
Quality / Stability / Reliability
-
3
-
False
-
-
False
-
CLOSED
-
-
-
CNV Virtualization Sprint 234, CNV Virtualization Sprint 235
-
Critical
-
None
Description of problem: Attempted to upgrade cnv from 4.12.1->v4.13.0.rhel9-1819 and see that hco is not able to update to 4.13.0
Version-Release number of selected component (if applicable):
v4.13.0.rhel9-1819
How reproducible:
one out of one attempt
Steps to Reproduce:
1. Try to upgrade cnv to v4.13.0.rhel9-1819
2.
3.
Actual results:
CSV is in succeeded state:
==============
cnv-qe-jenkins@cnv-qe-infra-01:~$ oc get csv -n openshift-cnv | grep kubevirt
kubevirt-hyperconverged-operator.v4.13.0 OpenShift Virtualization 4.13.0 kubevirt-hyperconverged-operator.v4.12.1 Succeeded
cnv-qe-jenkins@cnv-qe-infra-01:~$
===============
HCO shows this:
===============
cnv-qe-jenkins@cnv-qe-infra-01:~$ oc get hco kubevirt-hyperconverged -n openshift-cnv -o json | jq ".status.conditions"
[
,
,
,
,
{ "lastTransitionTime": "2023-03-20T18:22:35Z", "message": "Reconcile completed successfully", "observedGeneration": 8, "reason": "ReconcileCompleted", "status": "True", "type": "Upgradeable" }]
cnv-qe-jenkins@cnv-qe-infra-01:~$ oc get hco kubevirt-hyperconverged -n openshift-cnv -o json | jq ".status.versions"
[
]
cnv-qe-jenkins@cnv-qe-infra-01:~$
From hco operator log I see the following error. Will attach the full log:
=======================
{"level":"error","ts":"2023-03-20T18:49:00Z","msg":"Reconciler error","controller":"hyperconverged-controller","object":
{"name":"kubevirt-hyperconverged","namespace":"openshift-cnv"},"namespace":"openshift-cnv","name":"kubevirt-hyperconverged","reconcileID":"0be1f284-54ce-40f6-a9fc-998980a9f8b1","error":"Operation cannot be fulfilled on hyperconvergeds.hco.kubevirt.io \"kubevirt-hyperconverged\": the object has been modified; please apply your changes to the latest version and try again","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:329\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:274\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:235"}
{"level":"info","ts":"2023-03-20T18:49:00Z","logger":"controller_hyperconverged","msg":"Reconciling HyperConverged operator","Request.Namespace":"openshift-cnv","Request.Name":"kubevirt-hyperconverged"}{"level":"info","ts":"2023-03-20T18:49:00Z","logger":"controller_hyperconverged","msg":"Resource doesn't exist, there is nothing to remove","Request.Namespace":"openshift-cnv","Request.Name":"kubevirt-hyperconverged","Kind":"&TypeMeta
{Kind:,APIVersion:,}"}
{"level":"info","ts":"2023-03-20T18:49:00Z","logger":"controller_hyperconverged","msg":"Resource doesn't exist, there is nothing to remove","Request.Namespace":"openshift-cnv","Request.Name":"kubevirt-hyperconverged","Kind":"&TypeMeta
"}
{"level":"info","ts":"2023-03-20T18:49:00Z","logger":"controller_hyperconverged","msg":"Resource doesn't exist, there is nothing to remove","Request.Namespace":"openshift-cnv","Request.Name":"kubevirt-hyperconverged","Kind":"&TypeMeta
"}
{"level":"info","ts":"2023-03-20T18:49:00Z","logger":"controller_hyperconverged","msg":"reading quickstart guides","Request.Namespace":"openshift-cnv","Request.Name":"kubevirt-hyperconverged"} {"level":"info","ts":"2023-03-20T18:49:00Z","logger":"controller_hyperconverged","msg":"KubeVirtPriorityClass already exists","Request.Namespace":"openshift-cnv","Request.Name":"kubevirt-hyperconverged","KubeVirtPriorityClass.Namespace":"","KubeVirtPriorityClass.Name":"kubevirt-cluster-critical"} {"level":"info","ts":"2023-03-20T18:49:00Z","logger":"controller_hyperconverged","msg":"KubeVirt already exists","Request.Namespace":"openshift-cnv","Request.Name":"kubevirt-hyperconverged","KubeVirt.Namespace":"openshift-cnv","KubeVirt.Name":"kubevirt-kubevirt-hyperconverged"} {"level":"info","ts":"2023-03-20T18:49:00Z","logger":"controller_hyperconverged","msg":"could not complete the upgrade process. KubeVirt is not with the expected version. Check KubeVirt observed version in the status field of its CR","Request.Namespace":"openshift-cnv","Request.Name":"kubevirt-hyperconverged"}Expected results:
HCO should upgrade successfully.
Additional info:
- external trackers