-
Bug
-
Resolution: Done-Errata
-
None
-
Quality / Stability / Reliability
-
False
-
-
False
-
CLOSED
-
No
Description of problem: Metrics kubevirt_hco_out_of_band_modifications_count, does not reflect correct modification count when label is added to priorityclass/kubevirt-cluster-critical in a loop
Version-Release number of selected component (if applicable):
4.11.0
How reproducible:
Intermittently
Steps to Reproduce:
1. Update priorityclass/kubevirt-cluster-critical to add a random label
2. Wait for priorityclass/kubevirt-cluster-critical resource version change
3. Wait for reconciliation to take place and repeat the steps a few times
Actual results:updated PriorityClass resource version and stable status condition. However, kubevirt_hco_out_of_band_modifications_count
{component_name="priorityclass/kubevirt-cluster-critical"} metrics value does not get updated after first few times. I noticed the following error in hco-operator log=========== {"level":"info","ts":1651284657.6145072,"logger":"controller_hyperconverged","msg":"Reconciling for *v1.PriorityClass"} {"level":"info","ts":1651284657.6145704,"logger":"controller_hyperconverged","msg":"Reconciling for *v1.PriorityClass"} {"level":"info","ts":1651284657.6146228,"logger":"controller_hyperconverged","msg":"The reconciliation got triggered by a secondary CR object","Request.Namespace":"openshift-cnv","Request.Name":"hco-co ntrolled-cr-395acf97-848a-4940-bad9-5fd1334773c6"} {"level":"info","ts":1651284657.6147294,"logger":"controller_hyperconverged","msg":"KubeVirtPriorityClass already exists","Request.Namespace":"openshift-cnv","Request.Name":"kubevirt-hyperconverged","K ubeVirtPriorityClass.Namespace":"","KubeVirtPriorityClass.Name":"kubevirt-cluster-critical"} {"level":"info","ts":1651284657.6147447,"logger":"controller_hyperconverged","msg":"Reconciling an externally updated PriorityClass's Spec to its opinionated values","Request.Namespace":"openshift-cnv" ,"Request.Name":"kubevirt-hyperconverged"} {"level":"info","ts":1651284657.6408515,"logger":"controller_hyperconverged","msg":"Reconciling for *v1.PriorityClass"} {"level":"info","ts":1651284657.6568205,"logger":"controller_hyperconverged","msg":"Reconciling for *v1.PriorityClass"} {"level":"error","ts":1651284657.656815,"logger":"controller_hyperconverged","msg":"failed to ensure an operand","Request.Namespace":"openshift-cnv","Request.Name":"kubevirt-hyperconverged","error":"Pr iorityClass.scheduling.k8s.io \"kubevirt-cluster-critical\" not found","stacktrace":"github.com/kubevirt/hyperconverged-cluster-operator/controllers/hyperconverged.(*ReconcileHyperConverged).EnsureOper andAndComplete\n\t/remote-source/app/controllers/hyperconverged/hyperconverged_controller.go:480\ngithub.com/kubevirt/hyperconverged-cluster-operator/controllers/hyperconverged.(*ReconcileHyperConverge d).doReconcile\n\t/remote-source/app/controllers/hyperconverged/hyperconverged_controller.go:429\ngithub.com/kubevirt/hyperconverged-cluster-operator/controllers/hyperconverged.(*ReconcileHyperConverge d).Reconcile\n\t/remote-source/app/controllers/hyperconverged/hyperconverged_controller.go:293\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/remote-source/app/vend or/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:114\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/remote-source/app/vendor/sigs.k8s. io/controller-runtime/pkg/internal/controller/controller.go:311\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/remote-source/app/vendor/sigs.k8s.io/contro ller-runtime/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/remote-source/app/vendor/sigs.k8s.io/controller-runtime/pk g/internal/controller/controller.go:227"}
===================
Expected results:
kubevirt_hco_out_of_band_modifications_count{component_name="priorityclass/kubevirt-cluster-critical"}
output will show correct out of band modification count. No error message should be present in hco-operator log
Additional info: