Uploaded image for project: 'OpenShift Virtualization'
  1. OpenShift Virtualization
  2. CNV-46600

Metric kubevirt_virt_controller_ready getting wrong values

XMLWordPrintable

    • 0.42
    • False
    • Hide

      None

      Show
      None
    • False
    • None
    • ---
    • ---
    • CNV I/U Operators Sprint 262
    • None

      Description of problem:

      As part of testing LowReadyVirtControllersCount alert, the expression
      kubevirt_virt_controller_ready < kubevirt_virt_controller_up need to happen.
      the metric kubevirt_virt_controller_ready shows wrong value of ready virt-controller pods, the metric shows 2 even when 0 is ready.

      Version-Release number of selected component (if applicable):

      4.17

      How reproducible:

      100%

      Steps to Reproduce:

      1.Disable virt-operator and olm-operator
      oc scale deployment virt-operator -n openshift-cnv --replicas=0
      oc scale deployment olm-operator -n openshift-operator-lifecycle-manager --replicas=0
      2.Change under readinesProbe, the httpGet (or port) to something fake like /fake-leader
      3.Delete virt-controller ReplicaSet
      4.Look at the value of  kubevirt_virt_controller_ready and compare it with the output of the oc command to get the pods.
      
       oc get pods -n openshift-cnv | grep virt-controller

      Actual results:

      The oc command shows that the virt-controller is not ready but up while the metric shows that the pods is ready.

      Expected results:

      The metric should correlate to the real status of the virt-controller pods which is not ready in this situation meaning the value should be 0 in this case.

      Additional info:

      Added screenshots

            alitman@redhat.com Aviv Litman
            rh-ee-orevah Ohad Revah
            Ohad Revah Ohad Revah
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated: