Uploaded image for project: 'OpenShift Virtualization'
  1. OpenShift Virtualization
  2. CNV-54711

Fix flaky tests that check the healthy metric

XMLWordPrintable

    • 1
    • False
    • Hide

      None

      Show
      None
    • False
    • None
    • ---
    • ---
    • 1
    • None

      The HCO functional test checks the  kubevirt_hyperconverged_operator_health_status metric in several tests.

      the function that perform this check is verifyOperatorHealthMetricValue.

      This check is flaky and depend on the behavior of other components. For example, the KubeVirt behavior was changed and now it becomes degraded and produces error instead of the expected warning.

      Also, the check is depend on the initial value of the metric and will be skipped with no warning or log if the metric value is error:

      if alertImpact >= initialOperatorHealthMetricValue 

      We want to decide if we want to test it in the functional test, or maybe unit test is enough. If we do want to test it in the functional test, we need to think about new way to verify it, that will be more deterministic. For example, to check the current HCO condition to get the expected metric value. But then we don't know what we're about to check (healthy/warning/error), and won't cover all the cases.

       

              sradco Shirly Radco
              nunnatsa Nahshon Unna Tsameret
              Krzysztof Majcher Krzysztof Majcher
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Created:
                Updated: