Uploaded image for project: 'Observability UI'
  1. Observability UI
  2. OU-863

Inconsistent metrics scraped from OpenShift Virtualization resources

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Major Major
    • None
    • None
    • Console, monitoring-plugin
    • None
    • Quality / Stability / Reliability
    • False
    • Hide

      None

      Show
      None
    • 2
    • Moderate
    • None
    • None
    • CNV I/U Operators Sprint 272, Sprint 274, Sprint 276, Sprint 277
    • Customer Reported
    • None
    • None
    • None

      Description of problem:

      When checking kubevirt_vmi_* metrics in "Observe > Metrics" menu, there is a uniform gaps in scraping and looks like metrics from OpenShift Virtualization are not getting generated consistently itself.

      Version-Release number of selected component (if applicable):

      OpenShift Virtualization 4.17.z

      How reproducible:

      100%

      Steps to Reproduce:

      1. Install OpenShift Virtualization operator
      2. Configure Hyperconverged CR so that below metrics are exposed 
      
      kubevirt_vmi_cpu_affinity
      kubevirt_vmi_cpu_usage_seconds_total
      kubevirt_vmi_available_bytes

      Actual results:

      The metrics are uniformly generated because of which prometheus shows gaps in metrics while viewing in "Observe > Metrics" menu.

      Expected results:

      The metrics should be generated uniformly and there should be no gaps when querying metrics.

      Additional info:
      These are the metrics that have thus-far had gaps in their data

      rate(apiserver_request_aborts_total[5m])
      apiserver_flowcontrol_demand_seats_sum
      kube_running_pod_ready
      kubelet_volume_stats_available_bytes
      kubevirt_vmi_cpu_usage_seconds_total
      increase(apiserver_tls_handshake_errors_total[15m])
      histogram_quantile(0.95, sum(rate(argocd_git_request_duration_seconds_bucket[5m])) by (le))
      rate(argocd_git_request_duration_seconds_sum[5m]) / rate(argocd_git_request_duration_seconds_count[5m])
      authentication_duration_seconds_sum
      increase(cardinality_enforcement_unexpected_categorizations_total[15m])
      cco_controller_reconcile_seconds_bucket
      cluster_operator_up
      controller_runtime_active_workers
      controller_runtime_reconcile_errors_total
      controller_runtime_reconcile_time_seconds_sum
      rate(controller_runtime_reconcile_time_seconds_sum[5m])
      coredns_dns_response_size_bytes_count
      haproxy_server_http_average_response_latency_milliseconds
      haproxy_server_max_sessions
      haproxy_server_response_errors_total
      horizontal_pod_autoscaler_controller_reconciliation_duration_seconds_bucket
      http_client_dns_duration_seconds_bucket
      http_inflight_requests
      http_request_size_bytes_sum
      histogram_quantile(0.95, rate(http_response_size_bytes_bucket[5m]))
      job_namespace_route_statuscode:loki_request_duration_seconds_count:irate1m
      kube_daemonset_status_updated_number_scheduled
      kubelet_active_pods
      kubelet_desired_pods
      kubelet_pod_worker_duration_seconds_sum
      kubelet_runtime_operations_errors_total
      kubevirt_vmi_memory_actual_balloon_bytes
      ha_server_up

       

       

       

       

        1. all_tls_handshake_error_output_mmc_01.txt
          6.64 MB
        2. all_tls_handshake_error_output_mmc_05.txt
          1.34 MB
        3. image (11).png
          image (11).png
          24 kB
        4. image (12).png
          image (12).png
          24 kB
        5. image (13).png
          image (13).png
          27 kB
        6. image (14).png
          image (14).png
          43 kB
        7. image (15).png
          image (15).png
          25 kB
        8. image-2025-05-22-14-50-54-472.png
          image-2025-05-22-14-50-54-472.png
          29 kB
        9. image-2025-09-19-10-22-42-646.png
          image-2025-09-19-10-22-42-646.png
          131 kB

              gbernal@redhat.com Gabriel Bernal
              rhn-support-dgautam Dhruv Gautam
              None
              Evelyn Murasaki Evelyn Murasaki
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

                Created:
                Updated: