Uploaded image for project: 'OpenShift Virtualization'
  1. OpenShift Virtualization
  2. CNV-22934

[2148266] CPU metrics is missing

XMLWordPrintable

    • Important
    • None

      Description of problem:
      I'm working on adding cpu metrics to kubevirt: https://github.com/kubevirt/kubevirt/pull/8774, and we think there is a bug on libvirt since we can't see all cpu metrics when running `virsh domstats 1, the missing metrics are:
      cpu.time
      cpu.user
      cpu.system

      and when running `virsh cpu-stats 1`:
      virsh # cpu-stats 1
      error: Failed to retrieve CPU statistics for domain 'default_vm-cirros'
      error: Requested operation is not valid: cgroup CPUACCT controller is not mounted

      Version-Release number of selected component (if applicable):
      bash-5.1$ virsh version
      Authorization not available. Check if polkit service is running or see debug message for more information.
      Compiled against library: libvirt 8.7.0
      Using library: libvirt 8.7.0
      Using API: QEMU 8.7.0
      Running hypervisor: QEMU 7.1.0

      How reproducible:
      100%

      Steps to Reproduce:
      1.create kubevirt cluster:
      -git clone https://github.com/kubevirt/kubevirt.git
      -cd kubevirt
      -make cluster-up
      -make cluster-sync
      2. create a vm

      • k apply -f examples/vm-cirros.yaml (or other vm)
        3. exec into the launcher
      • k get pods
      • k exec -it virt-launcher-vm-cirros-6kzfd /bin/bash
        4. query libvirt metrics with virsh
      • virsh domstats 1
      • virsh cpu-stats 1

      Actual results:
      Stage 4 missing cpu metrics mentioned here: https://libvirt.org/html/libvirt-libvirt-domain.html#virConnectGetAllDomainStats:~:text=%22cpu.time%22%20%2D%20total%20cpu%20time%20spent%20for%20this%20domain%20in%20nanoseconds%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20as%20unsigned%20long%20long.%0A%20%20%22cpu.user%22%20%2D%20user%20cpu%20time%20spent%20in%20nanoseconds%20as%20unsigned%20long%20long.%0A%20%20%22cpu.system%22%20%2D%20system%20cpu%20time%20spent%20in%20nanoseconds%20as%20unsigned%20long%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20long.

      and we get the error:
      error: Failed to retrieve CPU statistics for domain 'default_vm-cirros'
      error: Requested operation is not valid: cgroup CPUACCT controller is not mounted

      Expected results:
      Stage 4 will show all cpu metrics mentioned here: https://libvirt.org/html/libvirt-libvirt-domain.html#virConnectGetAllDomainStats:~:text=VIR_DOMAIN_STATS_CPU_TOTAL%3A%20Return%20CPU,cache%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20bank%20%3Cindex%3E

      And CPUACCT will be mounted.

      Additional info:
      Luboslav Pivarc suggested that the issue is related to the fact there is cpu and cpuacct:
      https://github.com/libvirt/libvirt/blob/master/src/qemu/qemu_driver.c#L16266
      https://github.com/libvirt/libvirt/blob/master/src/util/vircgroupv1.c#L292

      Not sure how to add the xml of the VM, and libvirt debug logs, can you share the steps?

              sgott@redhat.com Stuart Gott
              alitman@redhat.com Aviv Litman
              Kedar Bidarkar Kedar Bidarkar
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated:
                Resolved: