Uploaded image for project: 'OpenShift Virtualization'
  1. OpenShift Virtualization
  2. CNV-46250

Hugepages setting in VM yaml file is not take effect

XMLWordPrintable

    • CNV Virtualization Sprint 258
    • None

      Description of problem:

      Start VM with 1Gi hugepage in VM yaml file, VM is started but it's without 1Gi hugepage

      Version-Release number of selected component (if applicable):

      CNV 4.16.1, iib:772966
      
      sh-5.1$ rpm -qa|grep libvirt
      libvirt-libs-10.0.0-6.3.el9_4.x86_64
      libvirt-client-10.0.0-6.3.el9_4.x86_64
      libvirt-daemon-common-10.0.0-6.3.el9_4.x86_64
      libvirt-daemon-log-10.0.0-6.3.el9_4.x86_64
      libvirt-daemon-driver-qemu-10.0.0-6.3.el9_4.x86_64
      
      sh-5.1$ rpm -qa|grep qemu-kvm
      qemu-kvm-common-8.2.0-11.el9_4.4.x86_64
      qemu-kvm-device-display-virtio-gpu-8.2.0-11.el9_4.4.x86_64
      qemu-kvm-core-8.2.0-11.el9_4.4.x86_64
      qemu-kvm-device-display-virtio-gpu-pci-8.2.0-11.el9_4.4.x86_64
      qemu-kvm-device-usb-host-8.2.0-11.el9_4.4.x86_64
      qemu-kvm-device-usb-redirect-8.2.0-11.el9_4.4.x86_64

      How reproducible:

      100%

      Steps to Reproduce:

      1. Create VM with hugepage settings in VM yaml file, more details in file: asb-vm-dv-ocs-hugepage.yaml
          spec:
            domain:
              resources:
                requests:
                  memory: "2Gi"
                  cpu: "4"
              memory:
                hugepages:
                  pageSize: "1Gi"
      
      #oc create -f /tmp/asb-vm-dv-ocs-hugepage.yaml
      virtualmachine.kubevirt.io/asb-vm-dv-ocs-hugepage created
      
      Check VM is running:
      #oc get vm
      NAME                     AGE   STATUS    READY
      asb-vm-dv-ocs-hugepage   14m   Running   True
      
      2. Check VM xml, there is no memoryBacking hugepages in VM xml, the VM is not using hugepages memory.
      --------------------------
        <memoryBacking>
          <hugepages/>
          <source type='memfd'/>
        </memoryBacking>
      --------------------------
      
      #oc get pod|grep virt-launcher
      virt-launcher-asb-vm-dv-ocs-hugepage-4gz9q   1/1     Running   0          23s
      
      #oc rsh virt-launcher-asb-vm-dv-ocs-hugepage-4gz9q
      sh-5.1$ virsh list --all
      Authorization not available. Check if polkit service is running or see debug message for more information.
       Id   Name                             State
      ------------------------------------------------
       1    default_asb-vm-dv-ocs-hugepage   running
      
      sh-5.1$ virsh dumpxml default_asb-vm-dv-ocs-hugepage| grep hugepages -A 10
      <No output>
      
      sh-5.1$ virsh dumpxml default_asb-vm-dv-ocs-hugepage|grep memory -A 10
      Authorization not available. Check if polkit service is running or see debug message for more information.
        <memory unit='KiB'>2097152</memory>
        <currentMemory unit='KiB'>2097152</currentMemory>
        <vcpu placement='static' current='2'>8</vcpu>
        <vcpus>
          <vcpu id='0' enabled='yes' hotpluggable='no' order='1'/>
          <vcpu id='1' enabled='yes' hotpluggable='yes' order='2'/>
          <vcpu id='2' enabled='no' hotpluggable='yes'/>
          <vcpu id='3' enabled='no' hotpluggable='yes'/>
          <vcpu id='4' enabled='no' hotpluggable='yes'/>
          <vcpu id='5' enabled='no' hotpluggable='yes'/>
          <vcpu id='6' enabled='no' hotpluggable='yes'/>
      --
            <cell id='0' cpus='0-7' memory='2097152' unit='KiB'/>
          </numa>
        </cpu>
      
      3. Check on node, there are allocatable 1Gi hugepages: 4Gi, and no 1Gi hugepages are allocated.
      
      #oc get vmi
      NAME                     AGE   PHASE     IP             NODENAME                                         READY
      asb-vm-dv-ocs-hugepage   38m   Running   10.129.0.255   cnv-qe-infra-04##   True
      
      #oc debug node/cnv-qe-infra-04##
      sh-5.1# cat /proc/sys/vm/nr_hugepages
      256
      sh-5.1# cat /proc/cmdline 
      BOOT_IMAGE=(hd0,gpt3)/boot/ostree/rhcos-39eeada61a2ec1055205a9b2b0506a8a707f28f9cf243571ddcdfdee5c1afd8c/vmlinuz-5.14.0-427.26.1.el9_4.x86_64 rw ostree=/ostree/boot.0/rhcos/39eeada61a2ec1055205a9b2b0506a8a707f28f9cf243571ddcdfdee5c1afd8c/0 ignition.platform.id=metal ip=dhcp fips=1 boot=LABEL=boot root=UUID=32caff12-0a11-4b20-ae00-097a24ae42ab rw rootflags=prjquota intel_iommu=on iommu=pt default_hugepagesz=2M hugepages=256 hugepagesz=1G hugepages=4 systemd.unified_cgroup_hierarchy=1 cgroup_no_v1=all psi=1
      
      #oc describe node cnv-qe-infra-04##|grep Allocatable -A 10
      Allocatable:
        cpu:                            111500m
        devices.kubevirt.io/kvm:        1k
        devices.kubevirt.io/tun:        1k
        devices.kubevirt.io/vhost-net:  1k
        ephemeral-storage:              1437506328243
        hugepages-1Gi:                  4Gi
        hugepages-2Mi:                  512Mi
        memory:                         125552208Ki
        nvidia.com/GA107GL_Ampere_A2:   1
        nvidia.com/GRID_A2_2Q:          0
      
      #oc describe node cnv-qe-infra-04.cnvqe3.lab.eng.rdu2.redhat.com|grep Allocated -A 15
      Allocated resources:
        (Total limits may be over 100 percent, i.e., overcommitted.)
        Resource                       Requests       Limits
        --------                       --------       ------
        cpu                            6985m (6%)     510m (0%)
        memory                         12307Mi (10%)  1224Mi (0%)
        ephemeral-storage              50M (0%)       0 (0%)
        hugepages-1Gi                  0 (0%)         0 (0%)
        hugepages-2Mi                  0 (0%)         0 (0%)
        devices.kubevirt.io/kvm        1              1
        devices.kubevirt.io/tun        1              1
        devices.kubevirt.io/vhost-net  1              1
        nvidia.com/GA107GL_Ampere_A2   0              0
        nvidia.com/GRID_A2_2Q          0              0
        nvidia.com/GRID_A2_4Q          0              0
      Events:                          <none>
       

      Expected results:

      In step1: VM is created with 1Gi hugepages
      In step2: Login into virt-launcher pod, check the VM xml include the memoryBacking hugepages part:
      -----------------------------
        <memoryBacking>     
        <hugepages/>     
          <source type='memfd'/>   
      </memoryBacking> 
      ----------------------------
      In step3: Check on the node, the allocated hugepages and allocatable hugepages are correct

      Additional info:

       

            acardace@redhat.com Antonio Cardace
            chhu@redhat.com Chenli Hu
            Kedar Bidarkar Kedar Bidarkar
            Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

              Created:
              Updated:
              Resolved: