Uploaded image for project: 'RHEL'
  1. RHEL
  2. RHEL-7133

The improvement of 'hv_tlbflush' flag is not enough sometimes for win11

    • None
    • Moderate
    • sst_virtualization_hwe
    • ssg_virtualization
    • None
    • False
    • Hide

      None

      Show
      None
    • None
    • None
    • None
    • None
    • If docs needed, set a value
    • None

      Description of problem:
      The improvement of 'hv_tlbflush' flag is not enough sometimes for win11

      Version-Release number of selected component (if applicable):
      qemu-kvm-6.1.0-5.el9.x86_64
      kernel-5.14.0-7.el9.x86_64
      seabios-bin-1.14.0-6.el9.noarch
      edk2-ovmf-20210527gite1999b264f1f-6.el9.noarch

      How reproducible:
      not 100%

      Steps to Reproduce:
      1. boot a win11 guest without the "hv_tlbflush" "hv_vpindex" and "hv_reset" flags.

      MALLOC_PERTURB_=1 /usr/libexec/qemu-kvm \
      -name 'avocado-vt-vm1' \
      -sandbox on \
      -blockdev node-name=file_ovmf_code,driver=file,filename=/usr/share/OVMF/OVMF_CODE.secboot.fd,auto-read-only=on,discard=unmap \
      -blockdev node-name=drive_ovmf_code,driver=raw,read-only=on,file=file_ovmf_code \
      -blockdev node-name=file_ovmf_vars,driver=file,filename=/home/kvm_autotest_root/images/avocado-vt-vm1_win11-64-virtio-scsi.qcow2_VARS.fd,auto-read-only=on,discard=unmap \
      -blockdev node-name=drive_ovmf_vars,driver=raw,read-only=off,file=file_ovmf_vars \
      -machine q35,memory-backend=mem-machine_mem,pflash0=drive_ovmf_code,pflash1=drive_ovmf_vars \
      -device pcie-root-port,id=pcie-root-port-0,multifunction=on,bus=pcie.0,addr=0x1,chassis=1 \
      -device pcie-pci-bridge,id=pcie-pci-bridge-0,addr=0x0,bus=pcie-root-port-0 \
      -nodefaults \
      -device VGA,bus=pcie.0,addr=0x2 \
      -m 15360 \
      -object memory-backend-ram,size=15360M,id=mem-machine_mem \
      -smp 20,maxcpus=20,cores=10,threads=1,dies=1,sockets=2 \
      -cpu 'Broadwell',hv_crash,hv_relaxed,hv_spinlocks=0x1fff,hv_vapic,hv_time,hv_frequencies,hv_runtime,hv_reenlightenment,+kvm_pv_unhalt \
      -object rng-random,filename=/dev/urandom,id=passthrough-Hw2x9JTs \
      -device pcie-root-port,id=pcie-root-port-1,port=0x1,addr=0x1.0x1,bus=pcie.0,chassis=2 \
      -device virtio-rng-pci,id=virtio-rng-pci-iLMnFY9D,rng=passthrough-Hw2x9JTs,bus=pcie-root-port-1,addr=0x0 \
      -device pcie-root-port,id=pcie-root-port-2,port=0x2,addr=0x1.0x2,bus=pcie.0,chassis=3 \
      -device qemu-xhci,id=usb1,bus=pcie-root-port-2,addr=0x0 \
      -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1 \
      -device pcie-root-port,id=pcie-root-port-3,port=0x3,addr=0x1.0x3,bus=pcie.0,chassis=4 \
      -device virtio-scsi-pci,id=virtio_scsi_pci0,bus=pcie-root-port-3,addr=0x0 \
      -blockdev node-name=file_image1,driver=file,auto-read-only=on,discard=unmap,aio=threads,filename=/home/win11-64-virtio-scsi.qcow2,cache.direct=on,cache.no-flush=off \
      -blockdev node-name=drive_image1,driver=qcow2,read-only=off,cache.direct=on,cache.no-flush=off,file=file_image1 \
      -device scsi-hd,id=image1,drive=drive_image1,write-cache=on \
      -device pcie-root-port,id=pcie-root-port-4,port=0x4,addr=0x1.0x4,bus=pcie.0,chassis=5 \
      -device virtio-net-pci,mac=9a:2a:42:7a:58:84,id=idqponYj,netdev=idW3Z2Yh,bus=pcie-root-port-4,addr=0x0 \
      -netdev tap,id=idW3Z2Yh \
      -blockdev node-name=file_cd1,driver=file,auto-read-only=on,discard=unmap,aio=threads,filename=/home/kvm_autotest_root/iso/windows/winutils.iso,cache.direct=on,cache.no-flush=off \
      -blockdev node-name=drive_cd1,driver=raw,read-only=on,cache.direct=on,cache.no-flush=off,file=file_cd1 \
      -device scsi-cd,id=cd1,drive=drive_cd1,write-cache=on \
      -vnc :10 \
      -rtc base=localtime,clock=host,driftfix=slew \
      -boot menu=off,order=cdn,once=c,strict=off \
      -chardev socket,id=char_vtpm_tpm0,path=/tmp/guest-swtpm.sock \
      -tpmdev emulator,chardev=char_vtpm_tpm0,id=emulator_vtpm_tpm0 \
      -device tpm-crb,id=tpm-crb_vtpm_tpm0,tpmdev=emulator_vtpm_tpm0 \
      -enable-kvm \
      -device pcie-root-port,id=pcie_extra_root_port_0,multifunction=on,bus=pcie.0,addr=0x3,chassis=6 \
      -enable-kvm \
      -qmp tcp:0:1231,server,nowait \
      -monitor stdio \

      2. Give the stress on host cpu.

      1. nohup stress --cpu 40 > /dev/null 2>&1& > /dev/null &

      3.Execute the command 30 times in powershell to get the average execution time.
      powershell -command "$m=0; for ($i=0; $i -lt 30; $i++)

      { $s = Get-Date; c:\hv_tlbflush.exe c:\tlb_flush_test.txt; $e=Get-Date; $t = $e - $s; $m=$m+$t; Write-Host $t }

      ; Write-Host $m"

      4. Shutdown the guest. Then boot the guest again with the "hv_tlbflush" "hv_vpindex" and "hv_reset" flags.
      -cpu 'Broadwell',hv_crash,hv_reset,hv_stimer,hv_synic,hv_vpindex,hv_relaxed,hv_spinlocks=0x1fff,hv_vapic,hv_time,hv_frequencies,hv_runtime,hv_tlbflush,hv_reenlightenment,hv_stimer_direct,hv_ipi,+kvm_pv_unhalt \

      5. Repeat the step 2 and step 3. Get the new average execution time. Then compare those results.

      Actual results:
      The execute time not faster (50%)when adding the flag

      Expected results:
      The execute time should faster (50%)when adding the flag

      Additional info:

      1. also hit on rhel860 and rhel850
      rhel860 :qemu-kvm-6.1.0-4.module+el8.6.0+13039+4b81a1dc.x86_64
      rhel850 :qemu-kvm-block-rbd-6.0.0-30.module+el8.5.0+12586+476da3e1.x86_64

      2. not hit on win10/win2016/win2019/win2022

            vkuznets@redhat.com Vitaly Kuznetsov
            rhn-support-menli Menghuan Li
            Vitaly Kuznetsov Vitaly Kuznetsov
            Peixiu Hou Peixiu Hou
            Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

              Created:
              Updated: