Uploaded image for project: 'RHEL'
  1. RHEL
  2. RHEL-65647

netkvm: code 56 error on startup with 27 NICs using prewhql266

    • rhel-sst-virtualization-windows
    • ssg_virtualization
    • 13
    • QE ack
    • False
    • Hide

      None

      Show
      None
    • None
    • Red Hat Enterprise Linux
    • None
    • x86_64
    • Windows
    • None

      What were you trying to do that didn't work?
      Boot up with 27nics installed, using the prewhql266 netkvm driver package. After booting, multiple network adapters show an error message: "Windows is still setting up the class configuration for this device (Code 56)" when checked in the device manager.

      Please provide the package NVR for which bug is seen:
       * virtio-win-prewhql-0.1-266
       * kernel-5.14.0-524.el9.x86_64
       * seabios-bin-1.16.3-2.el9.noarch
       * edk2-ovmf-20240524-7.el9.noarch
       * swtpm-0.8.0-2.el9_4.x86_64
       * qemu-kvm-core-9.1.0-1.el9.x86_64

      How reproducible:
      100%

      Steps to reproduce
      1. Boot a VM with 27 NICs that have installed the prewhql265 netkvm driver on a Windows Server 2019.
      2. Open Device Manager and check the status of each network adapter.
      3. Observe the NIC attribute information of the Device Manager and get an error: "Windows is still setting up the class configuration for this device (Code 56)".
      4. Maintain the observed error state and directly CTRL-C off the VM.
      5. Boot it again directly.
      6. Observe the NIC attribute information of the Device Manager, but the problem disappears.

      Expected results
      All 27 network adapters should initialize without error at the beginning.

      Actual results
      Many network adapters fail to initialize properly, displaying the message: "Windows is still setting up the class configuration for this device (Code 56)".

      Additional Notes:
      1. A screenshot will be provided as an attachment for investigation.
      2. The whole qemu-kvm cmdline is below.

      /usr/libexec/qemu-kvm \
       -name 'avocado-vt-vm1'  \
       -sandbox on,elevateprivileges=deny,obsolete=deny,resourcecontrol=deny \
       -blockdev '{"node-name": "file_ovmf_code", "driver": "file", "filename": "/usr/share/OVMF/OVMF_CODE.secboot.fd", "auto-read-only": true, "discard": "unmap"}' \
       -blockdev '{"node-name": "drive_ovmf_code", "driver": "raw", "read-only": true, "file": "file_ovmf_code"}' \
       -blockdev '{"node-name": "file_ovmf_vars", "driver": "file", "filename": "/root/avocado/data/avocado-vt/avocado-vt-vm1_win2019-64-virtio-scsi-ovmf_qcow2_filesystem_VARS.raw", "auto-read-only": true, "discard": "unmap"}' \
       -blockdev '{"node-name": "drive_ovmf_vars", "driver": "raw", "read-only": false, "file": "file_ovmf_vars"}' \
       -machine q35,pflash0=drive_ovmf_code,pflash1=drive_ovmf_vars,memory-backend=mem-machine_mem \
       -device '{"id": "pcie-root-port-0", "driver": "pcie-root-port", "multifunction": true, "bus": "pcie.0", "addr": "0x1", "chassis": 1}' \
       -device '{"id": "pcie-pci-bridge-0", "driver": "pcie-pci-bridge", "addr": "0x0", "bus": "pcie-root-port-0"}'  \
       -nodefaults \
       -device '{"driver": "VGA", "bus": "pcie.0", "addr": "0x2"}' \
       -m 14336 \
       -object '{"size": 15032385536, "id": "mem-machine_mem", "qom-type": "memory-backend-ram"}'  \
       -smp 32,maxcpus=32,cores=16,threads=1,dies=1,sockets=2  \
       -cpu 'EPYC-Milan',x2apic=on,tsc-deadline=on,hypervisor=on,tsc-adjust=on,vaes=on,vpclmulqdq=on,spec-ctrl=on,stibp=on,arch-capabilities=on,ssbd=on,cmp-legacy=on,overflow-recov=on,succor=on,stibp-always-on=on,virt-ssbd=on,amd-psfd=on,lbrv=on,tsc-scale=on,vmcb-clean=on,flushbyasid=on,pause-filter=on,pfthreshold=on,v-vmsave-vmload=on,vgif=on,no-nested-data-bp=on,lfence-always-serializing=on,null-sel-clr-base=on,rdctl-no=on,skip-l1dfl-vmentry=on,mds-no=on,pschange-mc-no=on,gds-no=on,rfds-no=on,erms=off,fsrm=off,hv_stimer,hv_synic,hv_vpindex,hv_relaxed,hv_spinlocks=0x1fff,hv_vapic,hv_time,hv_frequencies,hv_runtime,hv_tlbflush,hv_reenlightenment,hv_stimer_direct,hv_ipi,hv-xmm-input,hv_tlbflush_ext,kvm_pv_unhalt=on \
       -device '{"ioport": 1285, "driver": "pvpanic", "id": "idt6URb4"}' \
       -device '{"id": "pcie-root-port-1", "port": 1, "driver": "pcie-root-port", "addr": "0x1.0x1", "bus": "pcie.0", "chassis": 2}' \
       -device '{"driver": "qemu-xhci", "id": "usb1", "bus": "pcie-root-port-1", "addr": "0x0"}' \
       -device '{"driver": "usb-tablet", "id": "usb-tablet1", "bus": "usb1.0", "port": "1"}' \
       -device '{"id": "pcie-root-port-2", "port": 2, "driver": "pcie-root-port", "addr": "0x1.0x2", "bus": "pcie.0", "chassis": 3}' \
       -device '{"id": "virtio_scsi_pci0", "driver": "virtio-scsi-pci", "bus": "pcie-root-port-2", "addr": "0x0"}' \
       -blockdev '{"node-name": "file_image1", "driver": "file", "auto-read-only": true, "discard": "unmap", "aio": "threads", "filename": "/home/kvm_autotest_root/images/win2019-64-virtio-scsi-ovmf.qcow2", "cache": {"direct": true, "no-flush": false}}' \
       -blockdev '{"node-name": "drive_image1", "driver": "qcow2", "read-only": false, "cache": {"direct": true, "no-flush": false}, "file": "file_image1"}' \
       -device '{"driver": "scsi-hd", "id": "image1", "drive": "drive_image1", "write-cache": "on"}' \
       -device '{"id": "pcie-root-port-3", "port": 3, "driver": "pcie-root-port", "addr": "0x1.0x3", "bus": "pcie.0", "chassis": 4}' \
       -device '{"driver": "virtio-net-pci", "mac": "9a:17:08:ce:51:74", "id": "idUnbeuz", "mq": true, "vectors": 10, "netdev": "idhndcwq", "bus": "pcie-root-port-3", "addr": "0x0"}' \
       -netdev  '{"id": "idhndcwq", "type": "tap", "vhost": true }' \
       -device '{"id": "pcie-root-port-4", "port": 4, "driver": "pcie-root-port", "addr": "0x1.0x4", "bus": "pcie.0", "chassis": 5}' \
       -device '{"driver": "virtio-net-pci", "mac": "9a:17:08:ce:51:75", "id": "idZrqVI9", "mq": true, "vectors": 10, "netdev": "idDLVA2t", "bus": "pcie-root-port-4", "addr": "0x0"}' \
       -netdev  '{"id": "idDLVA2t", "type": "tap", "vhost": true }'  \
       -device '{"id": "pcie-root-port-5", "port": 5, "driver": "pcie-root-port", "addr": "0x1.0x5", "bus": "pcie.0", "chassis": 6}' \
       -device '{"driver": "virtio-net-pci", "mac": "9a:17:08:ce:51:76", "id": "idzxL0Yv", "mq": true, "vectors": 10, "netdev": "idNWA3tz", "bus": "pcie-root-port-5", "addr": "0x0"}' \
       -netdev  '{"id": "idNWA3tz", "type": "tap", "vhost": true }'  \
       -device '{"id": "pcie-root-port-6", "port": 6, "driver": "pcie-root-port", "addr": "0x1.0x6", "bus": "pcie.0", "chassis": 7}' \
       -device '{"driver": "virtio-net-pci", "mac": "9a:17:08:ce:51:77", "id": "idz3hQCH", "mq": true, "vectors": 10, "netdev": "id5O7M2H", "bus": "pcie-root-port-6", "addr": "0x0"}' \
       -netdev  '{"id": "id5O7M2H", "type": "tap", "vhost": true }'  \
       -device '{"id": "pcie-root-port-7", "port": 7, "driver": "pcie-root-port", "addr": "0x1.0x7", "bus": "pcie.0", "chassis": 8}' \
       -device '{"driver": "virtio-net-pci", "mac": "9a:17:08:ce:51:78", "id": "idxXahm3", "mq": true, "vectors": 10, "netdev": "idt1vap0", "bus": "pcie-root-port-7", "addr": "0x0"}' \
       -netdev  '{"id": "idt1vap0", "type": "tap", "vhost": true }'  \
       -device '{"id": "pcie-root-port-8", "port": 8, "driver": "pcie-root-port", "multifunction": true, "bus": "pcie.0", "addr": "0x3", "chassis": 9}' \
       -device '{"driver": "virtio-net-pci", "mac": "9a:17:08:ce:51:79", "id": "idcR7r7P", "mq": true, "vectors": 10, "netdev": "idbM4ZJF", "bus": "pcie-root-port-8", "addr": "0x0"}' \
       -netdev  '{"id": "idbM4ZJF", "type": "tap", "vhost": true }'  \
       -device '{"id": "pcie-root-port-9", "port": 9, "driver": "pcie-root-port", "addr": "0x3.0x1", "bus": "pcie.0", "chassis": 10}' \
       -device '{"driver": "virtio-net-pci", "mac": "9a:17:08:ce:51:7a", "id": "id64P7qM", "mq": true, "vectors": 10, "netdev": "idJmBhbV", "bus": "pcie-root-port-9", "addr": "0x0"}' \
       -netdev  '{"id": "idJmBhbV", "type": "tap", "vhost": true }'  \
       -device '{"id": "pcie-root-port-10", "port": 10, "driver": "pcie-root-port", "addr": "0x3.0x2", "bus": "pcie.0", "chassis": 11}' \
       -device '{"driver": "virtio-net-pci", "mac": "9a:17:08:ce:51:7b", "id": "idQBqMGv", "mq": true, "vectors": 10, "netdev": "idBC4qIN", "bus": "pcie-root-port-10", "addr": "0x0"}' \
       -netdev  '{"id": "idBC4qIN", "type": "tap", "vhost": true }'  \
       -device '{"id": "pcie-root-port-11", "port": 11, "driver": "pcie-root-port", "addr": "0x3.0x3", "bus": "pcie.0", "chassis": 12}' \
       -device '{"driver": "virtio-net-pci", "mac": "9a:17:08:ce:51:7c", "id": "idSiRkp5", "mq": true, "vectors": 10, "netdev": "idgWOtLj", "bus": "pcie-root-port-11", "addr": "0x0"}' \
       -netdev  '{"id": "idgWOtLj", "type": "tap", "vhost": true }'  \
       -device '{"id": "pcie-root-port-12", "port": 12, "driver": "pcie-root-port", "addr": "0x3.0x4", "bus": "pcie.0", "chassis": 13}' \
       -device '{"driver": "virtio-net-pci", "mac": "9a:17:08:ce:51:7d", "id": "idtyBm2h", "mq": true, "vectors": 10, "netdev": "idwxCeel", "bus": "pcie-root-port-12", "addr": "0x0"}' \
       -netdev  '{"id": "idwxCeel", "type": "tap", "vhost": true }'  \
       -device '{"id": "pcie-root-port-13", "port": 13, "driver": "pcie-root-port", "addr": "0x3.0x5", "bus": "pcie.0", "chassis": 14}' \
       -device '{"driver": "virtio-net-pci", "mac": "9a:17:08:ce:51:7e", "id": "idSpJBwN", "mq": true, "vectors": 10, "netdev": "id8nsDxX", "bus": "pcie-root-port-13", "addr": "0x0"}' \
       -netdev  '{"id": "id8nsDxX", "type": "tap", "vhost": true }'  \
       -device '{"id": "pcie-root-port-14", "port": 14, "driver": "pcie-root-port", "addr": "0x3.0x6", "bus": "pcie.0", "chassis": 15}' \
       -device '{"driver": "virtio-net-pci", "mac": "9a:17:08:ce:51:7f", "id": "idxrFZcF", "mq": true, "vectors": 10, "netdev": "idduNyfB", "bus": "pcie-root-port-14", "addr": "0x0"}' \
       -netdev  '{"id": "idduNyfB", "type": "tap", "vhost": true }'  \
       -device '{"id": "pcie-root-port-15", "port": 15, "driver": "pcie-root-port", "addr": "0x3.0x7", "bus": "pcie.0", "chassis": 16}' \
       -device '{"driver": "virtio-net-pci", "mac": "9a:17:08:ce:51:80", "id": "idTVFqEi", "mq": true, "vectors": 10, "netdev": "idkv8GUE", "bus": "pcie-root-port-15", "addr": "0x0"}' \
       -netdev  '{"id": "idkv8GUE", "type": "tap", "vhost": true }'  \
       -device '{"id": "pcie-root-port-16", "port": 16, "driver": "pcie-root-port", "multifunction": true, "bus": "pcie.0", "addr": "0x4", "chassis": 17}' \
       -device '{"driver": "virtio-net-pci", "mac": "9a:17:08:ce:51:81", "id": "idKknAbY", "mq": true, "vectors": 10, "netdev": "idOdIaVe", "bus": "pcie-root-port-16", "addr": "0x0"}' \
       -netdev  '{"id": "idOdIaVe", "type": "tap", "vhost": true }'  \
       -device '{"id": "pcie-root-port-17", "port": 17, "driver": "pcie-root-port", "addr": "0x4.0x1", "bus": "pcie.0", "chassis": 18}' \
       -device '{"driver": "virtio-net-pci", "mac": "9a:17:08:ce:51:82", "id": "idzCjpcw", "mq": true, "vectors": 10, "netdev": "idiXvQfe", "bus": "pcie-root-port-17", "addr": "0x0"}' \
       -netdev  '{"id": "idiXvQfe", "type": "tap", "vhost": true }'  \
       -device '{"id": "pcie-root-port-18", "port": 18, "driver": "pcie-root-port", "addr": "0x4.0x2", "bus": "pcie.0", "chassis": 19}' \
       -device '{"driver": "virtio-net-pci", "mac": "9a:17:08:ce:51:83", "id": "idxQwxyh", "mq": true, "vectors": 10, "netdev": "idQhu1S5", "bus": "pcie-root-port-18", "addr": "0x0"}' \
       -netdev  '{"id": "idQhu1S5", "type": "tap", "vhost": true }'  \
       -device '{"id": "pcie-root-port-19", "port": 19, "driver": "pcie-root-port", "addr": "0x4.0x3", "bus": "pcie.0", "chassis": 20}' \
       -device '{"driver": "virtio-net-pci", "mac": "9a:17:08:ce:51:84", "id": "iduaojpr", "mq": true, "vectors": 10, "netdev": "idBWz22H", "bus": "pcie-root-port-19", "addr": "0x0"}' \
       -netdev  '{"id": "idBWz22H", "type": "tap", "vhost": true }'  \
       -device '{"id": "pcie-root-port-20", "port": 20, "driver": "pcie-root-port", "addr": "0x4.0x4", "bus": "pcie.0", "chassis": 21}' \
       -device '{"driver": "virtio-net-pci", "mac": "9a:17:08:ce:51:85", "id": "idVywJY3", "mq": true, "vectors": 10, "netdev": "idOfpMRV", "bus": "pcie-root-port-20", "addr": "0x0"}' \
       -netdev  '{"id": "idOfpMRV", "type": "tap", "vhost": true }'  \
       -device '{"id": "pcie-root-port-21", "port": 21, "driver": "pcie-root-port", "addr": "0x4.0x5", "bus": "pcie.0", "chassis": 22}' \
       -device '{"driver": "virtio-net-pci", "mac": "9a:17:08:ce:51:86", "id": "idErLzMA", "mq": true, "vectors": 10, "netdev": "idJhrsR8", "bus": "pcie-root-port-21", "addr": "0x0"}' \
       -netdev  '{"id": "idJhrsR8", "type": "tap", "vhost": true }'  \
       -device '{"id": "pcie-root-port-22", "port": 22, "driver": "pcie-root-port", "addr": "0x4.0x6", "bus": "pcie.0", "chassis": 23}' \
       -device '{"driver": "virtio-net-pci", "mac": "9a:17:08:ce:51:87", "id": "idNbrqhr", "mq": true, "vectors": 10, "netdev": "idbBo9G7", "bus": "pcie-root-port-22", "addr": "0x0"}' \
       -netdev  '{"id": "idbBo9G7", "type": "tap", "vhost": true }'  \
       -device '{"id": "pcie-root-port-23", "port": 23, "driver": "pcie-root-port", "addr": "0x4.0x7", "bus": "pcie.0", "chassis": 24}' \
       -device '{"driver": "virtio-net-pci", "mac": "9a:17:08:ce:51:88", "id": "idYrARhA", "mq": true, "vectors": 10, "netdev": "idKqvL11", "bus": "pcie-root-port-23", "addr": "0x0"}' \
       -netdev  '{"id": "idKqvL11", "type": "tap", "vhost": true }'  \
       -device '{"id": "pcie-root-port-24", "port": 24, "driver": "pcie-root-port", "multifunction": true, "bus": "pcie.0", "addr": "0x5", "chassis": 25}' \
       -device '{"driver": "virtio-net-pci", "mac": "9a:17:08:ce:51:89", "id": "idMcMRzj", "mq": true, "vectors": 10, "netdev": "idXn5zOZ", "bus": "pcie-root-port-24", "addr": "0x0"}' \
       -netdev  '{"id": "idXn5zOZ", "type": "tap", "vhost": true }'  \
       -device '{"id": "pcie-root-port-25", "port": 25, "driver": "pcie-root-port", "addr": "0x5.0x1", "bus": "pcie.0", "chassis": 26}' \
       -device '{"driver": "virtio-net-pci", "mac": "9a:17:08:ce:51:8a", "id": "id60AQZw", "mq": true, "vectors": 10, "netdev": "ide55RJg", "bus": "pcie-root-port-25", "addr": "0x0"}' \
       -netdev  '{"id": "ide55RJg", "type": "tap", "vhost": true }'  \
       -device '{"id": "pcie-root-port-26", "port": 26, "driver": "pcie-root-port", "addr": "0x5.0x2", "bus": "pcie.0", "chassis": 27}' \
       -device '{"driver": "virtio-net-pci", "mac": "9a:17:08:ce:51:8b", "id": "idwtd0bQ", "mq": true, "vectors": 10, "netdev": "idjMbJ6Y", "bus": "pcie-root-port-26", "addr": "0x0"}' \
       -netdev  '{"id": "idjMbJ6Y", "type": "tap", "vhost": true }'  \
       -device '{"id": "pcie-root-port-27", "port": 27, "driver": "pcie-root-port", "addr": "0x5.0x3", "bus": "pcie.0", "chassis": 28}' \
       -device '{"driver": "virtio-net-pci", "mac": "9a:17:08:ce:51:8c", "id": "idQuPMAN", "mq": true, "vectors": 10, "netdev": "idHwBACO", "bus": "pcie-root-port-27", "addr": "0x0"}' \
       -netdev  '{"id": "idHwBACO", "type": "tap", "vhost": true }'  \
       -device '{"id": "pcie-root-port-28", "port": 28, "driver": "pcie-root-port", "addr": "0x5.0x4", "bus": "pcie.0", "chassis": 29}' \
       -device '{"driver": "virtio-net-pci", "mac": "9a:17:08:ce:51:8d", "id": "idKqVrKf", "mq": true, "vectors": 10, "netdev": "id6Qkz00", "bus": "pcie-root-port-28", "addr": "0x0"}' \
       -netdev  '{"id": "id6Qkz00", "type": "tap", "vhost": true }'  \
       -device '{"id": "pcie-root-port-29", "port": 29, "driver": "pcie-root-port", "addr": "0x5.0x5", "bus": "pcie.0", "chassis": 30}' \
       -device '{"driver": "virtio-net-pci", "mac": "9a:17:08:ce:51:8e", "id": "idgbKSWw", "mq": true, "vectors": 10, "netdev": "idJk53VD", "bus": "pcie-root-port-29", "addr": "0x0"}' \
       -netdev  '{"id": "idJk53VD", "type": "tap", "vhost": true }'  \
       -blockdev '{"node-name": "drive_cd1", "driver": "file", "read-only": true, "discard": "unmap", "aio": "threads", "filename": "/home/kvm_autotest_root/iso/windows/winutils.iso", "cache": {"direct": true, "no-flush": false}}' \
       -device '{"driver": "scsi-cd", "id": "cd1", "drive": "drive_cd1", "write-cache": "on"}'  \
       -vnc 0.0.0.0:17  \
       -rtc base=localtime,clock=host,driftfix=slew  \
       -boot menu=off,order=cdn,once=c,strict=off \
      -chardev socket,id=char_vtpm_tpm0,path=/tmp/guest-swtpmq.sock \
      -tpmdev emulator,chardev=char_vtpm_tpm0,id=emulator_vtpm_tpm0  \
      -device tpm-crb,id=tpm-crb_vtpm_tpm0,tpmdev=emulator_vtpm_tpm0 \
       -enable-kvm \
       -device '{"id": "pcie_extra_root_port_0", "driver": "pcie-root-port", "multifunction": true, "bus": "pcie.0", "addr": "0x6", "chassis": 31}'
      

        1. image.png
          image.png
          1.92 MB
        2. Ping_mutli-nics.txt
          12 kB
        3. TraceView_mutli-nics.out
          2.03 MB
        4. image-2024-11-12-18-22-47-155.png
          image-2024-11-12-18-22-47-155.png
          125 kB

              yuri.benditovich Yuri Benditovich
              rh-ee-wji Wenkang Ji
              Virt Windows SST Bugs Virt Windows SST Bugs
              Wenkang Ji Wenkang Ji
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

                Created:
                Updated: