Uploaded image for project: 'RHEL'
  1. RHEL
  2. RHEL-20450

[virtio-iommu][ice][hot-plug] virtio-iommu driver is not properly notified about the reserved regions

    • qemu-kvm-9.1.0-1.el9
    • None
    • Moderate
    • sst_virtualization
    • ssg_virtualization
    • 3
    • QE ack
    • False
    • Hide

      None

      Show
      None
    • None
    • Red Hat Enterprise Linux
    • None
    • x86_64
    • Linux
    • None

      What were you trying to do that didn't work?
      The qemu-kvm throws VFIO_MAP_DMA error when hot-plug 2 E810 PF into the VM has virtio-iommu device

      Please provide the package NVR for which bug is seen:
      Test env:
      qemu-kvm-8.2.0-1.el9.x86_64
      5.14.0-402.el9.x86_64

      How reproducible:
      100%

      Steps to reproduce
      [1] start a VM with virtio+iommu device
      [2] hot-plug 2 ice PFs into the VM
      [3] check the VM dmesg

      [   29.016009] pci 0000:04:00.0: [8086:1592] type 00 class 0x020000
      [   29.016631] pci 0000:04:00.0: reg 0x10: [mem 0x00000000-0x01ffffff 64bit pref]
      [   29.017595] pci 0000:04:00.0: reg 0x1c: [mem 0x00000000-0x0000ffff 64bit pref]
      [   29.018235] pci 0000:04:00.0: reg 0x30: [mem 0x00000000-0x000fffff pref]
      [   29.018672] pci 0000:04:00.0: Max Payload Size set to 128 (was 256, max 512)
      [   29.024645] pci 0000:04:00.0: 126.016 Gb/s available PCIe bandwidth, limited by 8.0 GT/s PCIe x16 link at 0000:00:02.3 (capable of 252.048 Gb/s with 16.0 GT/s PCIe x16 link)
      [   29.027846] pci 0000:04:00.0: BAR 0: assigned [mem 0x386000000000-0x386001ffffff 64bit pref]
      [   29.027942] pci 0000:04:00.0: BAR 6: assigned [mem 0xfe200000-0xfe2fffff pref]
      [   29.027944] pci 0000:04:00.0: BAR 3: assigned [mem 0x386002000000-0x38600200ffff 64bit pref]
      [   29.028037] pcieport 0000:00:02.3: PCI bridge to [bus 04]
      [   29.028049] pcieport 0000:00:02.3:   bridge window [io  0xe000-0xefff]
      [   29.031133] pcieport 0000:00:02.3:   bridge window [mem 0xfe200000-0xfe3fffff]
      [   29.032875] pcieport 0000:00:02.3:   bridge window [mem 0x386000000000-0x3867ffffffff 64bit pref]
      [   29.052285] gnss: GNSS driver registered with major 239
      [   29.136669] ice: Intel(R) Ethernet Connection E800 Series Linux Driver
      [   29.136672] ice: Copyright (c) 2018, Intel Corporation.
      [   29.136788] ice 0000:04:00.0: Adding to iommu group 22
      [   29.281754] ice 0000:04:00.0: enabling device (0000 -> 0002)
      [   29.689252] ice 0000:04:00.0: DDP package already present on device: ICE OS Default Package version 1.3.30.0
      [   30.170093] ice 0000:04:00.0: 126.016 Gb/s available PCIe bandwidth, limited by 8.0 GT/s PCIe x16 link at 0000:00:02.3 (capable of 252.048 Gb/s with 16.0 GT/s PCIe x16 link)
      [   30.189160] ice 0000:04:00.0: PTP init successful
      [   30.383208] ice 0000:04:00.0: DCB is enabled in the hardware, max number of TCs supported on this port are 8
      [   30.383211] ice 0000:04:00.0: FW LLDP is disabled, DCBx/LLDP in SW mode.
      [   30.383376] ice 0000:04:00.0: Commit DCB Configuration to the hardware
      [   30.498048] ice 0000:04:00.0 enp4s0: renamed from eth0
      [   30.701335] i40e: Intel(R) Ethernet Connection XL710 Network Driver
      [   30.701338] i40e: Copyright (c) 2013 - 2019 Intel Corporation.
      [   31.196193] ice 0000:04:00.0: Using 64-bit DMA addresses
      [   32.290574] ice 0000:04:00.0: IRDMA hardware initialization FAILED init_state=4 status=-110
      [   32.415252] irdma: probe of ice.roce.1 failed with error -110
      

      [4] check the qemu-kvm log

      # cat /var/log/libvirt/qemu/rhel94.log
      2024-01-03T03:14:32.683225Z qemu-kvm: warning: virtio-iommu-memory-region-29-0: Notified about new host reserved regions after probe
      2024-01-03T03:14:34.850471Z qemu-kvm: VFIO_MAP_DMA failed: Invalid argument
      2024-01-03T03:14:34.850500Z qemu-kvm: vfio_dma_map(0x55e92bdbc010, 0xffffffffffe00000, 0x200000, 0x7fa983800000) = -22 (Invalid argument)
      ......
      2024-01-03T03:17:03.138369Z qemu-kvm: VFIO_MAP_DMA failed: Invalid argument
      2024-01-03T03:17:03.138418Z qemu-kvm: vfio_dma_map(0x55e92bdbc010, 0xfffffffffffdb000, 0x1000, 0x7fa9ad285000) = -22 (Invalid argument)
      

      Expected results
      No VFIO_MAP_DMA error

      Actual results
      The qemu-kvm throws VFIO_MAP_DMA error

            eauger Eric Auger
            yanghliu@redhat.com YangHang Liu
            virt-maint virt-maint
            YangHang Liu YangHang Liu
            Votes:
            0 Vote for this issue
            Watchers:
            10 Start watching this issue

              Created:
              Updated: