Loading...

Type: Bug
Resolution: Unresolved
Priority: Normal
Fix Version/s: rhel-10.0
Affects Version/s: rhel-10.0.beta
Component/s: qemu-kvm / Devices
Labels:
- VFIO
- automation
- qe_no_customer_impact
- vIOMMU
- vfio-pf
- virt-arm

Fixed in Build:
qemu-kvm-9.1.0-1.el10
Regression:
None
Severity:
Moderate

Pool Team:

sst_virtualization
Sub-System Group:

ssg_virtualization

Story Points:
3
ACKs Check:

QE ack, Dev ack
Blocked:
False
Blocked Reason:

Hide

None

Show
None
Product Documentation Required:
None
Products:

Red Hat Enterprise Linux
Sprint:
None

Preliminary Testing:
Pass
Test Link:
https://polarion.engineering.redhat.com/polarion/#/project/RHELVIRT/workitem?id=VIRT-11435
Errata Link:
https://errata.engineering.redhat.com/advisory/139510
Test Coverage:

Automated

Experience:
Architecture:

x86_64
OS:
Linux

SFDC Cases Counter:
SFDC Cases Open:
SFDC Cases Links:

Planning:
None

What were you trying to do that didn't work?
The qemu-kvm throws VFIO_MAP_DMA error when hot-plug 2 E810 PF into the VM has virtio-iommu device

Please provide the package NVR for which bug is seen:
Test env:
qemu-kvm-9.0.0-7.el10.x86_64

How reproducible:
100%

Steps to reproduce
[1] start a VM with virtio+iommu device
[2] hot-plug 2 ice PFs into the VM

# virsh attach-device rhel10 /tmp/device/0000\:3b\:00.0.xml 
# virsh attach-device rhel10 /tmp/device/0000\:3b\:00.1.xml

[3] check the VM dmesg

[   46.478977] pci 0000:04:00.0: [8086:1592] type 00 class 0x020000 PCIe Endpoint
[   46.479564] pci 0000:04:00.0: BAR 0 [mem 0x00000000-0x01ffffff 64bit pref]
[   46.479930] pci 0000:04:00.0: BAR 3 [mem 0x00000000-0x0000ffff 64bit pref]
[   46.480283] pci 0000:04:00.0: ROM [mem 0x00000000-0x000fffff pref]
[   46.480496] pci 0000:04:00.0: Max Payload Size set to 128 (was 256, max 512)
[   46.483089] pci 0000:04:00.0: 126.016 Gb/s available PCIe bandwidth, limited by 8.0 GT/s PCIe x16 link at 0000:00:02.3 (capable of 252.048 Gb/s with 16.0 GT/s PCIe x16 link)
[   46.484899] pci 0000:04:00.0: BAR 0 [mem 0x381800000000-0x381801ffffff 64bit pref]: assigned
[   46.485070] pci 0000:04:00.0: ROM [mem 0x82800000-0x828fffff pref]: assigned
[   46.485085] pci 0000:04:00.0: BAR 3 [mem 0x381802000000-0x38180200ffff 64bit pref]: assigned
[   46.509326] gnss: GNSS driver registered with major 239
[   46.603492] ice: Intel(R) Ethernet Connection E800 Series Linux Driver
[   46.603501] ice: Copyright (c) 2018, Intel Corporation.
[   46.603643] ice 0000:04:00.0: Adding to iommu group 22
[   46.714704] ice 0000:04:00.0: enabling device (0000 -> 0002)
[   47.198643] ice 0000:04:00.0: DDP package already present on device: ICE OS Default Package version 1.3.36.0
[   47.563638] ice 0000:04:00.0: 126.016 Gb/s available PCIe bandwidth, limited by 8.0 GT/s PCIe x16 link at 0000:00:02.3 (capable of 252.048 Gb/s with 16.0 GT/s PCIe x16 link)
[   47.588522] ice 0000:04:00.0: PTP init successful
[   47.756508] ice 0000:04:00.0: DCB is enabled in the hardware, max number of TCs supported on this port are 8
[   47.757173] ice 0000:04:00.0: FW LLDP is disabled, DCBx/LLDP in SW mode.
[   47.757383] ice 0000:04:00.0: Commit DCB Configuration to the hardware
[   47.772831] ice 0000:04:00.0 enp4s0np0: renamed from eth0
[   47.964357] i40e: Intel(R) Ethernet Connection XL710 Network Driver
[   47.964367] i40e: Copyright (c) 2013 - 2019 Intel Corporation.
[   48.737604] ice 0000:04:00.0: Using 64-bit DMA addresses
[   49.896298] ice 0000:04:00.0: IRDMA hardware initialization FAILED init_state=4 status=-110
[   50.050257] irdma ice.roce.1: probe with driver irdma failed with error -110
[   57.460477] pci 0000:05:00.0: [8086:1592] type 00 class 0x020000 PCIe Endpoint
[   57.461136] pci 0000:05:00.0: BAR 0 [mem 0x00000000-0x01ffffff 64bit pref]
[   57.461551] pci 0000:05:00.0: BAR 3 [mem 0x00000000-0x0000ffff 64bit pref]
[   57.461817] pci 0000:05:00.0: ROM [mem 0x00000000-0x000fffff pref]
[   57.461998] pci 0000:05:00.0: Max Payload Size set to 128 (was 256, max 512)
[   57.465559] pci 0000:05:00.0: 126.016 Gb/s available PCIe bandwidth, limited by 8.0 GT/s PCIe x16 link at 0000:00:02.4 (capable of 252.048 Gb/s with 16.0 GT/s PCIe x16 link)
[   57.472155] pci 0000:05:00.0: BAR 0 [mem 0x382000000000-0x382001ffffff 64bit pref]: assigned
[   57.472753] pci 0000:05:00.0: ROM [mem 0x82600000-0x826fffff pref]: assigned
[   57.472768] pci 0000:05:00.0: BAR 3 [mem 0x382002000000-0x38200200ffff 64bit pref]: assigned
[   57.473395] ice 0000:05:00.0: Adding to iommu group 23
[   57.590319] ice 0000:05:00.0: enabling device (0000 -> 0002)
[   58.082219] ice 0000:05:00.0: DDP package already present on device: ICE OS Default Package version 1.3.36.0
[   58.314514] ice 0000:04:00.0 enp4s0np0: NIC Link is up 100 Gbps Full Duplex, Requested FEC: RS-FEC, Negotiated FEC: RS-FEC, Autoneg Advertised: On, Autoneg Negotiated: True, Flow Control: None
[   58.544625] ice 0000:05:00.0: 126.016 Gb/s available PCIe bandwidth, limited by 8.0 GT/s PCIe x16 link at 0000:00:02.4 (capable of 252.048 Gb/s with 16.0 GT/s PCIe x16 link)
[   58.556146] ice 0000:05:00.0 enp5s0np1: renamed from eth0
[   58.884501] ice 0000:05:00.0: Using 64-bit DMA addresses
[   60.081772] ice 0000:05:00.0: IRDMA hardware initialization FAILED init_state=4 status=-110
*[   60.214655] irdma ice.roce.2: probe with driver irdma failed with error -110*
[   60.215637] ice 0000:05:00.0: PTP init successful
[   60.412954] ice 0000:05:00.0: DCB is enabled in the hardware, max number of TCs supported on this port are 8
[   60.412964] ice 0000:05:00.0: FW LLDP is disabled, DCBx/LLDP in SW mode.
[   60.413134] ice 0000:05:00.0: Commit DCB Configuration to the hardware
[   68.594941] ice 0000:05:00.0 enp5s0np1: NIC Link is up 100 Gbps Full Duplex, Requested FEC: RS-FEC, Negotiated FEC: RS-FEC, Autoneg Advertised: On, Autoneg Negotiated: True, Flow Control: None

[4] check the qemu-kvm log

# cat /var/log/libvirt/qemu/rhel10.log
2024-09-03T03:36:56.737898Z qemu-kvm: warning: virtio-iommu-memory-region-29-0: Notified about new host reserved regions after probe
2024-09-03T03:36:58.761690Z qemu-kvm: VFIO_MAP_DMA failed: Invalid argument
2024-09-03T03:36:58.761726Z qemu-kvm: vfio_container_dma_map(0x562ddf98c5d0, 0xffffffffffe00000, 0x200000, 0x7f51f0000000) = -22 (Invalid argument)
...
2024-09-03T03:39:28.900136Z qemu-kvm: VFIO_MAP_DMA failed: Invalid argument
2024-09-03T03:39:28.900173Z qemu-kvm: vfio_container_dma_map(0x562ddf98c5d0, 0xfffffffffffed000, 0x1000, 0x7f5143183000) = -22 (Invalid argument)
2024-09-03T03:42:02.166104Z qemu-kvm: terminating on signal 15 from pid 11678 (/usr/sbin/virtqemud)
2024-09-03 03:42:02.636+0000: shutting down, reason=shutdown

Expected results
No VFIO_MAP_DMA error

Actual results
The qemu-kvm throws VFIO_MAP_DMA error

clones

RHEL-20450 [virtio-iommu][ice][hot-plug] virtio-iommu driver is not properly notified about the reserved regions

In Progress

is blocked by

RHEL-41246 Rebase qemu-9.1 for RHEL 10.0

Integration

links to

RHBA-2024:139510 qemu-kvm update

Details

Description

Attachments

Issue Links

Activity

People

Dates