What were you trying to do that didn't work?

Red Hat Enterprise Linux 9 on VMware ESXi with 3+ TiB of memory with AMD CPUs can lock up when both the hypervisor is providing a vIOMMU to guests and the guest has IOMMU enabled. VMware indicates their vIOMMU address space is deliberately set to 40 bits, however, for amd iommu v1, we assume it can handle 64-bits and for v2, we assume 48-bits. This can result in mistranslations in the guest to the vIOMMU and result in a hang, in particular where sg is used for iommu mappings (scsi_map_page for example). Setting iommu=pt` seems to work around the issue.

What is the impact of this issue to you?

Intermittent hangs

Please provide the package NVR for which the bug is seen:

5.14.0-503.26.1.el9_5.x86_64

How reproducible is this bug?:

Steps to reproduce

1. Have a RHEL system on VMware with over 1 TiB of memory
2. induce heavy IO activity
3. monitor for unresponsiveness

Expected results

operational

Actual results

hang eventually