Uploaded image for project: 'RHEL'
  1. RHEL
  2. RHEL-43408

Guest fails to boot with Intel IOMMU and > 1T of RAM

Linking RHIVOS CVEs to...Migration: Automation ...SWIFT: POC ConversionSync from "Extern...XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Undefined Undefined
    • None
    • rhel-9.2.0, rhel-9.3.0, rhel-9.4
    • qemu-kvm / Devices
    • None
    • Moderate
    • rhel-virt-core
    • ssg_virtualization
    • 5
    • False
    • False
    • Hide

      None

      Show
      None
    • None
    • None
    • None
    • None
    • x86_64
    • None

      What were you trying to do that didn't work?

      Setting up a VM with a large number of CPUs (>255) requires an IOMMU device, such as below

          <iommu model='intel'>
            <driver intremap='on' eim='on'/>
          </iommu>
      

       

      However, if the VM has a large about of memory (i.e. >= 1T), the Guest will then fail to boot with a loop on this:

       

      [    1.303054] DMAR: VT-d detected Invalidation Queue Error: Reason 0
      [    1.303054] DMAR: QI PRIOR: UNKNOWN qw0 = 0x0, qw1 = 0x0
      [    1.303054] DMAR: VT-d detected Invalidation Queue Error: Reason 0
      [    1.303054] DMAR: VT-d detected Invalidation Queue Error: Reason 0
      [    1.303054] DMAR: QI HEAD: Invalidation Wait qw0 = 0x200000025, qw1 = 0x10000061c04
      [    1.303054] DMAR: VT-d detected Invalidation Queue Error: Reason 0
      [    1.303054] DMAR: VT-d detected Invalidation Queue Error: Reason 0
      [    1.303054] DMAR: VT-d detected Invalidation Queue Error: Reason 0
      [    1.303054] DMAR: VT-d detected Invalidation Queue Error: Reason 0
      [    1.303054] DMAR: VT-d detected Invalidation Queue Error: Reason 0
      [    1.303054] DMAR: VT-d detected Invalidation Queue Error: Reason 0
      [    1.303054] DMAR: QI HEAD: Invalidation Wait qw0 = 0x200000025, qw1 = 0x10000061c04
      [    1.303054] DMAR: VT-d detected Invalidation Queue Error: Reason 0
       
      

       

      As a workaroud, booting the guest with the below in the command line makes it boot fine.
      intremap=off

      Please provide the package NVR for which bug is seen:

      Both Guest and Host RHEL 9.4

       

      kernel-5.14.0-427.23.1.el9_4.x86_64
      qemu-kvm-8.2.0-11.el9_4.3.x86_64
      

       

      How reproducible:

      Always

      Steps to reproduce

      1. Get a large machine for hypervisor
      2. Setup a VM with:
        • >=1 T of RAM
        • 255+ vCPUS (not really required to reproduce but its part of the use case)
        • Enable IOMMU with <driver intremap='on' eim='on'/>
      3. Boot it

      Expected results

      Guest Runs

      Actual results

      Guest Hangs

              mtsirkin Michael S Tsirkin (Inactive)
              rhn-support-gveitmic Germano Veit Michel
              Amnon Ilan
              virt-maint virt-maint
              Yiqian Wei Yiqian Wei
              Votes:
              0 Vote for this issue
              Watchers:
              17 Start watching this issue

                Created:
                Updated: