Uploaded image for project: 'RHEL'
  1. RHEL
  2. RHEL-83143

Live migration after workload update fails with operation failed: guest CPU doesn't match specification: missing features: waitpkg

Linking RHIVOS CVEs to...Migration: Automation ...Sync from "Extern...XMLWordPrintable

    • qemu-kvm-8.2.0-11.el9_4.11
    • None
    • Moderate
    • ZStream
    • rhel-virt-hwe
    • ssg_virtualization
    • 5
    • Dev ack
    • False
    • False
    • Hide

      None

      Show
      None
    • None
    • None
    • Approved Blocker
    • Unspecified
    • Unspecified
    • Unspecified
    • None

      Description of problem:

      After upgrade of CNV from v4.15.8 to v4.16.5, live migration of VM for workload update fails with 'operation failed: guest CPU doesn't match specification: missing features: waitpkg'
      ~~~
      2025-01-22T17:39:25.458557070Z {"component":"virt-launcher","level":"info","msg":"2025-01-22T17:39:24.907135Z qemu-kvm: warning: This feature depends on other features that were not requested: CPUID.07H:ECX.waitpkg [bit 5]","subcomponent":"qemu","timestamp":"2025-01-22T17:39:25.458323Z"}
      2025-01-22T17:39:25.941847222Z {"component":"virt-launcher","level":"error","msg":"operation failed: guest CPU doesn't match specification: missing features: waitpkg","pos":"virCPUx86UpdateLive:3107","subcomponent":"libvirt","thread":"33","timestamp":"2025-01-22T17:39:25.941000Z"}
      ~~~
      The only way to resolve this is to stop/start the VM.  After stopping and starting the VM, a migration succeeds, so the issue is only seen during the upgrade window. 
      The failing migration source pod:
      
      2025-01-11T20:32:45.827149786Z {"component":"virt-launcher","level":"info","msg":"libvirt version: 9.0.0, package: 10.10.el9_2 (Red Hat, Inc. <http://bugzilla.redhat.com/bugzilla>, 2024-09-12-10:51:56, )","subcomponent":"libvirt","thread":"45","timestamp":"2025-01-11T20:32:45.826000Z"}2025-01-22T17:40:17.990830574Z {"component":"virt-launcher","level":"error","msg":"operation failed: guest CPU doesn't match specification: missing features: waitpkg","pos":"virNetClientProgramDispatchError:172","subcomponent":"libvirt","thread":"31","timestamp":"2025-01-22T17:40:17.990000Z"}
      
      2025-01-22T17:40:17.997873194Z {"component":"virt-launcher","kind":"","level":"error","msg":"Live migration failed.","name":"ood-1","namespace":"example","pos":"live-migration-source.go:969","reason":"error encountered during MigrateToURI3 libvirt api call: virError(Code=9, Domain=31, Message='operation failed: guest CPU doesn't match specification: missing features: waitpkg')","timestamp":"2025-01-22T17:40:17.997579Z","uid":"369eb6f6-f134-4a6d-9f8c-a630a2eb06cc"}
      
      2025-01-22T17:40:18.083291028Z {"component":"virt-launcher","kind":"","level":"error","msg":"Received a live migration error. Will check the latest migration status.","name":"ood-1","namespace":"example","pos":"live-migration-source.go:714","reason":"error encountered during MigrateToURI3 libvirt api call: virError(Code=9, Domain=31, Message='operation failed: guest CPU doesn't match specification: missing features: waitpkg')","timestamp":"2025-01-22T17:40:18.082943Z","uid":"369eb6f6-f134-4a6d-9f8c-a630a2eb06cc"}
      
      The xml generated has:<feature policy=\"require\" name=\"waitpkg\">
      
      
      cat /proc/cpuinfo on the source and dest nodes both show  (abbreviated) 
      flags		: waitpkg

      Version-Release number of selected component (if applicable):

      source pod for migration:  libvirt version: 9.0.0, package: 10.10.el9_2 qemu version: 7.2.0qemu-kvm-7.2.0-14.el9_2.15,  kernel: 5.14.0-427.49.1.el9_4.x86_64
      
      dest pod for migration:  libvirt version: 10.0.0, package: 6.12.el9_4  qemu version: 8.2.0qemu-kvm-8.2.0-11.el9_4.8 kernel: 5.14.0-427.49.1.el9_4.x86_64,

      How reproducible:

      Customer upgraded from CNV v4.15.8 to v4.16.5

      Expected results:

      Workload Update Live migrations succeed.
       

        1. vm2.log
          6 kB
        2. virtqemud.log
          1024 kB
        3. screenshot-1.png
          screenshot-1.png
          45 kB
        4. rhel94.xml
          7 kB
        5. qemu-source.txt
          6 kB
        6. qemu-destination.txt
          6 kB
        7. domain.xml
          9 kB
        8. debugLogsForLibvirt.txt
          62 kB

              rhn-engineering-imammedo Igor Mammedov
              shaselde@redhat.com Sean Haselden
              virt-maint virt-maint
              NaNa Liu NaNa Liu
              Votes:
              0 Vote for this issue
              Watchers:
              21 Start watching this issue

                Created:
                Updated:
                Resolved: