What were you trying to do that didn't work?

What is the impact of this issue to you?

Please provide the package NVR for which the bug is seen:

RHEL 10.2

kernel-6.12.0-170.el10.x86_64
qemu-kvm-10.1.0-7.el10.x86_64
edk2-ovmf-20250822-4.el10.noarch
seabios-bin-1.16.3-8.el10.noarch

RHEL 9.2.z

kernel-5.14.0-284.148.1.el9_2.x86_64
qemu-kvm-7.2.0-14.el9_2.22.x86_64
edk2-ovmf-20221207gitfff6d81270b5-9.el9_2.8.noarch
seabios-bin-1.16.1-1.el9.noarch

How reproducible is this bug?:

1.boot up a VM on RHEL 10.2 host first

Cmdline = 

/usr/libexec/qemu-kvm -name "mouse-vm" -sandbox on -machine pc-q35-rhel9.2.0,pflash0=drive_ovmf_code,pflash1=drive_ovmf_vars -nodefaults  -cpu EPYC,hv_stimer,hv_synic,hv_vpindex,hv_relaxed,hv_spinlocks=0x1fff,hv_vapic,hv_time,hv_frequencies,hv_runtime,hv_tlbflush,hv_reenlightenment,hv_stimer_direct,hv_ipi,+kvm_pv_unhalt,host-phys-bits-limit=39 -chardev socket,id=charmonitor,path=/home/tmp1,server=on,wait=off -chardev pty,id=charserial0 -chardev socket,id=charchannel1,path=/home/tmp2,server=on,wait=off -chardev socket,id=seabioslog_id,path=/home/seabios,server=on,wait=off -device ide-cd,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 -device ahci,id=ahci0,bus=pcie.0,addr=0x3 -device ide-cd,bus=ahci0.0,unit=0,drive=drive-ide0-1-1,id=ide0-1-1 -device '{"driver":"pcie-root-port","port":1,"chassis":1,"id":"pcie-root-port0","multifunction":true,"bus":"pcie.0","addr":"0x4"}' -device '{"driver":"virtio-scsi-pci","id":"scsi0","bus":"pcie-root-port0"}' -device '{"driver":"pcie-root-port","port":2,"chassis":2,"id":"pcie-root-port1","bus":"pcie.0","addr":"0x4.0x1"}' -device '{"driver":"scsi-hd","bus":"scsi0.0","lun":0,"drive":"drive-virtio-disk0","id":"virtio-disk0","bootindex":1,"serial":"test_test_test_0","device_id":"drive-virtio-disk0"}' -device '{"driver":"scsi-hd","drive":"drive-scsi-disk","bus":"scsi0.0","lun":1,"id":"data-disk1","serial":"test_test_test_1","device_id":"drive-virtio-disk1"}' -device '{"driver":"pcie-root-port","port":3,"chassis":3,"id":"pcie-root-port2","bus":"pcie.0","addr":"0x4.0x2"}' -device '{"driver":"virtio-serial-pci","id":"virtio-serial0","bus":"pcie-root-port2"}' -device '{"driver":"isa-serial","id":"serial0","chardev":"charserial0"}' -device '{"driver":"intel-hda","id":"sound0","bus":"pcie.0","addr":"0x7"}' -device '{"driver":"hda-duplex","id":"sound0-codec0","bus":"sound0.0","cad":0,"audiodev": "audio1"}' -device '{"driver":"intel-hda","id":"sound1","bus":"pcie.0","addr":"0x8"}' -device '{"driver":"hda-micro","id":"sound1-codec0","bus":"sound1.0","audiodev": "audio1"}' -device '{"driver":"intel-hda","id":"sound2","bus":"pcie.0","addr":"0x9"}' -device '{"driver":"hda-output","id":"sound2-codec0","bus":"sound2.0","cad":0,"audiodev": "audio1"}' -device '{"driver":"ich9-intel-hda","id":"sound3","bus":"pcie.0","addr":"0xa"}' -device '{"driver":"hda-duplex","id":"sound3-codec0","bus":"sound3.0","cad":0,"audiodev":"audio1"}' -device '{"driver":"pvpanic","ioport":1285}' -device '{"driver":"pcie-root-port","port":4,"chassis":4,"id":"pcie-root-port3","bus":"pcie.0","addr":"0x4.0x3"}' -device '{"driver":"e1000e","netdev":"hostnet1","id":"virtio-net-pci1","mac":"52:54:ba:e9:d3:a1","bus":"pcie-root-port3"}' -device '{"driver":"pcie-root-port","port":5,"chassis":5,"id":"pcie-root-port4","bus":"pcie.0","addr":"0x4.0x4"}' -device '{"driver":"virtio-net-pci","netdev":"hostnet2","id":"virtio-net-pci2","mac":"52:54:54:13:5a:45","bus":"pcie-root-port4"}' -device ide-hd,drive=drive-data-disk,id=system-disk,logical_block_size=512,physical_block_size=512,min_io_size=512,opt_io_size=512,discard_granularity=512,ver=fuxc-ver,bus=ide.0,unit=0 -device '{"driver":"pcie-root-port","port":6,"chassis":6,"id":"pcie-root-port5","bus":"pcie.0","addr":"0x4.0x5"}' -device '{"driver":"ich9-usb-uhci6","id":"uhci6","bus":"pcie-root-port5"}' -device '{"driver":"usb-kbd","id":"kdb0","bus":"uhci6.0"}' -device '{"driver":"pcie-root-port","port":7,"chassis":7,"id":"pcie-root-port6","bus":"pcie.0","addr":"0x4.0x6"}' -device '{"driver":"ich9-usb-uhci5","id":"uhci5","bus":"pcie-root-port6"}' -device '{"driver":"usb-mouse","id":"mouse0","bus":"uhci5.0"}' -device '{"driver":"pcie-root-port","port":8,"chassis":8,"id":"pcie-root-port7","bus":"pcie.0","addr":"0x4.0x7"}' -device '{"driver":"qemu-xhci","id":"xhci","bus":"pcie-root-port7"}' -device '{"driver":"pcie-root-port","port":9,"id":"pcie-root-port8","multifunction":true,"bus":"pcie.0","addr":"0x10"}' -device '{"driver":"usb-ehci","id":"ehci","bus":"pcie-root-port8"}' -device '{"driver":"pcie-root-port","chassis":10,"port":10,"id":"pcie-root-port9","bus":"pcie.0","addr":"0x10.0x1"}' -device '{"driver":"piix3-usb-uhci","id":"usb","bus":"pcie-root-port9"}' -device '{"driver":"pcie-root-port","port":11,"chassis":11,"id":"pcie-root-port10","bus":"pcie.0","addr":"0x10.0x2"}' -device '{"driver":"ich9-usb-uhci3","id":"uhci","bus":"pcie-root-port10"}' -device '{"driver":"usb-storage","drive":"drive-usb-0","id":"usb-0","removable":true,"bus":"uhci.0","port":"1"}' -device '{"driver":"pcie-root-port","port":12,"chassis":12,"id":"pcie-root-port11","bus":"pcie.0","addr":"0x10.0x3"}' -device '{"driver":"pcie-root-port","port":13,"chassis":13,"id":"pcie-root-port12","bus":"pcie.0","addr":"0x10.0x4"}' -device '{"driver":"ich9-usb-ehci1","id":"ehci1","bus":"pcie-root-port11"}' -device '{"driver":"usb-storage","drive":"drive-usb-1","id":"usb-1","removable":true,"bus":"ehci.0","port":"1"}' -device '{"driver":"qemu-xhci","id":"xhci1","bus":"pcie-root-port12"}' -device '{"driver":"usb-storage","drive":"drive-usb-2","id":"usb-2","removable":true,"bus":"xhci1.0","port":"1"}' -device '{"driver":"pcie-root-port","port":14,"chassis":14,"id":"pcie-root-port13","bus":"pcie.0","addr":"0x10.0x5"}' -device '{"driver":"virtio-rng-pci","id":"rng0","bus":"pcie-root-port13","rng":"objrng0"}' -device '{"driver":"pcie-root-port","port":15,"chassis":15,"id":"pcie-root-port14","bus":"pcie.0","addr":"0x10.0x6"}' -device '{"driver":"virtio-balloon-pci","id":"balloon0","bus":"pcie-root-port14"}' -device isa-debugcon,chardev=seabioslog_id,iobase=0x402 -device virtio-vga -device '{"driver":"pcie-root-port","port":16,"chassis":16,"id":"pcie-root-port15","bus":"pcie.0","addr":"0x10.0x7"}' -device '{"driver":"pcie-pci-bridge","id":"pci.1","bus":"pcie-root-port15","addr":"0x0"}' -device '{"driver":"i6300esb","id":"wdt0","bus":"pci.1","addr":"0x1"}' -device '{"driver":"VGA","id":"video0","vgamem_mb":16,"bus":"pcie.0"}' -device '{"driver":"pcie-root-port","port":17,"chassis":17,"id":"pcie-root-port16","bus":"pcie.0","addr":"0x11"}' -device '{"driver":"vhost-vsock-pci","id":"vhost_vsock0","guest-cid":3,"bus":"pcie-root-port16"}' -device '{"driver":"piix3-usb-uhci","id":"piix3-usb-uhci0","bus":"pci.1","addr":"0x4"}' -device '{"driver":"pcie-root-port","port":18,"chassis":19,"id":"pcie-root-port18","bus":"pcie.0","addr":"0x12","multifunction":true}' -device '{"driver":"pcie-root-port","port":25,"chassis":20,"id":"pci.10","bus":"pcie.0","addr":"0x12.0x1"}' -device '{"driver":"nec-usb-xhci","p2":4,"p3":4,"id":"nex-usb-xhci0","bus":"pci.10","addr":"0x0"}' -audiodev '{"id":"audio1","driver":"none"}' -blockdev '{"driver":"file","cache":

{"direct":true,"no-flush":false}

,"filename":"/home/rhel920-nfs/a1.iso","node-name":"drive_sys1"}' -blockdev '{"driver":"raw","node-name":"drive-ide0-1-0","file":"drive_sys1"}' -blockdev '{"driver":"file","cache":

{"direct":true,"no-flush":false}

,"filename":"/home/rhel920-nfs/a2.iso","node-name":"drive_sys2"}' -blockdev '{"driver":"raw","node-name":"drive-ide0-1-1","file":"drive_sys2"}' -blockdev '{"driver":"file","cache":

{"direct":true,"no-flush":false}

,"filename":"/home/rhel920-nfs/rhel920-64-virtio-scsi-ovmf.qcow2","node-name":"drive_sys3"}' -blockdev '{"driver":"qcow2","node-name":"drive-virtio-disk0","file":"drive_sys3"}' -blockdev '{"driver":"file","cache":

{"direct":true,"no-flush":false}

,"filename":"/home/rhel920-nfs/d2.qcow2","node-name":"drive_sys4"}' -blockdev '{"driver":"qcow2","node-name":"drive-scsi-disk","file":"drive_sys4"}' -blockdev '{"driver":"file","cache":

{"direct":true,"no-flush":false}

,"filename":"/home/rhel920-nfs/d3.raw","node-name":"drive_sys5"}' -blockdev '{"driver":"raw","node-name":"drive-data-disk","file":"drive_sys5"}' -blockdev '{"driver":"file","cache":

{"direct":true,"no-flush":false}

,"filename":"/home/rhel920-nfs/d4.qcow2","node-name":"drive_sys6"}' -blockdev '{"driver":"qcow2","node-name":"usb-disk1","file":"drive_sys6"}' -blockdev '{"driver":"file","cache":

{"direct":true,"no-flush":false}

,"filename":"/home/rhel920-nfs/d5.qcow2","node-name":"drive_sys7"}' -blockdev '{"driver":"qcow2","node-name":"usb-disk2","file":"drive_sys7"}' -blockdev '{"driver":"file","cache":

{"direct":true,"no-flush":false}

,"filename":"/home/rhel920-nfs/d6.qcow2","node-name":"drive_sys8"}' -blockdev '{"driver":"qcow2","node-name":"usb-disk3","file":"drive_sys8"}' -blockdev '{"driver":"file","cache":

{"direct":true,"no-flush":false}

,"filename":"/home/rhel920-nfs/d7.qcow2","node-name":"drive_sys9"}' -blockdev '{"driver":"qcow2","node-name":"drive-usb-0","file":"drive_sys9"}' -blockdev '{"driver":"file","cache":

{"direct":true,"no-flush":false}

,"filename":"/home/rhel920-nfs/d8.qcow2","node-name":"drive_sys10"}' -blockdev '{"driver":"qcow2","node-name":"drive-usb-1","file":"drive_sys10"}' -blockdev '{"driver":"file","cache":

{"direct":true,"no-flush":false}

,"filename":"/home/rhel920-nfs/d9.qcow2","node-name":"drive_sys11"}' -blockdev '{"driver":"qcow2","node-name":"drive-usb-2","file":"drive_sys11"}' -blockdev '{"node-name":"file_ovmf_code","driver":"file","filename":"/usr/share/OVMF/OVMF_CODE.secboot.fd","auto-read-only":true,"discard":"unmap"}' -blockdev '{"node-name":"drive_ovmf_code","driver":"raw","read-only":true,"file":"file_ovmf_code"}' -blockdev '{"node-name":"file_ovmf_vars","driver":"file","filename":"/home/rhel920-nfs/rhel920-64-virtio-scsi-ovmf.qcow2_VARS.fd","auto-read-only":true,"discard":"unmap"}' -blockdev '{"node-name":"drive_ovmf_vars","driver":"raw","read-only":false,"file":"file_ovmf_vars"}' -object '

{"qom-type":"memory-backend-ram","id":"mem-1","prealloc":true,"size":2147483648,"host-nodes":[0],"policy":"bind"}

' -object '

{"qom-type":"memory-backend-ram","id":"mem-2","prealloc":true,"size":2147483648,"host-nodes":[0],"policy":"bind"}

' -object '{"qom-type":"rng-random","id":"objrng0","filename":"/dev/urandom"}' -numa node,memdev=mem-1 -numa node,memdev=mem-2 -netdev tap,id=hostnet1,vhost=on,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown -netdev tap,id=hostnet2,vhost=on,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown -m 4096,slots=256,maxmem=32G -overcommit mem-lock=off -smp 8,cores=1,threads=1,sockets=8 -vnc :10,audiodev=audio1 -rtc base=utc,clock=host -boot order=cdn,once=c,menu=on,strict=on -enable-kvm  -qmp tcp:0:3333,server,nowait -qmp tcp:0:9999,server=on,wait=off -qmp tcp:0:9888,server=on,wait=off -serial tcp:0:4444,server,nowait -monitor stdio -global ICH9-LPC.noreboot=off -watchdog-action reset

 

Dst : cmdline appending “- incoming defer”

 

2.Migrate vm from 10.2 to 9.2 - successfully

3.Migrate vm from 9.2 to 10.2 - failed 

Expected results

The migration can finish successfully

Actual results

On 10.2 - dst

1 -rtc base=utc,clock=host -boot order=cdn,once=c,menu=on,strict=on -enable-kvm  -qmp tcp:0:3333,server,nowait -qmp tcp:0:9999,server=on,wait=off -qmp tcp:0:9888,server=on,wait=off -serial tcp:0:4444,server,nowait -monitor stdio -global ICH9-LPC.noreboot=off -watchdog-action reset -incoming defer

char device redirected to /dev/pts/3 (label charserial0)

qemu-kvm: warning: Machine type 'pc-q35-rhel9.2.0' is deprecated: machines from the previous RHEL major release are subject to deletion in the next RHEL major release

QEMU 10.1.0 monitor - type 'help' for more information

(qemu) 

(qemu) migrate_incoming tcp:[::]:4000

(qemu) 

(qemu) 

(qemu) qemu-kvm: Failed to load virtio_pci/modern_queue_state:used

qemu-kvm: Failed to load virtio_pci/modern_state:vqs

qemu-kvm: Failed to load virtio/extra_state:extra_state

qemu-kvm: Failed to load virtio-vhost_vsock:virtio

qemu-kvm: error while loading state for instance 0x0 of device '0000:00:11.0:00.0/virtio-vhost_vsock'

On RHEL 9.2 - src

expected downtime: 300 ms

setup: 66 ms

transferred ram: 1293828 kbytes

throughput: 985.28 mbps

remaining ram: 1993280 kbytes

total ram: 4224072 kbytes

duplicate: 235387 pages

skipped: 0 pages

normal: 322310 pages

normal bytes: 1289240 kbytes

dirty sync count: 1

page size: 4 kbytes

multifd bytes: 0 kbytes

pages-per-second: 30293

downtime ram: 1293828 kbytes

(qemu) qemu-kvm: ../block.c:6768: int bdrv_inactivate_recurse(BlockDriverState *): Assertion `!(bs->open_flags & BDRV_O_INACTIVE)' failed.

Aborted (core dumped)