-
Bug
-
Resolution: Won't Do
-
Normal
-
None
-
rhel-9.0.0
-
None
-
Moderate
-
rhel-sst-virtualization-networking
-
ssg_virtualization
-
5
-
False
-
-
None
-
None
-
None
-
None
-
Known Issue
-
-
Done
-
-
x86_64
-
None
Description of problem:
live post-copy migration of the vm with failover VF device fails.
Version-Release number of selected component (if applicable):
guest:
4.18.0-192.el8.x86_64
host:
4.18.0-192.el8.x86_64
qemu-kvm-4.2.0-16.module+el8.2.0+6092+4f2391c1.x86_64
How reproducible:
100%
Steps to Reproduce:
1.On source host,create NetXtreme BCM57810 VF and set the mac address of the VF
- ip link set enp131s0f0 vf 0 mac 22:2b:62:bb:a9:82
2.start a source guest with NetXtreme BCM57810 VF which enables failover
/usr/libexec/qemu-kvm -name rhel8-2 -M q35 -enable-kvm \
-monitor stdio \
-nodefaults \
-m 4G \
-boot menu=on \
-cpu Haswell-noTSX-IBRS \
-device pcie-root-port,id=root.1,chassis=1,addr=0x2.0,multifunction=on \
-device pcie-root-port,id=root.2,chassis=2,addr=0x2.1 \
-device pcie-root-port,id=root.3,chassis=3,addr=0x2.2 \
-device pcie-root-port,id=root.4,chassis=4,addr=0x2.3 \
-device pcie-root-port,id=root.5,chassis=5,addr=0x2.4 \
-device pcie-root-port,id=root.6,chassis=6,addr=0x2.5 \
-device pcie-root-port,id=root.7,chassis=7,addr=0x2.6 \
-device pcie-root-port,id=root.8,chassis=8,addr=0x2.7 \
-smp 2,sockets=1,cores=2,threads=2,maxcpus=4 \
-qmp tcp:0:5555,server,nowait \
-blockdev node-name=back_image,driver=file,cache.direct=on,cache.no-flush=off,filename=/nfsmount/migra_test/192.qcow2,aio=threads \
-blockdev node-name=drive-virtio-disk0,driver=qcow2,cache.direct=on,cache.no-flush=off,file=back_image \
-device virtio-blk-pci,drive=drive-virtio-disk0,id=disk0,bus=root.1 \
-device VGA,id=video1,bus=root.2 \
-vnc :0 \
-netdev tap,id=hostnet0,vhost=on \
-device virtio-net-pci,netdev=hostnet0,id=net0,mac=22:2b:62:bb:a9:82,bus=root.3,failover=on \
-device vfio-pci,host=0000:83:01.0,id=hostdev0,bus=root.4,failover_pair_id=net0 \
3.On the source host,check the network info in guest
- ifconfig
enp3s0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 10.73.33.214 netmask 255.255.254.0 broadcast 10.73.33.255
inet6 2620:52:0:4920:202b:62ff:febb:a982 prefixlen 64 scopeid 0x0<global>
inet6 fe80::202b:62ff:febb:a982 prefixlen 64 scopeid 0x20<link>
ether 22:2b:62:bb:a9:82 txqueuelen 1000 (Ethernet)
RX packets 5087 bytes 377754 (368.9 KiB)
RX errors 0 dropped 5 overruns 0 frame 0
TX packets 101 bytes 11887 (11.6 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
enp3s0nsby: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether 22:2b:62:bb:a9:82 txqueuelen 1000 (Ethernet)
RX packets 4950 bytes 359401 (350.9 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 2 bytes 180 (180.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
enp4s0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether 22:2b:62:bb:a9:82 txqueuelen 1000 (Ethernet)
RX packets 137 bytes 18353 (17.9 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 99 bytes 11707 (11.4 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
device memory 0xfc800000-fc807fff
lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10<host>
loop txqueuelen 1000 (Local Loopback)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
4.On target host,create 82599ES VF and set the mac address of the VF
- ip link set enp6s0f0 vf 0 mac 22:2b:62:bb:a9:82
5.start a target guest in listening mode in order to wait for migrating from source guest
...
-netdev tap,id=hostnet0,vhost=on \
-device virtio-net-pci,netdev=hostnet0,id=net0,mac=22:2b:62:bb:a9:82,bus=root.3,failover=on \
-device vfio-pci,host=0000:06:10.0,id=hostdev0,bus=root.4,failover_pair_id=net0 \
-incoming tcp:0:5800 \
6.On source host and target host, set postcopy mode on
(qemu) migrate_set_capability postcopy-ram on
7.Migrate guest from source host to target host.
(qemu) migrate -d tcp:10.73.73.61:5800
8.
Before the migration is completed, change into postcopy mode:
(qemu) migrate_start_postcopy
9.The migration is completed within several seconds after starting the post-copy
10.check the migration info on source host
(qemu) info migrate
globals:
store-global-state: on
only-migratable: off
send-configuration: on
send-section-footer: on
decompress-error-check: on
clear-bitmap-shift: 18
Migration status: completed
total time: 13638 milliseconds
downtime: 6 milliseconds
setup: 7059 milliseconds
transferred ram: 643240 kbytes
throughput: 801.15 mbps
remaining ram: 0 kbytes
total ram: 4211528 kbytes
duplicate: 894347 pages
skipped: 0 pages
normal: 158535 pages
normal bytes: 634140 kbytes
dirty sync count: 2
page size: 4 kbytes
multifd bytes: 0 kbytes
pages-per-second: 29470
postcopy request count: 446
11.check the migration info on the target host
(qemu)
qemu-kvm: VFIO_MAP_DMA: -14
qemu-kvm: VFIO_MAP_DMA: -14
qemu-kvm: VFIO_MAP_DMA: -14
qemu-kvm: VFIO_MAP_DMA: -14
qemu-kvm: VFIO_MAP_DMA: -14
qemu-kvm: warning: vfio 0000:06:10.0: failed to setup container for group 63: memory listener initialization failed: Region vga.vram: vfio_dma_map(0x55943303cee0, 0xa0000, 0x10000, 0x7f05f0a00000) = -14 (Bad address)
(qemu) info migrate
globals:
store-global-state: on
only-migratable: off
send-configuration: on
send-section-footer: on
decompress-error-check: on
clear-bitmap-shift: 18
Migration status: completed
total time: 0 milliseconds
12.On the target host,check the network info in guest
- ifconfig
enp3s0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 10.73.33.214 netmask 255.255.254.0 broadcast 10.73.33.255
inet6 2620:52:0:4920:202b:62ff:febb:a982 prefixlen 64 scopeid 0x0<global>
inet6 fe80::202b:62ff:febb:a982 prefixlen 64 scopeid 0x20<link>
ether 22:2b:62:bb:a9:82 txqueuelen 1000 (Ethernet)
RX packets 5087 bytes 377754 (368.9 KiB)
RX errors 0 dropped 5 overruns 0 frame 0
TX packets 101 bytes 11887 (11.6 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
enp3s0nsby: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether 22:2b:62:bb:a9:82 txqueuelen 1000 (Ethernet)
RX packets 4950 bytes 359401 (350.9 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 2 bytes 180 (180.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10<host>
loop txqueuelen 1000 (Local Loopback)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
Actual results:
live post-copy migration of the vm with failover VF device fails.
Expected results:
live post-copy migration of the vm with failover VF device is complete.
The failover VF works well on both the source host and target host.
- external trackers