-
Bug
-
Resolution: Done-Errata
-
Normal
-
None
-
qemu-kvm-8.2.0-7.el9_4
-
None
-
Moderate
-
CustomerScenariosInitiative
-
rhel-sst-virtualization
-
ssg_virtualization
-
26
-
30
-
2
-
False
-
-
None
-
None
-
Approved Exception
-
Pass
-
RegressionOnly
-
-
All
-
None
What were you trying to do that didn't work?
I'm testing max vcpu on ampere-mtjade-altramax-01.khw.eng.rdu2.dc.redhat.com
[root@ampere-mtjade-altramax-01 ~]# lscpu
Architecture: aarch64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 256
On-line CPU(s) list: 0-255
Vendor ID: ARM
BIOS Vendor ID: Ampere(R)
Model name: Neoverse-N1
BIOS Model name: Ampere(R) Altra(R) Max Processor
When max vcpu is greater than or equal to 246
(qemu) qemu-kvm: virtio_bus_set_host_notifier: unable to init event
notifier: Too many open files (-24)
virtio-scsi: Failed to set host notifier (-24)
qemu-kvm: virtio_bus_start_ioeventfd: failed. Fallback to userspace (slower).
qemu-kvm: virtio_bus_set_host_notifier: unable to init event notifier:
Too many open files (-24)
virtio-scsi: Failed to set host notifier (-24)
qemu-kvm: virtio_bus_start_ioeventfd: failed. Fallback to userspace (slower).
Please provide the package NVR for which bug is seen:
hostname: ampere-mtjade-altramax-01.khw.eng.rdu2.dc.redhat.com
RHEL-9.4.0-20240218.15
edk2-aarch64-20231122-4.el9.noarch
qemu version: qemu-kvm-8.2.0-5.el9
guest kernel: 5.14.0-422.el9.aarch64
host kernel: 5.14.0-422.el9.aarch64
How reproducible:
100%
Steps to reproduce
Boot guest with 246 vcpu
/usr/libexec/qemu-kvm \
-name 'avocado-vt-vm1' \
-sandbox on \
-blockdev '{"node-name": "file_aavmf_code", "driver": "file", "filename": "/usr/share/edk2/aarch64/QEMU_EFI-silent-pflash.qcow2", "auto-read-only": true, "discard": "unmap"}' \
-blockdev '{"node-name": "drive_aavmf_code", "driver": "qcow2", "read-only": true, "file": "file_aavmf_code"}' \
-blockdev '{"node-name": "file_aavmf_vars", "driver": "file", "filename": "/home/kar/workspace/root/avocado/data/avocado-vt/avocado-vt-vm1_rhel940-aarch64-64k-virtio-scsi_qcow2_filesystem_VARS.qcow2", "auto-read-only": true, "discard": "unmap"}' \
-blockdev '{"node-name": "drive_aavmf_vars", "driver": "qcow2", "read-only": false, "file": "file_aavmf_vars"}' \
-machine virt,gic-version=host,pflash0=drive_aavmf_code,pflash1=drive_aavmf_vars,memory-backend=mem-machine_mem \
-device '{"id": "pcie-root-port-0", "driver": "pcie-root-port", "multifunction": true, "bus": "pcie.0", "addr": "0x1", "chassis": 1}' \
-device '{"id": "pcie-pci-bridge-0", "driver": "pcie-pci-bridge", "addr": "0x0", "bus": "pcie-root-port-0"}' \
-nodefaults \
-device '{"id": "pcie-root-port-1", "port": 1, "driver": "pcie-root-port", "addr": "0x1.0x1", "bus": "pcie.0", "chassis": 2}' \
-device '{"driver": "virtio-gpu-pci", "bus": "pcie-root-port-1", "addr": "0x0"}' \
-m 8192 \
-object '{"size": 8589934592, "id": "mem-machine_mem", "qom-type": "memory-backend-ram"}' \
-smp 246 \
-cpu 'host' \
-chardev socket,path=/var/tmp/monitor-qmpmonitor1-20240219-011759-O8gOdseU,wait=off,id=qmp_id_qmpmonitor1,server=on \
-mon chardev=qmp_id_qmpmonitor1,mode=control \
-serial unix:'/var/tmp/serial-serial0',server=on,wait=off \
-device '{"id": "pcie-root-port-2", "port": 2, "driver": "pcie-root-port", "addr": "0x1.0x2", "bus": "pcie.0", "chassis": 3}' \
-device '{"driver": "qemu-xhci", "id": "usb1", "bus": "pcie-root-port-2", "addr": "0x0"}' \
-device '{"driver": "usb-tablet", "id": "usb-tablet1", "bus": "usb1.0", "port": "1"}' \
-device '{"id": "pcie-root-port-3", "port": 3, "driver": "pcie-root-port", "addr": "0x1.0x3", "bus": "pcie.0", "chassis": 4}' \
-device '{"id": "virtio_scsi_pci0", "driver": "virtio-scsi-pci", "bus": "pcie-root-port-3", "addr": "0x0"}' \
-blockdev '{"node-name": "file_image1", "driver": "file", "auto-read-only": true, "discard": "unmap", "aio": "threads", "filename": "/home/kvm_autotest_root/images/rhel940-aarch64-64k-virtio-scsi.qcow2", "cache": {"direct": true, "no-flush": false}}' \
-blockdev '{"node-name": "drive_image1", "driver": "qcow2", "read-only": false, "cache":
, "file": "file_image1"}' \
-device '{"driver": "scsi-hd", "id": "image1", "drive": "drive_image1", "write-cache": "on"}' \
-device '{"id": "pcie-root-port-4", "port": 4, "driver": "pcie-root-port", "addr": "0x1.0x4", "bus": "pcie.0", "chassis": 5}' \
-device '{"driver": "virtio-net-pci", "mac": "9a:07:b6:1d:f6:ae", "rombar": 0, "id": "idRWghpz", "netdev": "idVWHq77", "bus": "pcie-root-port-4", "addr": "0x0"}' \
-netdev '{"id": "idVWHq77", "type": "tap", "vhost": true}' \
-vnc :20 \
-rtc base=utc,clock=host \
-enable-kvm \
-monitor stdio
Hit issue:
(qemu) qemu-kvm: virtio_bus_set_host_notifier: unable to init event
notifier: Too many open files (-24)
virtio-scsi: Failed to set host notifier (-24)
qemu-kvm: virtio_bus_start_ioeventfd: failed. Fallback to userspace (slower).
qemu-kvm: virtio_bus_set_host_notifier: unable to init event notifier:
Too many open files (-24)
virtio-scsi: Failed to set host notifier (-24)
qemu-kvm: virtio_bus_start_ioeventfd: failed. Fallback to userspace (slower).
After debugging, I found that this is due to the host's default
DefaultLimitNOFILE=1024:524288.
[root@ampere-mtjade-altramax-01 ~]# for pid in $(pidof qemu-kvm); do
prlimit -p $pid | grep NOFILE; ls -1 /proc/$pid/fd/ | wc -l; done
NOFILE max number of open files 1024 524288 files
1024
[root@ampere-mtjade-altramax-01 ~]# cat /etc/systemd/system.conf |grep
DefaultLimitNOFILE
#DefaultLimitNOFILE=1024:524288
And I saw a patch with a similar issue:
[PULL 03/17] qemu_init: increase NOFILE soft limit on POSIX
- is related to
-
RHEL-28813 qemu crash with kvm_irqchip_commit_routes: Assertion `ret == 0' failed if booting with many virtio disks and vcpus
- Closed
- links to
-
RHBA-2023:121202 qemu-kvm bug fix and enhancement update