Uploaded image for project: 'RHEL'
  1. RHEL
  2. RHEL-12201

Qemu core dumped after stop NFS server while writing data

    • Yes
    • Critical
    • Regression, CustomerScenariosInitiative
    • rhel-sst-virtualization-storage
    • ssg_virtualization
    • None
    • False
    • Hide

      None

      Show
      None
    • None
    • Red Hat Enterprise Linux
    • None
    • None

      What were you trying to do that didn't work?

      Stop the NFS server during writing data on it.

      Please provide the package NVR for which bug is seen:

      qemu-kvm-8.1.0-2.el9

      kernel-5.14.0-368.kpq1.el9.x86_64

      How reproducible:

      100%

      Steps to reproduce

       

      1. Prepare a data image over NFS share directory
      # systemctl start nfs-server.service
      
      # exportfs 10.73.72.90:/root/avocado/job-results/job-2023-10-09T04.33-942706e/test-results/1-Host_RHEL.m9.u4.ovmf.qcow2.virtio_scsi.up.virtio_net.Guest.RHEL.9.4.0.x86_64.io-github-autotest-qemu.nfs_corrupt.with_raw_format.q35/tmp_dirsjy8cjuf/nfs_dir -o rw,no_root_squash
      # mount 10.73.72.90:/root/avocado/job-results/job-2023-10-09T04.33-942706e/test-results/1-Host_RHEL.m9.u4.ovmf.qcow2.virtio_scsi.up.virtio_net.Guest.RHEL.9.4.0.x86_64.io-github-autotest-qemu.nfs_corrupt.with_raw_format.q35/tmp_dirsjy8cjuf/nfs_dir /root/avocado/job-results/job-2023-10-09T04.33-942706e/test-results/1-Host_RHEL.m9.u4.ovmf.qcow2.virtio_scsi.up.virtio_net.Guest.RHEL.9.4.0.x86_64.io-github-autotest-qemu.nfs_corrupt.with_raw_format.q35/tmp_dirsjy8cjuf/mnt_dir -o rw,soft,timeo=30,retrans=1,vers=3
      # qemu-img create -f raw /root/avocado/job-results/job-2023-10-09T04.33-942706e/test-results/1-Host_RHEL.m9.u4.ovmf.qcow2.virtio_scsi.up.virtio_net.Guest.RHEL.9.4.0.x86_64.io-github-autotest-qemu.nfs_corrupt.with_raw_format.q35/tmp_dirsjy8cjuf/mnt_dir/nfs_corrupt.raw 10G
       
      2. Start a guest with the data image above and writing data in it
      #/usr/libexec/qemu-kvm \
      -S  \
      -name 'avocado-vt-vm1'  \
      -sandbox on \
      -blockdev '{"node-name": "file_ovmf_code", "driver": "file", "filename": "/usr/share/OVMF/OVMF_CODE.secboot.fd", "auto-read-only": true, "discard": "unmap"}' \
      -blockdev '{"node-name": "drive_ovmf_code", "driver": "raw", "read-only": true, "file": "file_ovmf_code"}' \
      -blockdev '{"node-name": "file_ovmf_vars", "driver": "file", "filename": "/root/avocado/data/avocado-vt/avocado-vt-vm1_rhel940-64-virtio-scsi-ovmf_qcow2_filesystem_VARS.raw", "auto-read-only": true, "discard": "unmap"}' \
      -blockdev '{"node-name": "drive_ovmf_vars", "driver": "raw", "read-only": false, "file": "file_ovmf_vars"}' \
      -machine q35,pflash0=drive_ovmf_code,pflash1=drive_ovmf_vars,memory-backend=mem-machine_mem \
      -device '{"id": "pcie-root-port-0", "driver": "pcie-root-port", "multifunction": true, "bus": "pcie.0", "addr": "0x1", "chassis": 1}' \
      -device '{"id": "pcie-pci-bridge-0", "driver": "pcie-pci-bridge", "addr": "0x0", "bus": "pcie-root-port-0"}'  \
      -nodefaults \
      -device '{"driver": "VGA", "bus": "pcie.0", "addr": "0x2"}' \
      -m 30720 \
      -object '{"size": 32212254720, "id": "mem-machine_mem", "qom-type": "memory-backend-ram"}'  \
      -smp 20,maxcpus=20,cores=10,threads=1,dies=1,sockets=2  \
      -cpu 'Broadwell',+kvm_pv_unhalt \
      -chardev socket,wait=off,id=qmp_id_qmpmonitor1,server=on,path=/var/tmp/avocado_ydrkn6vm/monitor-qmpmonitor1-20231009-043346-rXR9AWjd  \
      -mon chardev=qmp_id_qmpmonitor1,mode=control \
      -chardev socket,wait=off,id=qmp_id_catch_monitor,server=on,path=/var/tmp/avocado_ydrkn6vm/monitor-catch_monitor-20231009-043346-rXR9AWjd  \
      -mon chardev=qmp_id_catch_monitor,mode=control \
      -device '{"ioport": 1285, "driver": "pvpanic", "id": "id9DnrM3"}' \
      -chardev socket,wait=off,id=chardev_serial0,server=on,path=/var/tmp/avocado_ydrkn6vm/serial-serial0-20231009-043346-rXR9AWjd \
      -device '{"id": "serial0", "driver": "isa-serial", "chardev": "chardev_serial0"}'  \
      -chardev socket,id=seabioslog_id_20231009-043346-rXR9AWjd,path=/var/tmp/avocado_ydrkn6vm/seabios-20231009-043346-rXR9AWjd,server=on,wait=off \
      -device isa-debugcon,chardev=seabioslog_id_20231009-043346-rXR9AWjd,iobase=0x402 \
      -device '{"id": "pcie-root-port-1", "port": 1, "driver": "pcie-root-port", "addr": "0x1.0x1", "bus": "pcie.0", "chassis": 2}' \
      -device '{"driver": "qemu-xhci", "id": "usb1", "bus": "pcie-root-port-1", "addr": "0x0"}' \
      -device '{"driver": "usb-tablet", "id": "usb-tablet1", "bus": "usb1.0", "port": "1"}' \
      -device '{"id": "pcie-root-port-2", "port": 2, "driver": "pcie-root-port", "addr": "0x1.0x2", "bus": "pcie.0", "chassis": 3}' \
      -device '{"id": "virtio_scsi_pci0", "driver": "virtio-scsi-pci", "bus": "pcie-root-port-2", "addr": "0x0"}' \
      -blockdev '{"node-name": "file_image1", "driver": "file", "auto-read-only": true, "discard": "unmap", "aio": "threads", "filename": "/home/kvm_autotest_root/images/rhel940-64-virtio-scsi-ovmf.qcow2", "cache": {"direct": true, "no-flush": false}}' \
      -blockdev '{"node-name": "drive_image1", "driver": "qcow2", "read-only": false, "cache":
      {"direct": true, "no-flush": false}
      , "file": "file_image1"}' \
      -device '{"driver": "scsi-hd", "id": "image1", "drive": "drive_image1", "write-cache": "on"}' \
      -blockdev '{"node-name": "file_stg", "driver": "file", "auto-read-only": true, "discard": "unmap", "aio": "threads", "filename": "/root/avocado/job-results/job-2023-10-09T04.33-942706e/test-results/1-Host_RHEL.m9.u4.ovmf.qcow2.virtio_scsi.up.virtio_net.Guest.RHEL.9.4.0.x86_64.io-github-autotest-qemu.nfs_corrupt.with_raw_format.q35/tmp_dirsjy8cjuf/mnt_dir/nfs_corrupt.raw", "cache": {"direct": true, "no-flush": false}}' \
      -blockdev '{"node-name": "drive_stg", "driver": "raw", "read-only": false, "cache":
      {"direct": true, "no-flush": false}
      , "file": "file_stg"}' \
      -device '{"driver": "scsi-hd", "id": "stg", "drive": "drive_stg", "write-cache": "on", "werror": "stop", "serial": "TARGET_DISK"}' \
      -device '{"id": "pcie-root-port-3", "port": 3, "driver": "pcie-root-port", "addr": "0x1.0x3", "bus": "pcie.0", "chassis": 4}' \
      -device '{"driver": "virtio-net-pci", "mac": "9a:da:19:be:62:e9", "id": "idQdARih", "netdev": "id7Av1OV", "bus": "pcie-root-port-3", "addr": "0x0"}'  \
      -netdev tap,id=id7Av1OV,vhost=on  \
      -vnc :0  \
      -rtc base=utc,clock=host,driftfix=slew  \
      -boot menu=off,order=cdn,once=c,strict=off \
      -enable-kvm \
      -monitor stdio \
      -device '{"id": "pcie_extra_root_port_0", "driver": "pcie-root-port", "multifunction": true, "bus": "pcie.0", "addr": "0x3", "chassis": 5}'
      (guest) # dd if=/dev/zero of=/dev/sdb oflag=direct
       
      3. Stop NFS via iptables during dd in the guest
      #iptables -t filter -A OUTPUT -d 10.73.72.90 -m state --state NEW,RELATED,ESTABLISHED -p tcp --dport 2049 -j REJECT'
      

       

      Expected results

      No crash, the writing flow just hang.

      Actual results

      Qemu crashed after step 3.

      (qemu) qemuraw.sh: line 46:  3753 Floating point exception(core dumped) /usr/libexec/qemu-kvm -S -name 'avocado-vt-vm1' -sandbox on *******

       

      And the trace log is:

      Stack trace of thread 3098:
                      #0  0x000055c983831a79 get_zones_wp (qemu-kvm + 0x86ba79)
                      #1  0x000055c983831f97 raw_co_prw (qemu-kvm + 0x86bf97)
                      #2  0x000055c9837d7d84 bdrv_driver_pwritev (qemu-kvm + 0x811d84)
                      #3  0x000055c9837d26ed bdrv_aligned_pwritev (qemu-kvm + 0x80c6ed)
                      #4  0x000055c9837d1d43 bdrv_co_pwritev_part (qemu-kvm + 0x80bd43)
                      #5  0x000055c983813f44 raw_co_pwritev (qemu-kvm + 0x84df44)
                      #6  0x000055c9837d7d84 bdrv_driver_pwritev (qemu-kvm + 0x811d84)
                      #7  0x000055c9837d26ed bdrv_aligned_pwritev (qemu-kvm + 0x80c6ed)
                      #8  0x000055c9837d1d43 bdrv_co_pwritev_part (qemu-kvm + 0x80bd43)
                      #9  0x000055c9837bcd16 blk_co_do_pwritev_part.llvm.8165632186031058405 (qemu-kvm + 0x7f6d16)
                      #10 0x000055c9837bd642 blk_aio_write_entry.llvm.8165632186031058405 (qemu-kvm + 0x7f7642)
                      #11 0x000055c9839ad476 coroutine_trampoline.llvm.6566130761695863925 (qemu-kvm + 0x9e7476)
                      #12 0x00007f94ca02a360 n/a (libc.so.6 + 0x2a360)
                      ELF object binary architecture: AMD x86-64

              hreitz@redhat.com Hanna Czenczek
              timao@redhat.com Tingting Mao
              virt-maint virt-maint
              Tingting Mao Tingting Mao
              Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

                Created:
                Updated:
                Resolved: