Uploaded image for project: 'RHEL'
  1. RHEL
  2. RHEL-58314

Qemu crash when migrate with virtiofs device

    • Yes
    • Important
    • CustomerScenariosInitiative
    • rhel-sst-virtualization
    • ssg_virtualization
    • None
    • QE ack
    • False
    • Hide

      None

      Show
      None
    • None
    • Red Hat Enterprise Linux
    • None
    • None
    • None
    • s390x
    • Unspecified
    • None

      What were you trying to do that didn't work?
      Qemu crash when migrate with virtiofs device

      Please provide the package NVR for which bug is seen:
      hosts: kernel-6.11.0-0.rc5.22.el10.s390x && qemu-kvm-9.0.0-8.el10.s390x

      How reproducible:
      100%

      Steps to reproduce
      1. setup virtiofs on the source and destination host:
      [root@host ~]# mkdir -p /tmp/virtiofs_test
      [root@host ~]# /usr/libexec/virtiofsd --socket-path=/tmp/vhostqemu -o source=/tmp/virtiofs_test -o cache=auto > /dev/null 2>&1&

      2. Boot VM with virtiofs device:
      /usr/libexec/qemu-kvm \
      -name "mouse-vm" \
      -sandbox on \
      -machine s390-ccw-virtio \
      -cpu host \
      -nodefaults \
      -chardev socket,id=qmp_id_qmpmonitor1,path=/var/tmp/monitor-qmpmonitor1,server=on,wait=off \
      -chardev socket,id=qmp_id_catch_monitor,path=/var/tmp/monitor-catch_monitor,server=on,wait=off \
      -chardev socket,id=char0,path=/tmp/vhostqemu \
      -mon chardev=qmp_id_qmpmonitor1,mode=control \
      -mon chardev=qmp_id_catch_monitor,mode=control \
      -device '

      {"driver":"virtio-scsi-ccw","id":"virtio_scsi_ccw0","devno":"fe.0.0000"}

      ' \
      -device '

      {"driver":"scsi-hd","id":"image1","device_id":"drive-image1","drive":"drive_image1","bus":"virtio_scsi_ccw0.0","bootindex":0,"write-cache":"on"}

      ' \
      -device '

      {"driver":"virtio-net-ccw","mac":"9a:fc:6e:3c:a6:f0","id":"net0","netdev":"tap0","devno":"fe.0.0001"}

      ' \
      -device '

      {"driver":"virtio-mouse-ccw","id":"input_mouse1","devno":"fe.0.0002"}

      ' \
      -device '

      {"driver":"virtio-keyboard-ccw","id":"input_keyboard1","devno":"fe.0.0003"}

      ' \
      -device '

      {"driver":"virtio-balloon-ccw","id":"balloon0","devno":"fe.0.0004"}

      ' \
      -device '

      {"driver":"vhost-user-fs-ccw","queue-size":1024,"chardev":"char0","tag":"myfs"}

      ' \
      -blockdev '{"driver":"file","auto-read-only":true,"discard":"unmap","aio":"threads","cache":

      {"direct":true,"no-flush":false}

      ,"filename":"/mnt/xiaohli/rhel8100-s390x-virtio-scsi.qcow2","node-name":"drive_sys1"}' \
      -blockdev '{"driver":"qcow2","node-name":"drive_image1","read-only":false,"cache":

      {"direct":true,"no-flush":false}

      ,"file":"drive_sys1"}' \
      -netdev tap,id=tap0,vhost=on \
      -m 4096 \
      -smp 1,maxcpus=1,cores=1,threads=1,sockets=1 \
      -nographic \
      -rtc base=utc,clock=host,driftfix=slew \
      -boot strict=on \
      -enable-kvm \
      -qmp tcp:0:3333,server=on,wait=off \
      -qmp tcp:0:9999,server=on,wait=off \
      -qmp tcp:0:9888,server=on,wait=off \
      -serial tcp:0:4444,server=on,wait=off \
      -monitor stdio \
      -object '

      {"qom-type":"memory-backend-file","id":"mem","size":4294967296,"mem-path":"/dev/shm","share":true}

      ' \

      3. Migrate VM from the source to the destination host

      Expected results
      Migration succeeds with virtiofs device.
      Because we support virtiofs migration from RHEL 9.5:
      https://issues.redhat.com/browse/RHEL-7383

      Actual results
      Qemu crash during migration in step 3:
      (1) on the source hmp:
      qemu-kvm: Error saving back-end state of virtio-user-fs device /machine/peripheral-anon/device[0]/virtio-backend (tag: "myfs"): Failed to initiate state transfer: Failed to send SET_DEVICE_STATE_FD message: Invalid argument
      1.sh: line 34: 175272 Segmentation fault (core dumped) ...

      (2) on the destination hmp:
      (qemu) migrate_incoming tcp:[::]:1234
      (qemu) qemu-kvm: Failed to load virtqueue_state:vring.avail
      qemu-kvm: Failed to load virtio/virtqueues:vq
      qemu-kvm: Failed to load vhost-user-fs:virtio
      qemu-kvm: error while loading state for instance 0x0 of device '/fe.0.0005/vhost-user-fs'
      qemu-kvm: load of migration failed: Input/output error

      Additional info:
      1. didn't reproduce this issue on x86 with the latest rhel 10-beta
      2. this is a regression issue because qemu shows don't support virtiofd migration on qemu-kvm-9.0.0-2.el10.s390x when query migration status before migration:

      {"execute": "query-migrate", "id": "f6uQSFQd"}

      {"return":

      {"blocked-reasons": ["Migration disabled: vhost-user backend lacks VHOST_USER_PROTOCOL_F_LOG_SHMFD feature."]}

      , "id": "f6uQSFQd"}

              hreitz@redhat.com Hanna Czenczek
              rhn-support-xiaohli Xiaohui Li
              virt-maint virt-maint
              Xiaohui Li Xiaohui Li
              Votes:
              0 Vote for this issue
              Watchers:
              14 Start watching this issue

                Created:
                Updated:
                Resolved: