-
Bug
-
Resolution: Unresolved
-
Minor
-
rhel-9.6
-
qemu-kvm-9.1.0-16.el9
-
No
-
Moderate
-
1
-
rhel-virt-core-live-migration
-
ssg_virtualization
-
4
-
6
-
300
-
False
-
False
-
-
None
-
LM-Mar-Apr-Sprint3
-
Pass
-
RegressionOnly
-
-
x86_64
-
None
What were you trying to do that didn't work?
Guest crashed on the target host when the migration was canceled
What is the impact of this issue to you?
Please provide the package NVR for which the bug is seen:
libvirt-10.9.0-1.el9.aarch64
qemu-kvm-9.1.0-5.el9.aarch64
How reproducible is this bug?:
10%
Steps to reproduce
- Do migration and cancel it during:
# virsh reboot vm2; virsh -c 'qemu:///system' migrate --live --verbose --domain vm2 --desturi qemu+ssh://test.com/system --persistent Domain 'vm2' is being rebootedMigration: [61.74 %]^Cerror: operation aborted: job 'migration out' canceled by client
2.vm on source guest was still running but can see guest crash on the target host sometimes:
# coredumpctl list TIME PID UID GID SIG COREFILE EXE SIZE Tue 2024-12-03 01:12:15 EST 48647 107 107 SIGABRT present /usr/libexec/qemu-kvm 1.0M
3.The backtrace is as following:
Core was generated by `/usr/libexec/qemu-kvm -name guest=vm2,debug-threads=on -S -object {"qom-type":"'. Program terminated with signal SIGABRT, Aborted. #0 __pthread_kill_implementation (threadid=281473211392032, signo=signo@entry=6, no_tid=no_tid@entry=0) at pthread_kill.c:44 44 return INTERNAL_SYSCALL_ERROR_P (ret) ? INTERNAL_SYSCALL_ERRNO (ret) : 0; [Current thread is 1 (Thread 0xffff96c76020 (LWP 47801))] (gdb) t a a btThread 7 (Thread 0xffff8cdfe900 (LWP 47823)): #0 __futex_abstimed_wait_common64 (private=0, cancel=true, abstime=0x0, op=393, expected=0, futex_word=0xaaaae1180508) at futex-internal.c:57 #1 __futex_abstimed_wait_common (cancel=true, private=0, abstime=0x0, clockid=0, expected=0, futex_word=0xaaaae1180508) at futex-internal.c:87 #2 __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0xaaaae1180508, expected=expected@entry=0, clockid=clockid@entry=0, abstime=abstime@entry=0x0, private=private@entry=0) at futex-internal.c:139 #3 0x0000ffff95b11c50 in __pthread_cond_wait_common (abstime=0x0, clockid=0, mutex=0xaaaae1180518, cond=0xaaaae11804e0) at pthread_cond_wait.c:504 #4 ___pthread_cond_wait (cond=0xaaaae11804e0, mutex=0xaaaae1180518) at pthread_cond_wait.c:619 #5 0x0000aaaacb662e20 in qemu_cond_wait_impl (cond=0xaaaae1180508, mutex=0xaaaae1180518, file=0xaaaacb6a5b6c "../ui/vnc-jobs.c", line=248) at ../util/qemu-thread-posix.c:225 #6 0x0000aaaacb040a98 in vnc_worker_thread_loop (queue=0xaaaae11804e0) at ../ui/vnc-jobs.c:248 #7 vnc_worker_thread (arg=arg@entry=0xaaaae11804e0) at ../ui/vnc-jobs.c:362 #8 0x0000aaaacb6639e8 in qemu_thread_start (args=0xaaaae1180580) at ../util/qemu-thread-posix.c:541 #9 0x0000ffff95b12918 in start_thread (arg=0x80e140) at pthread_create.c:443 #10 0x0000ffff95b7cf1c in thread_start () at ../sysdeps/unix/sysv/linux/aarch64/clone.S:79Thread 6 (Thread 0xffff949fe900 (LWP 47817)): #0 syscall () at ../sysdeps/unix/sysv/linux/aarch64/syscall.S:38 #1 0x0000aaaacb663780 in qemu_futex_wait (f=<optimized out>, val=<optimized out>) at /usr/src/debug/qemu-kvm-9.1.0-5.el9.aarch64/include/qemu/futex.h:29 #2 qemu_event_wait (ev=ev@entry=0xaaaacc4881cc <rcu_call_ready_event>) at ../util/qemu-thread-posix.c:464 #3 0x0000aaaacb66ff2c in call_rcu_thread (opaque=<optimized out>) at ../util/rcu.c:278 #4 0x0000aaaacb6639e8 in qemu_thread_start (args=0xaaaae0a092e0) at ../util/qemu-thread-posix.c:541 #5 0x0000ffff95b12918 in start_thread (arg=0x80e140) at pthread_create.c:443 #6 0x0000ffff95b7cf1c in thread_start () at ../sysdeps/unix/sysv/linux/aarch64/clone.S:79Thread 5 (Thread 0xffff8fefd900 (LWP 47818)): #0 __futex_abstimed_wait_common64 (private=0, cancel=true, abstime=0xffff8fefcf38, op=393, expected=0, futex_word=0xaaaae0a57658) at futex-internal.c:57 #1 __futex_abstimed_wait_common (cancel=true, private=0, abstime=0xffff8fefcf38, clockid=0, expected=0, futex_word=0xaaaae0a57658) at futex-internal.c:87 #2 __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0xaaaae0a57658, expected=expected@entry=0, clockid=clockid@entry=0, abstime=abstime@entry=0xffff8fefcf38, private=private@entry=0) at futex-internal.c:139 #3 0x0000ffff95b11f60 in __pthread_cond_wait_common (abstime=0xffff8fefcf38, clockid=0, mutex=0xaaaae0a575c0, cond=0xaaaae0a57630) at pthread_con--Type <RET> for more, q to quit, c to continue without paging-- d_wait.c:504 #4 ___pthread_cond_timedwait64 (cond=0xaaaae0a57630, mutex=0xaaaae0a575c0, abstime=0xffff8fefcf38) at pthread_cond_wait.c:644 #5 0x0000aaaacb663038 in qemu_cond_timedwait_ts (cond=0xaaaae0a57658, cond@entry=0xaaaae0a57630, mutex=mutex@entry=0xaaaae0a575c0, ts=0x0, ts@entry=0xffff8fefcf38, file=file@entry=0xaaaacb7566b2 "../util/thread-pool.c", line=line@entry=91) at ../util/qemu-thread-posix.c:239 #6 0x0000aaaacb662f90 in qemu_cond_timedwait_impl (cond=0xaaaae0a57630, mutex=0xaaaae0a575c0, ms=10000, file=0xaaaacb7566b2 "../util/thread-pool.c", line=91) at ../util/qemu-thread-posix.c:253 #7 0x0000aaaacb67ced8 in worker_thread (opaque=opaque@entry=0xaaaae0a575b0) at ../util/thread-pool.c:91 #8 0x0000aaaacb6639e8 in qemu_thread_start (args=0xaaaae0b4d6b0) at ../util/qemu-thread-posix.c:541 #9 0x0000ffff95b12918 in start_thread (arg=0x80e140) at pthread_create.c:443 #10 0x0000ffff95b7cf1c in thread_start () at ../sysdeps/unix/sysv/linux/aarch64/clone.S:79Thread 4 (Thread 0xffff8e408900 (LWP 47822)): #0 __futex_abstimed_wait_common64 (private=0, cancel=true, abstime=0x0, op=393, expected=0, futex_word=0xaaaae0c147cc) at futex-internal.c:57 #1 __futex_abstimed_wait_common (cancel=true, private=0, abstime=0x0, clockid=0, expected=0, futex_word=0xaaaae0c147cc) at futex-internal.c:87 #2 __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0xaaaae0c147cc, expected=expected@entry=0, clockid=clockid@entry=0, abstime=abstime@entry=0x0, private=private@entry=0) at futex-internal.c:139 #3 0x0000ffff95b11c50 in __pthread_cond_wait_common (abstime=0x0, clockid=0, mutex=0xaaaacc45bf40 <bql>, cond=0xaaaae0c147a0) at pthread_cond_wait.c:504 #4 ___pthread_cond_wait (cond=0xaaaae0c147a0, mutex=0xaaaacc45bf40 <bql>) at pthread_cond_wait.c:619 #5 0x0000aaaacb662e20 in qemu_cond_wait_impl (cond=0xaaaae0c147cc, mutex=0xaaaacc45bf40 <bql>, file=0xaaaacb6c2155 "../system/cpus.c", line=462) at ../util/qemu-thread-posix.c:225 #6 0x0000aaaacb100570 in qemu_wait_io_event (cpu=cpu@entry=0xaaaae0c1f3c0) at ../system/cpus.c:462 #7 0x0000aaaacb4b51f0 in kvm_vcpu_thread_fn (arg=arg@entry=0xaaaae0c1f3c0) at ../accel/kvm/kvm-accel-ops.c:55 #8 0x0000aaaacb6639e8 in qemu_thread_start (args=0xaaaae0c378b0) at ../util/qemu-thread-posix.c:541 #9 0x0000ffff95b12918 in start_thread (arg=0x80e140) at pthread_create.c:443 #10 0x0000ffff95b7cf1c in thread_start () at ../sysdeps/unix/sysv/linux/aarch64/clone.S:79Thread 3 (Thread 0xffff8ec18900 (LWP 47821)): #0 __futex_abstimed_wait_common64 (private=0, cancel=true, abstime=0x0, op=393, expected=0, futex_word=0xaaaae0a41ddc) at futex-internal.c:57 #1 __futex_abstimed_wait_common (cancel=true, private=0, abstime=0x0, clockid=0, expected=0, futex_word=0xaaaae0a41ddc) at futex-internal.c:87 #2 __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0xaaaae0a41ddc, expected=expected@entry=0, clockid=clockid@entry=0, abstime=abstime@entry=0x0, private=private@entry=0) at futex-internal.c:139 #3 0x0000ffff95b11c50 in __pthread_cond_wait_common (abstime=0x0, clockid=0, mutex=0xaaaacc45bf40 <bql>, cond=0xaaaae0a41db0) at pthread_cond_wait.c:504 #4 ___pthread_cond_wait (cond=0xaaaae0a41db0, mutex=0xaaaacc45bf40 <bql>) at pthread_cond_wait.c:619 --Type <RET> for more, q to quit, c to continue without paging-- #5 0x0000aaaacb662e20 in qemu_cond_wait_impl (cond=0xaaaae0a41ddc, mutex=0xaaaacc45bf40 <bql>, file=0xaaaacb6c2155 "../system/cpus.c", line=462) at ../util/qemu-thread-posix.c:225 #6 0x0000aaaacb100570 in qemu_wait_io_event (cpu=cpu@entry=0xaaaae0bcac20) at ../system/cpus.c:462 #7 0x0000aaaacb4b51f0 in kvm_vcpu_thread_fn (arg=arg@entry=0xaaaae0bcac20) at ../accel/kvm/kvm-accel-ops.c:55 #8 0x0000aaaacb6639e8 in qemu_thread_start (args=0xaaaae0be42c0) at ../util/qemu-thread-posix.c:541 #9 0x0000ffff95b12918 in start_thread (arg=0x80e140) at pthread_create.c:443 #10 0x0000ffff95b7cf1c in thread_start () at ../sysdeps/unix/sysv/linux/aarch64/clone.S:79Thread 2 (Thread 0xffff8f529900 (LWP 47820)): #0 0x0000ffff95b72ca0 in __GI___poll (fds=0xfffe800035d0, nfds=3, timeout=<optimized out>) at ../sysdeps/unix/sysv/linux/poll.c:41 #1 0x0000ffff95f65f20 in g_main_context_poll (priority=<optimized out>, n_fds=3, fds=0xfffe800035d0, timeout=<optimized out>, context=0xaaaae0b7d390) at ../glib/gmain.c:4458 #2 g_main_context_iterate.constprop.0 (context=0xaaaae0b7d390, block=block@entry=1, dispatch=dispatch@entry=1, self=<optimized out>) at ../glib/gmain.c:4150 #3 0x0000ffff95f0f71c in g_main_loop_run (loop=0xaaaae0a61660) at ../glib/gmain.c:4353 #4 0x0000aaaacb52a298 in iothread_run (opaque=opaque@entry=0xaaaae0a3ca00) at ../iothread.c:70 #5 0x0000aaaacb6639e8 in qemu_thread_start (args=0xaaaae0b7d450) at ../util/qemu-thread-posix.c:541 #6 0x0000ffff95b12918 in start_thread (arg=0x80e140) at pthread_create.c:443 #7 0x0000ffff95b7cf1c in thread_start () at ../sysdeps/unix/sysv/linux/aarch64/clone.S:79 Thread 1 (Thread 0xffff96c76020 (LWP 47801)): #0 __pthread_kill_implementation (threadid=281473211392032, signo=signo@entry=6, no_tid=no_tid@entry=0) at pthread_kill.c:44 #1 0x0000ffff95b145b4 in __pthread_kill_internal (signo=6, threadid=<optimized out>) at pthread_kill.c:78 #2 0x0000ffff95acc6bc in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26 #3 0x0000ffff95ab8fb4 in __GI_abort () at abort.c:79 #4 0x0000ffff95ac6010 in __assert_fail_base (fmt=0xffff95be41d8 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", assertion=assertion@entry=0xaaaacb75350e "mutex->initialized", file=file@entry=0xaaaacb753521 "../util/qemu-thread-posix.c", line=line@entry=92, function=function@entry=0xaaaacb753575 "void qemu_mutex_lock_impl(QemuMutex *, const char *, const int)") at assert.c:92 #5 0x0000ffff95ac6080 in __assert_fail (assertion=0xaaaacb75350e "mutex->initialized", file=0xaaaacb753521 "../util/qemu-thread-posix.c", line=line@entry=92, function=0xaaaacb753575 "void qemu_mutex_lock_impl(QemuMutex *, const char *, const int)") at assert.c:101 #6 0x0000aaaacb6627cc in qemu_mutex_lock_impl (mutex=<optimized out>, file=<optimized out>, line=<optimized out>) at ../util/qemu-thread-posix.c:92 #7 0x0000aaaacb128a4c in qemu_lockable_mutex_lock (x=<optimized out>) at /usr/src/debug/qemu-kvm-9.1.0-5.el9.aarch64/include/qemu/lockable.h:56 #8 qemu_lockable_lock (x=<optimized out>) at /usr/src/debug/qemu-kvm-9.1.0-5.el9.aarch64/include/qemu/lockable.h:110 #9 qemu_lockable_auto_lock (x=<optimized out>) at /usr/src/debug/qemu-kvm-9.1.0-5.el9.aarch64/include/qemu/lockable.h:120 --Type <RET> for more, q to quit, c to continue without paging-- #10 migrate_set_error (s=s@entry=0xaaaae0a3acf0, error=0xaaaae14bd250) at ../migration/migration.c:1461 #11 0x0000aaaacb12975c in process_incoming_migration_co (opaque=<optimized out>) at ../migration/migration.c:836 #12 0x0000aaaacb67c2cc in coroutine_trampoline (i0=<optimized out>, i1=<optimized out>) at ../util/coroutine-ucontext.c:175 #13 0x0000ffff95adb840 in ?? () at ../sysdeps/unix/sysv/linux/aarch64/setcontext.S:123 from /lib64/libc.so.6 Backtrace stopped: not enough registers or memory available to unwind further
4.Do step 2 after rebooting guest os can reproduce the issue easier:
# virsh reboot vm2; virsh -c 'qemu:///system' migrate --live --verbose --domain vm2 --desturi qemu+ssh://test.com/system --persistent
Expected results
Guest should not crash on the target host when the migration was canceled.
Actual results
Guest crashed on the target host when the migration was canceled.
- is cloned by
-
RHEL-69776 [rhel10]Guest crashed on the target host when the migration was canceled
-
- Release Pending
-
- links to
-
RHBA-2025:147368 qemu-kvm bug fix and enhancement update