Description of problem:
Warm Migrating VMs from RHV fails due to following error Unable to connect to imageio data source: Sending request failed Following error is reported in the forklift-controller Fault reason is \"Operation Failed\". Fault detail is \"[Cannot remove Snapshot. The VM is performing an operation on a Snapshot. Please wait for the operation to finish, and try again.]\". HTTP response code is \"409\". HTTP response message is \"409 Conflict\{code} Version-Release number of selected component (if applicable): MTV 2.7.9
How reproducible:
2 customers reported the same error
Steps to Reproduce:
1. Start Warm VM migration from RHV 2. Migration fails on CopyDisks with Unable to connect to imageio data
Actual results:
RHV VM migration fails after updating MTV from 2.5 to 2.7.9
Expected results:
Additional info:
~~~ Error: phase: CopyDisks reasons: - 'Unable to connect to imageio data source: Sending request failed: Get "xxxxxx:54322/images/c8a15907-73f8-4f45-a63e-44f6311185e5": Unknown Code' message: 'Unable to connect to imageio data source: Error sending transfer image request: Fault reason is "Operation Failed". Fault detail is "[Cannot transfer Virtual Disk. The disk ocp_test_vm_disk is plugged to non-Down VMs: ocp-test-vm]". HTTP response code is "409". HTTP response message is "409 Conflict".' reason: Error ~~~ Looking at the RHV engine.log it fails here ~~~ 2025-01-28 16:18:03,822+01 WARN [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotCommand] (default task-544730) [068d61ec_finalize] Validation of action 'RemoveSnapshot' failed for user admin@internal-authz. Reasons: VAR__TYPE__SNAPSHOT,VAR__ACTION__REMOVE,ACTION_TYPE_FAILED_VM_IS_DURING_SNAPSHOT 2025-01-28 16:18:03,822+01 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotCommand] (default task-544730) [068d61ec_finalize] Lock freed to object 'EngineLock:{exclusiveLocks='[56320177-6198-43bb-b1ef-8825ec2b1edd=VM]', sharedLocks=''}' 2025-01-28 16:18:03,826+01 ERROR [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (default task-544730) [] Operation Failed: [Cannot remove Snapshot. The VM is performing an operation on a Snapshot. Please wait for the operation to finish, and try again.] <<<< Image transfer fails because VM is not down: 2025-01-28 16:18:28,175+01 INFO [org.ovirt.engine.core.bll.storage.disk.image.TransferDiskImageCommand] (default task-544730) [97d84dea-5177-4dfe-b816-722a9d956c95] Lock Acquired to object 'EngineLock:{exclusiveLocks='[f00fe8ba-56e7-4a95-acb9-3a96c6f2cc4f=DISK]', sharedLocks='[]'}' 2025-01-28 16:18:28,238+01 WARN [org.ovirt.engine.core.bll.storage.disk.image.TransferDiskImageCommand] (default task-544730) [97d84dea-5177-4dfe-b816-722a9d956c95] Validation of action 'TransferDiskImage' failed for user admin@internal-authz. Reasons: VAR__TYPE__DISK,VAR__ACTION__TRANSFER,ACTION_TYPE_FAILED_DISK_PLUGGED_TO_NON_DOWN_VMS,$DiskName ocp_test_vm_disk,$VmList ocp-test-vm ~~~