Uploaded image for project: 'OpenShift Virtualization'
  1. OpenShift Virtualization
  2. CNV-64972

[CCLM] Failed to migrate the VM with hotplug disk

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done-Errata
    • Icon: Major Major
    • CNV v4.20.0
    • None
    • Storage Platform
    • None
    • Quality / Stability / Reliability
    • 8
    • False
    • Hide

      None

      Show
      None
    • False
    • CNV v4.20.0.rhel9-131
    • CNV Storage 275, CNV Storage 276
    • None

      Description of problem:

      [CCLM] Failed to migrate the VM with hotplug disk

      Version-Release number of selected component (if applicable):

      CNV 4.19

      How reproducible:

      Always

      Steps to Reproduce:

      1. Create a source VM with hotplug disk on cluster1
      2. Create a target VM (blank disk) with hotplug disk on cluster2
      3. Create source VMIM and target VMIM to trigger the cross cluster live migration 
      

      Actual results:

      Target VMIM failed. 
      $ oc get vmim
      NAME          PHASE    VMI
      vmim-target   Failed   vm-fedora-datavolume
      
      describe the vmim
      Events:
        Type     Reason            Age    From                       Message
        ----     ------            ----   ----                       -------
        Normal   SuccessfulCreate  6m35s  virtualmachine-controller  Created migration target pod virt-launcher-vm-fedora-datavolume-jwfj4
        Warning  FailedMigration   21s    virtualmachine-controller  Migration failed because target pod shutdown during migration
      
      
      target virt-launcher pod
          State:      Terminated
            Reason:   Error
            Message:  07-09T13:15:33.582000Z"}
      {"component":"virt-launcher","level":"info","msg":"hostname: vm-fedora-datavolume","subcomponent":"libvirt","thread":"40","timestamp":"2025-07-09T13:15:33.582000Z"}
      {"component":"virt-launcher","level":"error","msg":"internal error: Unable to get session bus connection: Cannot spawn a message bus without a machine-id: Unable to load /var/lib/dbus/machine-id or /etc/machine-id: Failed to open file "/var/lib/dbus/machine-id": No such file or directory","pos":"virGDBusGetSessionBus:126","subcomponent":"libvirt","thread":"40","timestamp":"2025-07-09T13:15:33.582000Z"}
      {"component":"virt-launcher","level":"error","msg":"internal error: Unable to get system bus connection: Could not connect: No such file or directory","pos":"virGDBusGetSystemBus:99","subcomponent":"libvirt","thread":"40","timestamp":"2025-07-09T13:15:33.582000Z"}
      {"component":"virt-launcher","level":"info","msg":"Connected to libvirt daemon","pos":"libvirt.go:646","timestamp":"2025-07-09T13:15:33.841858Z"}
      {"component":"virt-launcher","level":"info","msg":"Registered libvirt event notify callback","pos":"client.go:596","timestamp":"2025-07-09T13:15:33.844633Z"}
      {"component":"virt-launcher","level":"info","msg":"Marked as ready","pos":"virt-launcher.go:77","timestamp":"2025-07-09T13:15:33.844778Z"}
      panic: timed out waiting for domain to be defined
      {"component":"virt-launcher-monitor","level":"info","msg":"Reaped Launcher main pid","pos":"virt-launcher-monitor.go:128","timestamp":"2025-07-09T13:21:22.850656Z"}
      {"component":"virt-launcher-monitor","level":"info","msg":"Reaped pid 8 with status 512","pos":"virt-launcher-monitor.go:131","timestamp":"2025-07-09T13:21:22.850795Z"}
      {"component":"virt-launcher-monitor","level":"error","msg":"dirty virt-launcher shutdown: exit-code 2","pos":"virt-launcher-monitor.go:145","timestamp":"2025-07-09T13:21:22.850811Z"}
      {"component":"virt-launcher-monitor","level":"info","msg":"virt-launcher-monitor: Exiting...","pos":"virt-launcher-monitor.go:91","timestamp":"2025-07-09T13:21:30.660127Z"}
      
      
      source vmim keeps in scheduling
      $ oc get vmim -w
      NAME          PHASE        VMI
      vmim-source   Scheduling   vm-fedora-datavolume

      Expected results:

      migration succeeded

      Additional info:

      received vm yaml attached. 
      
      we can not hotplug disk by "virtctl addvolume" for a receive vm, since the vm is not a normal running vm, but we can define the hotplug disk in the yaml as attachment. 
      $ oc get vmi
      NAME                   AGE   PHASE            IP    NODENAME   READY
      vm-fedora-datavolume   66s   WaitingForSync                    False
      $ virtctl addvolume vm-fedora-datavolume  --volume-name=blank-dv --serial=1234567890 --persist
      Successfully submitted add volume request to VM vm-fedora-datavolume for volume blank-dv
      $ virtctl removevolume vm-fedora-datavolume  --volume-name=blank-dv
      error removing volume, Operation cannot be fulfilled on virtualmachineinstance.kubevirt.io "vm-fedora-datavolume": VMI is not running

              rhn-support-awels Alexander Wels
              yadu1@redhat.com Yan Du
              Yan Du Yan Du
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Created:
                Updated:
                Resolved: