Currently, MTV allows warm migration if CBT is enabled for the virtual machine, i.e. ctkEnabled = TRUE is set in VMware. After setting this option, MTV allows warm migration and the user may get the impression that everything works as expected.

As per VMware documentation, for the warm migration to work, the CBT for individual disks needs to be enabled as well by setting scsi0:0.ctkEnabled = TRUE. Unfortunately, MTV doesn't issue any warning if the user omits this setting. In this case, the warm migration seems to work; it just appears to perform very slowly and the VM incurs a long downtime similar to the cold migration.

This RFE requests to issue a warning to the user if a warm migration is performed while some disks don't have the CBT enabled. Ideally, this warning would be displayed in the Web UI. Dropping a warning message into the logs is another good option, for example:

WARN CBT disabled for disk XYZ while performing warm migration. The whole disk snapshot will be copied after the cutover resulting in a prolonged VM downtime.

When performing a warm migration with disk CBT disabled, the whole disk is copied over and over from VMware to OpenShift. You can recognize this situation by looking at the DataVolume.spec.checkpoints list. The previous field is an empty string "".

$ oc get dv migration2-vm-20567-9mgql -o yaml 

apiVersion: cdi.kubevirt.io/v1beta1
kind: DataVolume
metadata:
  name: migration2-vm-20567-9mgql
...
spec:
  checkpoints:
  - current: snapshot-20696
    previous: ""
  - current: snapshot-20697
    previous: ""
  - current: snapshot-20698
    previous: ""
  finalCheckpoint: true

The worst part comes after the cutover. The VM is shut down by MTV but doesn't come up in OpenShift any time soon as the whole snapshot needs to be copied to OpenShift. This behavior is similar to cold migration and totally unexpected by the user.

When performing a warm migration with disk CBT enabled, only the differences between snapshots (checkpoints)  are copied from VMware to OpenShift:

$ oc get dv migration1-vm-33016-dtn5b -o yaml

apiVersion: cdi.kubevirt.io/v1beta1
kind: DataVolume
metadata:...
  name: migration1-vm-33016-dtn5b
...
spec:
  checkpoints:
  - current: snapshot-36007
    previous: ""
  - current: snapshot-36008
    previous: 52 02 9c 0f 4c 34 45 de-2e af bc f1 5d 05 06 fc/9
  - current: snapshot-36009
    previous: 52 02 9c 0f 4c 34 45 de-2e af bc f1 5d 05 06 fc/13
  - current: snapshot-36010
    previous: 52 02 9c 0f 4c 34 45 de-2e af bc f1 5d 05 06 fc/17
  - current: snapshot-36011
    previous: 52 02 9c 0f 4c 34 45 de-2e af bc f1 5d 05 06 fc/21
  finalCheckpoint: true