Uploaded image for project: 'Migration Toolkit for Virtualization'
  1. Migration Toolkit for Virtualization
  2. MTV-3798

Performance degradation in MTV 2.10 compared to 2.9 during multi-VM migrations across multiple ESXi hosts

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Normal Normal
    • None
    • 2.10.0
    • Controller
    • Quality / Stability / Reliability
    • False
    • Hide

      None

      Show
      None
    • False

      Description:

      During performance validation for MTV 2.10, significant degradation was observed in total migration duration when running large-scale migration scenarios involving multiple ESXi hosts and concurrent VMs.

      The degradation is consistent across multiple cycles and appears to be correlated with increased DiskTransferV2v phase durations.
      This behavior was not observed in MTV 2.9, where overall migration times were significantly shorter under identical test conditions.

      Test Scenarios Impacted:

      TC 2.8 – 80 VMs, 1 Disk (50GB, 35% usage), 8 ESXi Hosts

       

      Metric MTV 2.9 (Mean Sec) MTV 2.10 (Mean Sec) Δ (%)
      Initialize 45.06 65.44 +45.21
      DiskAllocation 31.78 17.66 -44.4
      ImageConversion 81.45 88.41 +8.55
      DiskTransferV2v 1426.80 2151.11 +50.77
      VirtualMachineCreation 0.20 0.21 +6.25
      MigrationTime 1622.73 2369.24 +46.00
      Avg Overhead 37.44 46.40 +23.95

      Avg Migration Duration:

      • MTV 2.9 → 0:37:32
      • MTV 2.10 → 0:45:33

       

       

      TC 3.5 – 50 VMs, 1 Disk (50GB, 35% usage), 5 ESXi Hosts

      Metric MTV 2.9 (Mean Sec) MTV 2.10 (Mean Sec) Δ (%)
      MigrationTime 1310.04 1805.08 +37.79
      DiskAllocation 20.16 31.78 +57.64
      DiskTransferV2v 1157.94 1627.50 +40.55
      Initialize 26.64 34.32 +28.83
      ImageConversion 77.94 80.40 +3.16
      VirtualMachineCreation 0.26 0.20 -23.08

       

      Avg Migration Duration:

      • MTV 2.9 → 0:21:32
      • MTV 2.10 → 0:36:07

       

      Key Observations:

      • The DiskTransferV2v phase shows a ~40–50% increase in duration across both test cases.
      • The degradation scales with the number of concurrent ESXi hosts and VMs, indicating possible contention or scheduling inefficiency introduced in MTV 2.10.
      • No infrastructure or network-level changes were introduced between 2.9 and 2.10 cycles.

      Environment Details:

        • vCenter: VC7-1
        • DataStores:
      • PerfTest_VC7_1_FC_10TB
      • PerfTest_VC7_1_ISCSI_24TB
        • ESXi Hosts: 5 - 8 hosts per test
        • Each VM: RHEL 7.9, 50GB disk (35% usage)
        • Migration type: Cold
        • Parallel VM limit: Max 10 per ESXi (20 in-flight total)

      Suspected Area:

       

        • Possibly introduced regression in I/O scheduling or throttling logic between MTV 2.9 and MTV 2.10.
        • Potential bottleneck in DiskTransferV2v phase handling when distributing multiple concurrent VM transfers across multiple ESXi hosts.
        • It’s also suspected that additional internal validations, conditions, or checks were added in MTV 2.10 compared to 2.9, which could be increasing processing overhead during the migration flow especially under multi-host and high-concurrency conditions.

       

              gcheresh@redhat.com Genadi Chereshnya
              tzahia Tzahi Ashkenazi
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Created:
                Updated: