Uploaded image for project: 'OpenShift Virtualization'
  1. OpenShift Virtualization
  2. CNV-72782

Failed migration of vm in UDN

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Undefined Undefined
    • None
    • None
    • CNV Network
    • Quality / Stability / Reliability
    • 0.42
    • False
    • Hide

      None

      Show
      None
    • False
    • None
    • None

      Description of problem:

      During iperf3 server vm live migration, after client successful live migration, the server vmb virt-launcher pod is stuck on `Pending`

      Version-Release number of selected component (if applicable):

      cnv 4.21.0-ec.2
      

      How reproducible:

      Flaky

      Steps to Reproduce:

      All manifests attached for manual reproduction (Also can be reproduced with automation)
      
      1. Create namespace: oc create -f ns.yaml
      2. Create UDN: oc apply -f udn.yaml
      3. Create and run vmb: oc apply -f vmb.yaml, virtctl start vmb-udn-9ac7b54ab94740b9, copy vmb ipv4 address
      4. Create and run vma: oc apply -f vma.yaml, virtctl start vma-udn-c0a9f442fc794528
      5. Create iperf3 server on vmb: iperf3 --server --port 5201 --one-off
      6. Create iperf3 client on vma: iperf3 --client <ipv4-address-vmb> --time 0 --port 5201 --connect-timeout 300 &
      7. Migrate vma: virtctl migrate vma-udn-c0a9f442fc794528 
      8. Check TCP connectivity
      9. Migrate vmb: virtctl migrate vmb-udn-9ac7b54ab94740b9
      10. Check TCP connectivity

      Actual results:

      ERROR Status of VMIM vmb-udn-9ac7b54ab94740b9 is Scheduling
      
      oc get pods -w:
      NAME                                           READY   STATUS      RESTARTS   AGE
      virt-launcher-vma-udn-c0a9f442fc794528-bf2bf   3/3     Running     0          48s
      virt-launcher-vma-udn-c0a9f442fc794528-mlq8z   0/3     Completed   0          79s
      virt-launcher-vmb-udn-9ac7b54ab94740b9-lsjrr   3/3     Running     0          111s
      virt-launcher-vmb-udn-9ac7b54ab94740b9-tclds   0/3     Pending     0          23s
      
      
      End of oc describe pod post migration :
      Events: Events:
        Type     Reason            Age   From               Message
        ----     ------            ----  ----               -------
        Warning  FailedScheduling  65s   default-scheduler  0/3 nodes are available: 3 node(s) didn't match pod anti-affinity rules. preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod.
       
      
      Migration of server is unsuccessful

       

      Expected results:

      Migration of server successful (after successful client migration) and TCP connectivity preserves

      Additional info:

      - Can use the automation to recreate the bug
      - The test fails on bm10b-tlv2 cluster with cnv-4.21
      - This failure occurs only while the vms run iperf3 client and server

        1. ns.yaml
          0.1 kB
          Asia Zhivov Khromov
        2. pod_after_migration.yaml
          15 kB
          Asia Zhivov Khromov
        3. pod_before_migration.yaml
          13 kB
          Asia Zhivov Khromov
        4. udn.yaml
          0.2 kB
          Asia Zhivov Khromov
        5. vma.yaml
          1 kB
          Asia Zhivov Khromov
        6. vmb.yaml
          1 kB
          Asia Zhivov Khromov
        7. vmim.yaml
          2 kB
          Asia Zhivov Khromov

              phoracek@redhat.com Petr Horacek
              rh-ee-azhivovk Asia Zhivov Khromov
              Yossi Segev Yossi Segev
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Created:
                Updated: