Uploaded image for project: 'Migration Toolkit for Virtualization'
  1. Migration Toolkit for Virtualization
  2. MTV-2774

Reconciliation of every plan randomly fails if there is a plan that's referring to an inexistent VM

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Major Major
    • 2.9.1
    • 2.8.5
    • Controller
    • Moderate

      Description of problem:

      I have some warm migration plans already running in the cluster. I created a new plan for the VM and waited for DV to pause. It is referring to the VM MOID vm-77272:

      # oc get plan nijin-rhel9-clone -o json|jq '.spec.vms'
      [
        {
          "id": "vm-77272",
          "name": "nijin-rhel9-clone"
        }
      ]

      When the DV is paused, I removed the VM from the vCenter, the plan reconciliation started failing as expected for this plan:

      {"level":"error","ts":"2025-06-20 08:43:27.022","logger":"plan|bjpng","msg":"Reconcile failed.","plan":{"name":"nijin-rhel9-clone","namespace":"openshift-mtv"},"error":"Resource ref.Ref{ID:\"vm-77272\", Name:\"\", Namespace:\"\", Type:\"\"} not found.","stacktrace":"github.com/konveyor/forklift-controller/pkg/controller/base.(*Reconciler).Ended\n\t/remote-source/app/pkg/controller/base/controller.go:57\ngithub.com/konveyor/forklift-controller/pkg/controller/plan.Reconciler.Reconcile.func1\n\t/remote-source/app/pkg/controller/plan/controller.go:171\ngithub.com/konveyor/forklift-controller/pkg/controller/plan.Reconciler.Reconcile\n\t/remote-source/app/pkg/controller/plan/controller.go:259\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:119\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:316\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:227"}

      However, after this, every other plans which were running fine started randomly failing on reconciliation referring to the same vm MOID "vm-77272":

       

      {"level":"error","ts":"2025-06-20 08:43:08.307","logger":"plan|t84ld","msg":"Reconcile failed.","plan":{"name":"test-plan-14","namespace":"openshift-mtv"},"error":"Resource ref.Ref{ID:\"vm-77272\", Name:\"\", Namespace:\"\", Type:\"\"} not found.","stacktrace":"github.com/konveyor/forklift-controller/pkg/controller/base.(*Reconciler).Ended\n\t/remote-source/app/pkg/controller/base/controller.go:57\ngithub.com/konveyor/forklift-controller/pkg/controller/plan.Reconciler.Reconcile.func1\n\t/remote-source/app/pkg/controller/plan/controller.go:171\ngithub.com/konveyor/forklift-controller/pkg/controller/plan.Reconciler.Reconcile\n\t/remote-source/app/pkg/controller/plan/controller.go:259\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:119\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:316\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:227"}
       
      {"level":"error","ts":"2025-06-20 08:43:23.991","logger":"plan|5pcmk","msg":"Reconcile failed.","plan":{"name":"test-plan-21","namespace":"openshift-mtv"},"error":"Resource ref.Ref{ID:\"vm-77272\", Name:\"\", Namespace:\"\", Type:\"\"} not found.","stacktrace":"github.com/konveyor/forklift-controller/pkg/controller/base.(*Reconciler).Ended\n\t/remote-source/app/pkg/controll.....ler/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:227"}
      {"level":"error","ts":"2025-06-20 08:43:18.254","logger":"plan|s8vhg","msg":"Reconcile failed.","plan":{"name":"test-plan-n","namespace":"openshift-mtv"},"error":"Resource ref.Ref{ID:\"vm-77272\", Name:\"\", Namespace:\"\", Type:\"\"} not found.","stacktrace":"github.com/konveyor/forklift-controller/pkg/controller/base.(*Reconciler).Ended\n\t/remote-source/app/pkg/contro........roller).Start.func2.2\n\t/remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:227"}
       
      {"level":"error","ts":"2025-06-20 08:43:11.461","logger":"plan|82wd2","msg":"Reconcile failed.","plan":{"name":"test-plan-a","namespace":"openshift-mtv"},"error":"Resource ref.Ref{ID:\"vm-77272\", Name:\"\", Namespace:\"\", Type:\"\"} not found.","stacktrace":"github.com/konveyor/forklift-controller/pkg/controller/base.(*Reconciler).Ended\n\t/remote-source/app/pkg/controller/base/controller.go:57\ngithub.com/konveyor/forklift-controller/pkg/controller/plan.Rec..........go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:227"}
       
      

       

      Version-Release number of selected component (if applicable):

      Migration Toolkit for Virtualization Operator 2.8.5

      How reproducible:

      Always
      

      Steps to Reproduce:

      1. Create a new plan for a VM in vSphere.
      2. Start the plan and wait for it to pause.
      3. Once it is pause, delete the VM from vSphere.
      4. Create a new plan for another VM and start it.
      5. Reconciliation will fail randomly for this plan referring to the deleted VMs MOID. 

      Actual results:

      Reconciliation of every plan randomly fails if there is a plan that's referring to an inexistent VM 

      Expected results:

      A broken migration plan shouldn't affect other plans.

      Additional info:

       

              rh-ee-ehazan Elad Hazan
              rhn-support-nashok Nijin Ashok
              Chenli Hu Chenli Hu
              Votes:
              1 Vote for this issue
              Watchers:
              6 Start watching this issue

                Created:
                Updated:
                Resolved: