Uploaded image for project: 'OpenShift Virtualization'
  1. OpenShift Virtualization
  2. CNV-75942

Failed to create VMI on target cluster when map VM network to cluster user defined primary network

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Major Major
    • None
    • None
    • None
    • None
    • Quality / Stability / Reliability
    • 0.42
    • False
    • Hide

      None

      Show
      None
    • True
    • None

      Description of problem:

      Create cluster user defined primary network, migrate VM from vCenter to OCP cluster, map the VM network to default network (cluster user defined primary network) in cudn-ns namespace, conversion pod with correct network, the plan executed successfully.

      But the VMI failed to create on target cluster, the virt-launcher pod hit error as below, it tried to add openshift-ovn-kubernetes default network to VM, but that network is not exist.

      ERRORED: error configuring pod [cudn-ns/virt-launcher-mtv-feature-win2019-p4m4f] networking: Multus: [cudn-ns/virt-launcher-mtv-feature-win2019-p4m4f/84cab598-8e86-42f8-b244-624c869f3da3]: error loading k8s delegates k8s args: TryLoadPodDelegates: error in loading K8s cluster default network from pod annotation: tryLoadK8sPodDefaultNetwork: failed getting the delegate: getKubernetesDelegate: cannot find a network-attachment-definition (default) in namespace (openshift-ovn-kubernetes): networkattachmentdefinition.k8s.cni.cncf.io "default" not found

      Version-Release number of selected component (if applicable):

      MTV 2.10.0-6 build (IIB: forklift-fbc-prod-v420:on-pr-52e94dd9cd11eaf7e225644c31cb966f2c8e4494) with CNV 4.20.0

      How reproducible:

      Always

      Steps to Reproduce:

      1) Create the namespace: cudn-ns

      $ cat cudn-ns.yaml
      apiVersion: v1
      kind: Namespace
      metadata:
        name: cudn-ns
        labels:
          k8s.ovn.org/primary-user-defined-network: ""
      $ oc apply -f cudn-ns.yaml
      namespace/cudn-ns created
      $ oc label namespace/cudn-ns cluster-udn=cudn-test
      namespace/cudn-ns labeled

      2) Create the cluster user defined network: cudn-l2-net

      $ cat cudn.yaml
      apiVersion: k8s.ovn.org/v1
      kind: ClusterUserDefinedNetwork
      metadata:
        name: cudn-l2-net
      spec:
        namespaceSelector:
          matchLabels:
            cluster-udn: cudn-test
        network:
          topology: Layer2
          layer2:
            role: Primary
            ipam:
              lifecycle: Persistent
            subnets:
              - 10.232.18.0/24
      
      $ oc apply -f cudn.yaml
      clusteruserdefinednetwork.k8s.ovn.org/cudn-l2-net created
      
      $ oc get net-attach-def -A
      NAMESPACE   NAME          AGE
      cudn-ns     cudn-l2-net   142m

      3) In cudn-ns namespace, create the vCenter provider: v7-us-cudn, OCP cluster: ocpd-04-host

      apiVersion: forklift.konveyor.io/v1beta1
      kind: Provider
      metadata:
        name: ocpd-04-host
        namespace: cudn-ns
      spec:
        secret: {}
        type: openshift

      4) Create a cold migration plan to migrate VM: mtv-feature-win2019 from v7-us-cudn to ocp cluster: ocpd-04-host to target project: cudn-ns, map the VM network to default network

      5) Login to the conversion pod, check the pod with pod network: 10.129.3.90/23 and cudn network: 10.232.18.51/24, and the default route: 10.232.18.1 is using udn network , the pod can ping outside network

      $ oc describe pod cold-win2019-dhcp-cudn-vm-2202-2gb6p
      Name:              cold-win2019-dhcp-cudn-vm-2202-2gb6p
      Namespace:         cudn-ns
      Priority:          0
      Service Account:   default
      Node:              qemtvd-04-rngzt-worker-0-nw6nx/192.168.2.81
      Start Time:        Mon, 22 Sep 2025 20:04:05 +0800
      Labels:            forklift.app=virt-v2v
                         migration=57c602d7-3144-4f5d-9740-52b216fdbb06
                         plan=3806bb9a-300e-43d4-acc4-a1d934baf7c3
                         vmID=vm-2202
      Annotations:       k8s.ovn.org/open-default-ports:
                           - protocol: tcp
                             port: 2112
                           - protocol: tcp
                             port: 8080
                         k8s.ovn.org/pod-networks:
                           {"cudn-ns/cudn-l2-net":{"ip_addresses":["10.232.18.51/24"],"mac_address":"0a:58:0a:e8:12:33","gateway_ips":["10.232.18.1"],"routes":[{"des...
                         k8s.v1.cni.cncf.io/network-status:
                           [{
                               "name": "ovn-kubernetes",
                               "interface": "eth0",
                               "ips": [
                                   "10.129.3.90"
                               ],
                               "mac": "0a:58:0a:81:03:5a",
                               "dns": {}
                           },{
                               "name": "ovn-kubernetes",
                               "interface": "ovn-udn1",
                               "ips": [
                                   "10.232.18.51"
                               ],
                               "mac": "0a:58:0a:e8:12:33",
                               "default": true,
                               "dns": {}
                           }]
                         openshift.io/scc: forklift-controller-scc
                         seccomp.security.alpha.kubernetes.io/pod: localhost/profiles/unshare.json
                         security.openshift.io/validated-scc-subject-type: user
      Status:            Succeeded

      6) Plan executed successfully

      7) Check the VM yaml file on target cluster as below:

      $ oc get vm mtv-feature-win2019 -o yaml| grep interface -A 5
                interfaces:
                - binding:
                    name: l2bridge
                  macAddress: 00:50:56:b4:ad:82
                  model: virtio
                  name: net-0
      $ oc get vm mtv-feature-win2019 -o yaml| grep network -A 3
            networks:
            - name: net-0
              pod: {}

      8) VM failed to boot, the pod: virt-launcher-mtv-feature-win2019-p4m4f with error below, more details in attached files:  virt-launcher-mtv-feature-win2019-p4m4f-describe, virt-launcher-mtv-feature-win2019-p4m4f-yaml

      $ oc get pod virt-launcher-mtv-feature-win2019-p4m4f -o yaml
      apiVersion: v1
      kind: Pod
      metadata:
        annotations:
          descheduler.alpha.kubernetes.io/request-evict-only: ""
          k8s.ovn.org/pod-networks: '{"cudn-ns/cudn-l2-net":{"ip_addresses":["10.232.18.53/24"],"mac_address":"0a:58:0a:e8:12:35","gateway_ips":["10.232.18.1"],"routes":[{"dest":"172.30.0.0/16","nextHop":"10.232.18.1"},{"dest":"100.65.0.0/16","nextHop":"10.232.18.1"}],"ip_address":"10.232.18.53/24","gateway_ip":"10.232.18.1","tunnel_id":60,"role":"primary"},"default":{"ip_addresses":["10.129.3.105/23"],"mac_address":"00:50:56:b4:ad:82","routes":[{"dest":"10.128.0.0/14","nextHop":"10.129.2.1"},{"dest":"100.64.0.0/16","nextHop":"10.129.2.1"}],"ip_address":"10.129.3.105/23","role":"infrastructure-locked"}}'
          k8s.ovn.org/primary-udn-ipamclaim: mtv-feature-win2019.net-0......
          v1.multus-cni.io/default-network: '[{"name":"default","namespace":"openshift-ovn-kubernetes","mac":"00:50:56:b4:ad:82","ipam-claim-reference":"mtv-feature-win2019.net-0"}]'
          vm.kubevirt.io/os: windows
      
      $ oc describe pod virt-launcher-mtv-feature-win2019-p4m4f
      Name:              virt-launcher-mtv-feature-win2019-p4m4f
      Namespace:         cudn-ns
      Priority:          0
      Service Account:   default
      Node:              qemtvd-04-rngzt-worker-0-nw6nx/192.168.2.81
      Events:.......
      ERRORED: error configuring pod [cudn-ns/virt-launcher-mtv-feature-win2019-p4m4f] networking: Multus: [cudn-ns/virt-launcher-mtv-feature-win2019-p4m4f/84cab598-8e86-42f8-b244-624c869f3da3]: error loading k8s delegates k8s args: TryLoadPodDelegates: error in loading K8s cluster default network from pod annotation: tryLoadK8sPodDefaultNetwork: failed getting the delegate: getKubernetesDelegate: cannot find a network-attachment-definition (default) in namespace (openshift-ovn-kubernetes): networkattachmentdefinition.k8s.cni.cncf.io "default" not found

      Actual results:

      In step8: The VM failed to start

      Expected results:

      In step8: The VM start successfully

      Additional info:

       

              mduarted@redhat.com Miguel Duarte de Mora Barroso
              chhu@redhat.com Chenli Hu
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

                Created:
                Updated: