Uploaded image for project: 'Migration Toolkit for Virtualization'
  1. Migration Toolkit for Virtualization
  2. MTV-2261

Populator pod cannot be created when using a migration network

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Major Major
    • None
    • 2.7.11
    • Controller
    • Incidents & Support
    • False
    • Hide

      None

      Show
      None
    • True
    • Important

      Description of problem:

      When using a migration network, the populator pod is stuck in ContainerCreating state and this event is logged:
      
      0s          Warning   FailedCreatePodSandBox        pod/populate-e96a5378-a8ae-49d5-bdb1-75d07e88df19                  Failed to create pod sandbox: rpc error: code = Unknown desc = failed to create pod network sandbox k8s_populate-e96a5378-a8ae-49d5-bdb1-75d07e88df19_openshift-mtv_f1e4b939-eaf8-4f65-b4c8-41196ec46560_0(3ae8a92556e3eff24d5785d8f8009114e79a505d8374cb82ba6cb74f120dd26a): error adding pod openshift-mtv_populate-e96a5378-a8ae-49d5-bdb1-75d07e88df19 to CNI network "multus-cni-network": plugin type="multus-shim" name="multus-cni-network" failed (add): CmdAdd (shim): CNI request failed with status 400: 'ContainerID:"3ae8a92556e3eff24d5785d8f8009114e79a505d8374cb82ba6cb74f120dd26a" Netns:"/var/run/netns/034f537f-2f3a-439d-be85-bd45b8098ca3" IfName:"eth0" Args:"IgnoreUnknown=1;K8S_POD_NAMESPACE=openshift-mtv;K8S_POD_NAME=populate-e96a5378-a8ae-49d5-bdb1-75d07e88df19;K8S_POD_INFRA_CONTAINER_ID=3ae8a92556e3eff24d5785d8f8009114e79a505d8374cb82ba6cb74f120dd26a;K8S_POD_UID=f1e4b939-eaf8-4f65-b4c8-41196ec46560" Path:"" ERRORED: error configuring pod [openshift-mtv/populate-e96a5378-a8ae-49d5-bdb1-75d07e88df19] networking: [openshift-mtv/populate-e96a5378-a8ae-49d5-bdb1-75d07e88df19/f1e4b939-eaf8-4f65-b4c8-41196ec46560:ovn-localnet-1]: error adding container to network "ovn-localnet-1": CNI request failed with status 400: '[openshift-mtv/populate-e96a5378-a8ae-49d5-bdb1-75d07e88df19 3ae8a92556e3eff24d5785d8f8009114e79a505d8374cb82ba6cb74f120dd26a network ovn-localnet-1 NAD openshift-mtv/mtv-migrations] [openshift-mtv/populate-e96a5378-a8ae-49d5-bdb1-75d07e88df19 3ae8a92556e3eff24d5785d8f8009114e79a505d8374cb82ba6cb74f120dd26a network ovn-localnet-1 NAD openshift-mtv/mtv-migrations] failed to get pod annotation: timed out waiting for annotations: context deadline exceeded...

      Version-Release number of selected component (if applicable):

      OCP 4.17
      mtv-operator.v2.7.11

      How reproducible:

      Always

      Steps to Reproduce:

      1. Create a NAD for the migration:
      
      apiVersion: k8s.cni.cncf.io/v1
      kind: NetworkAttachmentDefinition
      metadata:
        annotations:
          forklift.konveyor.io/route: 10.37.201.254
        name: mtv-migrations
        namespace: openshift-mtv
      spec:
        config: |
          {
                  "cniVersion": "0.3.1",
                  "name": "ovn-localnet-1",
                  "type": "ovn-k8s-cni-overlay",
                  "topology": "localnet",
                  "netAttachDefName": "openshift-mtv/mtv-migrations",
                  "subnets": "10.37.200.0/23"
          }
      
      
      2. Configure the OCP provider to use the network:
      
      apiVersion: forklift.konveyor.io/v1beta1
      kind: Provider
      metadata:
        annotations:
          forklift.konveyor.io/defaultTransferNetwork: openshift-mtv/mtv-migrations
        name: host
        namespace: openshift-mtv
      spec:
        secret: {}
        type: openshift
        url: ""
      
      
      3. Run a migration

      Actual results:

      Populator pod stuck in ContainerCreating, with FailedCreatePodSandBox events logged.
      
      The populator pod has the 'v1.multus-cni.io/default-network' annotation:
      
      apiVersion: v1
      kind: Pod
      metadata:
        annotations:
          k8s.ovn.org/pod-networks: '{"default":{"ip_addresses":["10.129.2.159/23"],"mac_address":"0a:58:0a:81:02:9f","gateway_ips":["10.129.2.1"],"routes":[{"dest":"10.128.0.0/14","nextHop":"10.129.2.1"},{"dest":"172.30.0.0/16","nextHop":"10.129.2.1"},{"dest":"169.254.169.5/32","nextHop":"10.129.2.1"},{"dest":"100.64.0.0/16","nextHop":"10.129.2.1"}],"ip_address":"10.129.2.159/23","gateway_ip":"10.129.2.1","role":"primary"}}'
          openshift.io/scc: forklift-controller-scc
          seccomp.security.alpha.kubernetes.io/pod: runtime/default
          v1.multus-cni.io/default-network: openshift-mtv/mtv-migrations
      

      Expected results:

      - No errors creating the pod.
      - Migration performed over the defined network.

      Additional info:

      Looks like this change is missing in v2.7:
      
      https://github.com/kubev2v/forklift/pull/1175

              mnecas@redhat.com Martin Necas
              rhn-support-jortialc Juan Orti
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated: