Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-54341

Failed to recreate pod: SRIOV-CNI failed to load netconf: pci address 0000:af:05.0 is already allocated

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Not a Bug
    • Icon: Undefined Undefined
    • None
    • 4.19
    • Networking / SR-IOV
    • None
    • Quality / Stability / Reliability
    • False
    • Hide

      None

      Show
      None
    • 4
    • Important
    • None
    • None
    • None
    • None
    • CNF Network Sprint 269, CNF Network Sprint 273
    • 2
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      While recreating a deployment it sporadically fails to start:

        Warning  FailedCreatePodSandBox  43s   kubelet            Failed to create pod sandbox: rpc error: code = Unknown desc = failed to create pod network sandbox k8s_rdscore-pod-level-bond-two-76fcd48c6d-5drdv_rds-bonded-sriov-wlkd_33d50d6a-9c69-49c3-ac65-a4680c6894ea_0(441abf7f4980975d422972fcec8756f622d3d3e3e3dfe666c952555d502cbd16): error adding pod rds-bonded-sriov-wlkd_rdscore-pod-level-bond-two-76fcd48c6d-5drdv to CNI network "multus-cni-network": plugin type="multus-shim" name="multus-cni-network" failed (add): CmdAdd (shim): CNI request failed with status 400: 'ContainerID:"441abf7f4980975d422972fcec8756f622d3d3e3e3dfe666c952555d502cbd16" Netns:"/var/run/netns/05ded0b3-89f3-41fe-9e4c-f13fcbc681e1" IfName:"eth0" Args:"IgnoreUnknown=1;K8S_POD_NAMESPACE=rds-bonded-sriov-wlkd;K8S_POD_NAME=rdscore-pod-level-bond-two-76fcd48c6d-5drdv;K8S_POD_INFRA_CONTAINER_ID=441abf7f4980975d422972fcec8756f622d3d3e3e3dfe666c952555d502cbd16;K8S_POD_UID=33d50d6a-9c69-49c3-ac65-a4680c6894ea" Path:"" ERRORED: error configuring pod [rds-bonded-sriov-wlkd/rdscore-pod-level-bond-two-76fcd48c6d-5drdv] networking: [rds-bonded-sriov-wlkd/rdscore-pod-level-bond-two-76fcd48c6d-5drdv/33d50d6a-9c69-49c3-ac65-a4680c6894ea:sriov-mlx-two-bond]: error adding container to network "sriov-mlx-two-bond": SRIOV-CNI failed to load netconf: pci address 0000:af:05.0 is already allocated
      ': StdinData: {"binDir":"/var/lib/cni/bin","clusterNetwork":"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf","cniVersion":"0.3.1","daemonSocketDir":"/run/multus/socket","globalNamespaces":"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv","logLevel":"verbose","logToStderr":true,"name":"multus-cni-network","namespaceIsolation":true,"type":"multus-shim"}
        Warning  FailedCreatePodSandBox  28s  kubelet  Failed to create pod sandbox: rpc error: code = Unknown desc = failed to create pod network sandbox k8s_rdscore-pod-level-bond-two-76fcd48c6d-5drdv_rds-bonded-sriov-wlkd_33d50d6a-9c69-49c3-ac65-a4680c6894ea_0(f63eb0f097413fc6c1302891b8508e16cf8fe67a3d904d8754bd5ffe74ff205f): error adding pod rds-bonded-sriov-wlkd_rdscore-pod-level-bond-two-76fcd48c6d-5drdv to CNI network "multus-cni-network": plugin type="multus-shim" name="multus-cni-network" failed (add): CmdAdd (shim): CNI request failed with status 400: 'ContainerID:"f63eb0f097413fc6c1302891b8508e16cf8fe67a3d904d8754bd5ffe74ff205f" Netns:"/var/run/netns/7432a0a6-9e6b-4e3a-8f7c-0f245658302d" IfName:"eth0" Args:"IgnoreUnknown=1;K8S_POD_NAMESPACE=rds-bonded-sriov-wlkd;K8S_POD_NAME=rdscore-pod-level-bond-two-76fcd48c6d-5drdv;K8S_POD_INFRA_CONTAINER_ID=f63eb0f097413fc6c1302891b8508e16cf8fe67a3d904d8754bd5ffe74ff205f;K8S_POD_UID=33d50d6a-9c69-49c3-ac65-a4680c6894ea" Path:"" ERRORED: error configuring pod [rds-bonded-sriov-wlkd/rdscore-pod-level-bond-two-76fcd48c6d-5drdv] networking: [rds-bonded-sriov-wlkd/rdscore-pod-level-bond-two-76fcd48c6d-5drdv/33d50d6a-9c69-49c3-ac65-a4680c6894ea:sriov-mlx-two-bond]: error adding container to network "sriov-mlx-two-bond": SRIOV-CNI failed to load netconf: pci address 0000:af:05.0 is already allocated
      ': StdinData: {"binDir":"/var/lib/cni/bin","clusterNetwork":"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf","cniVersion":"0.3.1","daemonSocketDir":"/run/multus/socket","globalNamespaces":"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv","logLevel":"verbose","logToStderr":true,"name":"multus-cni-network","namespaceIsolation":true,"type":"multus-shim"}
        Normal   AddedInterface          14s (x3 over 43s)  multus   Add eth0 [10.130.0.58/23 fd01:0:0:3::3a/64] from ovn-kubernetes
      

       

      Steps to reproduce

      • create a deployment that utilizes SR-IOV pod level bonding
      • wait for pod to start and get into Running state
      • delete deployment
      • wait for deployment to go
      • recreate the deployment

              sscheink@redhat.com Sebastian Scheinkman
              yprokule@redhat.com Yurii Prokulevych
              None
              None
              Yurii Prokulevych Yurii Prokulevych
              None
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

                Created:
                Updated:
                Resolved: