Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-16008

pods assigned with Multus whereabouts IP get stuck in ContainerCreating state after OCP upgrading [Backport 4.12]

XMLWordPrintable

    • +
    • No
    • CNF Network Sprint 239, CNF Network Sprint 240, CNF Network Sprint 241, CNF Network Sprint 247, CNF Network Sprint 248
    • 5
    • Rejected
    • False
    • Hide

      None

      Show
      None
    • Hide
      Previously, pods assigned an IP from the pool created by the Whereabouts CNI plugin were getting stuck in `ContainerCreating` state after a node force reboot. With this release, the Whereabouts CNI plugin issue associated with the IP allocation after a node force reboot is resolved. (link:https://issues.redhat.com/browse/OCPBUGS-16008[OCPBUGS-16008])
      Show
      Previously, pods assigned an IP from the pool created by the Whereabouts CNI plugin were getting stuck in `ContainerCreating` state after a node force reboot. With this release, the Whereabouts CNI plugin issue associated with the IP allocation after a node force reboot is resolved. (link: https://issues.redhat.com/browse/OCPBUGS-16008 [ OCPBUGS-16008 ])
    • Bug Fix
    • Done
    • Customer Escalated
    • Hide
      11/7: tbd for 4.12.z, testing in progress
      9/10: u/s PR has merged, waiting for backports
      8/22: PR pending merge then backports; KNIECO-8019
      Show
      11/7: tbd for 4.12.z, testing in progress 9/10: u/s PR has merged, waiting for backports 8/22: PR pending merge then backports; KNIECO-8019

      Description of problem:

      pods assigned with Multus whereabouts IP get stuck in ContainerCreating state after OCP upgrading from 4.12.15 to 4.12.22. Not sure if upgrading cause the issue or node rebooting directly cause the issue.

The error message is:
(combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to create pod network sandbox mypod-0-0-1-0_testproject_8c8500e1-1643-4716-8fd7-e032292c62ab_0(2baa045a1b19291769ed56bab288b60802179ff3138ffe0d16a14e78f9cb5e4f): error adding pod testproject_mypod-0-0-1-0 to CNI network "multus-cni-network": plugin type="multus" name="multus-cni-network" failed (add): [testproject/mypod-0-0-1-0/8c8500e1-1643-4716-8fd7-e032292c62ab:testproject-net-svc-kernel-bond]: error adding container to network "testproject-net-svc-kernel-bond": error at storage engine: k8s get error: client rate limiter Wait returned an error: rate: Wait(n=1) would exceed context deadline

      Version-Release number of selected component (if applicable):

      How reproducible:

      Not sure if it is reproducible

      Steps to Reproduce:

      1.
2.
3.

      Actual results:

      Pods stuck in ContainerCreating state

      Expected results:

      Pods creates normally

      Additional info:

      Customer responded deleting statefulset and recreated it didn't work.
The pods can be created normally after deleting corresponding ippools.whereabouts.cni.cncf.io manually
$ oc delete ippools.whereabouts.cni.cncf.io 172.21.24.0-22 -n openshift-multus

              lnoy@redhat.com Lior Noy (Inactive)
              rhn-support-yhuang Ying Huang
              Weibin Liang Weibin Liang
              Carlos Goncalves
              Votes:
              2 Vote for this issue
              Watchers:
              34 Start watching this issue

                Created:
                Updated:
                Resolved: