Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-27367

pods assigned with Multus whereabouts IP get stuck in ContainerCreating state after OCP upgrading [Backport 4.13]

    XMLWordPrintable

Details

    • No
    • CNF Network Sprint 248
    • 1
    • Rejected
    • False
    • Hide

      None

      Show
      None
    • Hide
      Previously, pods assigned an IP from the pool created by the Whereabouts CNI plugin were getting stuck in `ContainerCreating` state after a node force reboot. With this release, the Whereabouts CNI plugin issue associated with the IP allocation after a node force reboot is resolved. (link:https://issues.redhat.com/browse/OCPBUGS-27367[OCPBUGS-27367])
      Show
      Previously, pods assigned an IP from the pool created by the Whereabouts CNI plugin were getting stuck in `ContainerCreating` state after a node force reboot. With this release, the Whereabouts CNI plugin issue associated with the IP allocation after a node force reboot is resolved. (link: https://issues.redhat.com/browse/OCPBUGS-27367 [ OCPBUGS-27367 ])
    • Bug Fix
    • Done
    • Customer Escalated
    • Hide
      11/7: tbd for 4.12.z, testing in progress
      9/10: u/s PR has merged, waiting for backports
      8/22: PR pending merge then backports; KNIECO-8019
      Show
      11/7: tbd for 4.12.z, testing in progress 9/10: u/s PR has merged, waiting for backports 8/22: PR pending merge then backports; KNIECO-8019

    Description

      Description of problem:

      pods assigned with Multus whereabouts IP get stuck in ContainerCreating state after OCP upgrading from 4.12.15 to 4.12.22. Not sure if upgrading cause the issue or node rebooting directly cause the issue.
      
      The error message is:
      (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to create pod network sandbox mypod-0-0-1-0_testproject_8c8500e1-1643-4716-8fd7-e032292c62ab_0(2baa045a1b19291769ed56bab288b60802179ff3138ffe0d16a14e78f9cb5e4f): error adding pod testproject_mypod-0-0-1-0 to CNI network "multus-cni-network": plugin type="multus" name="multus-cni-network" failed (add): [testproject/mypod-0-0-1-0/8c8500e1-1643-4716-8fd7-e032292c62ab:testproject-net-svc-kernel-bond]: error adding container to network "testproject-net-svc-kernel-bond": error at storage engine: k8s get error: client rate limiter Wait returned an error: rate: Wait(n=1) would exceed context deadline
      
      
      
      

      Version-Release number of selected component (if applicable):

       

      How reproducible:

      Not sure if it is reproducible

      Steps to Reproduce:

      1.
      2.
      3.
      

      Actual results:

      Pods stuck in ContainerCreating state 

      Expected results:

      Pods creates normally

      Additional info:

      Customer responded deleting statefulset and recreated it didn't work.
      The pods can be created normally after deleting corresponding ippools.whereabouts.cni.cncf.io manually
      $ oc delete ippools.whereabouts.cni.cncf.io 172.21.24.0-22 -n openshift-multus
      

      Attachments

        Issue Links

          Activity

            People

              lnoy@redhat.com Lior Noy
              rhn-support-yhuang Ying Huang
              Weibin Liang Weibin Liang
              Carlos Goncalves
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: