Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-8070

Egress router pods in pending state post upgrading cluster to 4.11

XMLWordPrintable

    • +
    • Critical
    • Yes
    • SDN Sprint 232, SDN Sprint 233, SDN Sprint 234, SDN Sprint 235
    • 4
    • Rejected
    • False
    • Hide

      None

      Show
      None
    • N/A
    • Release Note Not Required
    • Customer Escalated

      Description of problem:

      After upgrading cluster from 4.10.47 to 4.11.25 issue is observed with Egress router pod, pods are in pending state. 

      Version-Release number of selected component (if applicable):

      4.11.25

      How reproducible:

       

      Steps to Reproduce:

      1. Upgrade from 4.10.47 to 4.11.25
      2. Check if co network is in Managed state
      3. Verify that egress pods are not created with errors like :
      55s         Warning   FailedCreatePodSandBox   pod/******     (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to create pod network sandbox *******_d6918859-a4e9-4e5b-ba44-acc70499fa7c_0(9c464935ebaeeeab7be0b056c3f7ed1b7279e21445b9febea29eb280f7ee7429): error adding pod ****** to CNI network "multus-cni-network": plugin type="multus" name="multus-cni-network" failed (add): [ns/pod/d6918859-a4e9-4e5b-ba44-acc70499fa7c:openshift-sdn]: error adding container to network "openshift-sdn": CNI request failed with status 400: 'could not open netns "/var/run/netns/503fb77f-3b96-4f23-8356-43e7ae1e1b49": unknown FS magic on "/var/run/netns/503fb77f-3b96-4f23-8356-43e7ae1e1b49": 1021994
       
      
      

      Actual results:

      Egress router pods in pending state with error message as below:
      $ omg get events 
      ...
      49s        Warning  FailedCreatePodSandBox  pod/xxxx  (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to create pod network sandbox k8s_xxxx_379fa7ec-4702-446c-9162-55c2f76989f6_0(86f8c76e9724216143bef024996cb14a7614d3902dcf0d3b7ea858298766630c): error adding pod xxx to CNI network "multus-cni-network": plugin type="multus" name="multus-cni-network" failed (add): [xxxx/xxxx/379fa7ec-4702-446c-9162-55c2f76989f6:openshift-sdn]: error adding container to network "openshift-sdn": CNI request failed with status 400: 'could not open netns "/var/run/netns/0d39f378-29fd-4858-a947-51c5c06f1598": unknown FS magic on "/var/run/netns/0d39f378-29fd-4858-a947-51c5c06f1598": 1021994

      Expected results:

      Egress router pods in running state

      Additional info:

      Workaround from https://access.redhat.com/solutions/6986283 works :
      Edit sdn DS in openshift-sdn namespace : 
      - mountPath: /host/var/run/netns <<<<< /var/run/netns
        mountPropagation: HostToContainer
        name: host-run-netns   
        readOnly: true 

              mkennell@redhat.com Martin Kennelly
              rhn-support-sgurnale Sunil Gurnale
              Jean Chen Jean Chen
              Votes:
              0 Vote for this issue
              Watchers:
              16 Start watching this issue

                Created:
                Updated:
                Resolved: