Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-8070

Egress router pods in pending state post upgrading cluster to 4.11

    XMLWordPrintable

Details

    • +
    • Critical
    • Yes
    • SDN Sprint 232, SDN Sprint 233, SDN Sprint 234, SDN Sprint 235
    • 4
    • Rejected
    • False
    • Hide

      None

      Show
      None
    • N/A
    • Release Note Not Required
    • Customer Escalated

    Description

      Description of problem:

      After upgrading cluster from 4.10.47 to 4.11.25 issue is observed with Egress router pod, pods are in pending state.

      Version-Release number of selected component (if applicable):

      4.11.25

      How reproducible:

       

      Steps to Reproduce:

      1. Upgrade from 4.10.47 to 4.11.25
2. Check if co network is in Managed state
3. Verify that egress pods are not created with errors like :
55s         Warning   FailedCreatePodSandBox   pod/******     (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to create pod network sandbox *******_d6918859-a4e9-4e5b-ba44-acc70499fa7c_0(9c464935ebaeeeab7be0b056c3f7ed1b7279e21445b9febea29eb280f7ee7429): error adding pod ****** to CNI network "multus-cni-network": plugin type="multus" name="multus-cni-network" failed (add): [ns/pod/d6918859-a4e9-4e5b-ba44-acc70499fa7c:openshift-sdn]: error adding container to network "openshift-sdn": CNI request failed with status 400: 'could not open netns "/var/run/netns/503fb77f-3b96-4f23-8356-43e7ae1e1b49": unknown FS magic on "/var/run/netns/503fb77f-3b96-4f23-8356-43e7ae1e1b49": 1021994

      Actual results:

      Egress router pods in pending state with error message as below:
$ omg get events 
...
49s        Warning  FailedCreatePodSandBox  pod/xxxx  (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to create pod network sandbox k8s_xxxx_379fa7ec-4702-446c-9162-55c2f76989f6_0(86f8c76e9724216143bef024996cb14a7614d3902dcf0d3b7ea858298766630c): error adding pod xxx to CNI network "multus-cni-network": plugin type="multus" name="multus-cni-network" failed (add): [xxxx/xxxx/379fa7ec-4702-446c-9162-55c2f76989f6:openshift-sdn]: error adding container to network "openshift-sdn": CNI request failed with status 400: 'could not open netns "/var/run/netns/0d39f378-29fd-4858-a947-51c5c06f1598": unknown FS magic on "/var/run/netns/0d39f378-29fd-4858-a947-51c5c06f1598": 1021994

      Expected results:

      Egress router pods in running state

      Additional info:

      Workaround from https://access.redhat.com/solutions/6986283 works :
Edit sdn DS in openshift-sdn namespace : 
- mountPath: /host/var/run/netns <<<<< /var/run/netns
  mountPropagation: HostToContainer
  name: host-run-netns   
  readOnly: true

      Attachments

        Issue Links

          causes

          Bug - A problem that impairs or prevents the functions of the product. OCPBUGS-28920 OCP 4.13.30 - allow-from-ingress NetworkPolicy does not consistently allow traffic from HostNetworked pods or from node IP's (packet timeout)

          • Blocker - To be worked above all other priorities. This term is chosen when the severity of the issue is very high, has no workaround, or the effort for the change is comparatively low. Issues that may be very publicly visible and could generate significant media attention may also drive a higher priority.
          • Verified
          relates to

          Bug - A problem that impairs or prevents the functions of the product. OCPBUGS-3744 Egress router POD creation is failing while using openshift-sdn network plugin

          • Blocker - To be worked above all other priorities. This term is chosen when the severity of the issue is very high, has no workaround, or the effort for the change is comparatively low. Issues that may be very publicly visible and could generate significant media attention may also drive a higher priority.
          • Closed

          Bug - A problem that impairs or prevents the functions of the product. OCPBUGS-3889 Egress router POD creation is failing while using openshift-sdn network plugin

          • Blocker - To be worked above all other priorities. This term is chosen when the severity of the issue is very high, has no workaround, or the effort for the change is comparatively low. Issues that may be very publicly visible and could generate significant media attention may also drive a higher priority.
          • Closed

          Bug - A problem that impairs or prevents the functions of the product. OCPBUGS-3911 Egress router POD creation is failing while using openshift-sdn network plugin

          • Blocker - To be worked above all other priorities. This term is chosen when the severity of the issue is very high, has no workaround, or the effort for the change is comparatively low. Issues that may be very publicly visible and could generate significant media attention may also drive a higher priority.
          • Closed
          links to

          GitHub openshift/cluster-network-operator#1763: OCPBUGS-8070: Depreciate legacy field manager

          Web Link RHEA-2023:5006 rpm

          Activity

            People

              mkennell@redhat.com Martin Kennelly
              rhn-support-sgurnale Sunil Gurnale
              Jean Chen Jean Chen
              Votes:
              0 Vote for this issue
              Watchers:
              16 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: