Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-4909

Can't reach own service when egress netpol are enabled

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Normal Normal
    • None
    • 4.11.z
    • None

      Description of problem:

      When a netpol with policyType:Egress is active in a namespace, connections from a pod to a service pointing to the same pod are failing.
      Any other pods in the cluster and the namespace can reach that service without any issue

      Version-Release number of selected component (if applicable):

      Tested on 4.10.40 and 4.11.17

      How reproducible:

      Create a httpd pod with curl command in it and create a netpol with PolicyTypes:egress
      Run a curl command from the pod to the service targeting the same pod
      If there's only one pod, the issue occurs 100%
      If there's more than one pod, the issue occurs randomly from all pods that are targeting by the service

      Steps to Reproduce:

      1. oc new-project netpol
      2. oc new-app httpd  <== create a pod and the associated service
      3. oc create deployment otherpod --image=image-registry.openshift-image-registry.svc:5000/openshift/httpd <== just another pod not targeted by the service
      
      4. Check curl from pods to httpd.netpol.svc without any netpol: no issue
      for j in {1..3};do for i in $(oc get pod -o name);do echo -n "Test from $i: ";oc rsh $i curl --connect-timeout 3  http://httpd.netpol.svc:8080 &>/dev/null && echo "PASS" || echo "FAIL" ;done;echo "---";done
      Test from pod/httpd-78cc7b99b-p9768: PASS     <== always succeed
      Test from pod/otherpod-9f4bdfb4b-9h7vm: PASS  <== always succeed
       ---
      Test from pod/httpd-78cc7b99b-p9768: PASS
      Test from pod/otherpod-9f4bdfb4b-9h7vm: PASS
      ---
      Test from pod/httpd-78cc7b99b-p9768: PASS
      Test from pod/otherpod-9f4bdfb4b-9h7vm: PASS
      
      5. Add egress NetworkPolicies
      cat << EOF| oc create -f -
      apiVersion: networking.k8s.io/v1
      kind: NetworkPolicy
      metadata:
        name: allow-to-openshift-dns
      spec:
        egress:
        - ports:
          - port: 5353
            protocol: TCP
          - port: 5353
            protocol: UDP
        - to:
          - namespaceSelector:
              matchLabels:
                kubernetes.io/metadata.name: openshift-dns
        podSelector: {}
        policyTypes:
        - Egress
      EOF
      
      cat << EOF| oc create -f -
      apiVersion: networking.k8s.io/v1
      kind: NetworkPolicy
      metadata:
        name: allow-to-same-namespace
      spec:
        egress:
        - to:
          - podSelector: {}
        podSelector: {}
        policyTypes:
        - Egress
      EOF
      
      6. Check again the curl results, fail from the pod targeted by the service, no issue on the other pods
      for j in {1..3};do for i in $(oc get pod -o name);do echo -n "Test from $i: ";oc rsh $i  curl --connect-timeout 3  http://httpd.netpol.svc:8080 &>/dev/null && echo "PASS" || echo "FAIL" ;done;echo "---";done
      Test from pod/httpd-78cc7b99b-p9768: FAIL     <== always fail
      Test from pod/otherpod-9f4bdfb4b-9h7vm: PASS  <== always succeed
      ---
      Test from pod/httpd-78cc7b99b-p9768: FAIL
      Test from pod/otherpod-9f4bdfb4b-9h7vm: PASS
      ---
      Test from pod/httpd-78cc7b99b-p9768: FAIL
      Test from pod/otherpod-9f4bdfb4b-9h7vm: PASS
      
      
      7. Scale the httpd pod deployment to 2 to observe the "random" issue
      oc scale --replicas=2 deploy/httpd
      for j in {1..4};do for i in $(oc get pod -o name);do echo -n "Test from $i: ";oc rsh $i  curl --connect-timeout 3  http://httpd.netpol.svc:8080 &>/dev/null && echo "PASS" || echo "FAIL" ;done;echo "---";done
      Test from pod/httpd-78cc7b99b-p9768: FAIL    <== randomly fail
      Test from pod/httpd-78cc7b99b-s6n69: PASS    <== randomly fail
      Test from pod/otherpod-9f4bdfb4b-9h7vm: PASS <== always succeed
      ---
      Test from pod/httpd-78cc7b99b-p9768: PASS
      Test from pod/httpd-78cc7b99b-s6n69: FAIL
      Test from pod/otherpod-9f4bdfb4b-9h7vm: PASS
      ---
      Test from pod/httpd-78cc7b99b-p9768: FAIL
      Test from pod/httpd-78cc7b99b-s6n69: FAIL
      Test from pod/otherpod-9f4bdfb4b-9h7vm: PASS
      ---
      Test from pod/httpd-78cc7b99b-p9768: PASS
      Test from pod/httpd-78cc7b99b-s6n69: PASS
      Test from pod/otherpod-9f4bdfb4b-9h7vm: PASS
      

      Actual results:

       

      Expected results:

      Connections to the service should work regardless of the pod that initiates the connection 

      Additional info:

       

              bpickard@redhat.com Ben Pickard
              rhn-support-fgleizes Florian Gleizes (Inactive)
              Mike Fiedler Mike Fiedler
              Red Hat Employee
              Arti Sood
              Votes:
              1 Vote for this issue
              Watchers:
              13 Start watching this issue

                Created:
                Updated:
                Resolved: