Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-42159

High number of redundant kubeproxy rules present in OCP 4.15 with OpenshiftSDN

XMLWordPrintable

    • Important
    • None
    • False
    • Hide

      None

      Show
      None
    • Hide
      * Previously, a local patch to kube-proxy caused OpenShift SDN to add another duplicate copy of a particular rule to the iptables ruleset every time it resynchronized. The synchronization would slow down and eventually trigger the `NodeProxyApplySlow` alert. With this release, the kube-proxy patch has been fixed and the alert no longer appears. (link:https://issues.redhat.com/browse/OCPBUGS-42159[*OCPBUGS-42159*])
      _____________
      Cause: an incorrect local patch to kube-proxy
      Consequence: openshift-sdn added another duplicate copy of a particular rule to the iptables ruleset every time it resynchronized, causing the synchronization to slow down and eventually trigger the NodeProxyApplySlow alert.
      Fix: fixed the patch
      Result: no alert
      Show
      * Previously, a local patch to kube-proxy caused OpenShift SDN to add another duplicate copy of a particular rule to the iptables ruleset every time it resynchronized. The synchronization would slow down and eventually trigger the `NodeProxyApplySlow` alert. With this release, the kube-proxy patch has been fixed and the alert no longer appears. (link: https://issues.redhat.com/browse/OCPBUGS-42159 [* OCPBUGS-42159 *]) _____________ Cause: an incorrect local patch to kube-proxy Consequence: openshift-sdn added another duplicate copy of a particular rule to the iptables ruleset every time it resynchronized, causing the synchronization to slow down and eventually trigger the NodeProxyApplySlow alert. Fix: fixed the patch Result: no alert
    • Bug Fix
    • In Progress

      Description of problem:

      A high number of kubeproxy rules is observed on OCP cluster installed with version 4.15.19 and OpenshiftSDN network plugin. The quantity of redundant rules increases continuosly and it seems more related to the rules from openshift-ingress namespace. In the following example, it is present in a cluster node 157k redundant rules related to KUBE-MARK-MASQ  rule:

      $ less iptables-nat_rules.txt | grep openshift-ingress/router-nodeport-<svc-name> | wc -l
157761  <-----

 0     0 KUBE-MARK-MASQ  all  --  !tun0  *       0.0.0.0/0            0.0.0.0/0            /* masquerade traffic for openshift-ingress/router-nodeport-<svc-name>:http external destinations */

      Nodeport services seems to be more affected by the issue. 

      After a the node reboot, the quantity of rules drop, however some hours later, the issue reoccurs.

      Version-Release number of selected component (if applicable): OCP 4.15.19

      How reproducible: Not easily

      Actual results: Affected nodes are firing alerts NodeProxyApplySlow and ClusterProxyApplySlow 

      Expected results: Cluster shouldn't create this high quantity of redundant rules

      Additional info:

      Please fill in the following template while reporting a bug and provide as much relevant information as possible. Doing so will give us the best chance to find a prompt resolution.

      Affected Platforms: RHOCP 

              dwinship@redhat.com Dan Winship
              rhn-support-bgomes Bruno Gomes
              Zhanqi Zhao Zhanqi Zhao
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

                Created:
                Updated:
                Resolved: