Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-2016

Sorting of Virtual IPs added through NMState Operator

XMLWordPrintable

    • Quality / Stability / Reliability
    • False
    • Hide

      None

      Show
      None
    • None
    • Moderate
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Description of problem:

      In order to be able to segregate IP traffic on the firewall, SIX is exploring the possibility to use virtual IP addresses assigned to
      OpenShift worker nodes through the NMState operator [1]. Configuring additional IP addresses seems to work conceptually by adding all
      the IP addresses through a NodeNetworkConfigurationPolicy object, although we noticed that the operator (or probably the nmstate code)
      seems to sort the IP addresses alphanumerically, rather than "ip-numerically". One side effect is the following: The Kubelet startup
      procedure chooses the first IP address in the alphanumerically sorted list of the ip addresses. In our example, the host ip is
      10.147.216.2 (10.147.216.1 is the gateway), and the node has been joined to the cluster with this address originally. But since .10,
      .100, .101, .102, etc. alphanumerically comes before .2, after the next reboot the Kubelet tries to re-join the cluster with
      10.147.216.10. This problem can be circumvented by applying this procedure: [2]
      
      [1] https://docs.openshift.com/container-platform/4.10/networking/k8s_nmstate/k8s-nmstate-about-the-k8s-nmstate-operator.html
      [2] https://docs.openshift.com/container-platform/4.10/support/troubleshooting/troubleshooting-network-issues.html#nw-how-nw-iface-selected_troubleshooting-network-issues
      

      Version-Release number of selected component (if applicable):

       

      How reproducible:

      Always

      Steps to Reproduce:

      1. Add multiple IPs to a network interface with NodeNetworkConfigurationPolicy
      2. Check the status of the network interface with NodeNetworkState 
      

      Actual results:

      The IPs are listed alphanumerically

      Expected results:

      The IPs are listed "ip-numerically"

      Additional info:

      nodeip-configuration.service picks the first IP address it finds on the primary interface. So if nmstate happens to reconfigure the primary interface with additional IPs that alphanumerically sits before the originally configured IP, Kubelet will use this new IP rather than the original one.

              bnemec@redhat.com Benjamin Nemec
              rhn-support-rbobek Roman Bobek
              None
              None
              Victor Voronkov Victor Voronkov
              None
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Created:
                Updated:
                Resolved: