Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-48285

Once the available node-subnets runs out, the OVN-Kubernetes keeps assigning duplicate node-subnets to the nodes

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Normal Normal
    • None
    • 4.14
    • None
    • Quality / Stability / Reliability
    • False
    • Hide

      None

      Show
      None
    • None
    • Important
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Description of problem:

      In a scenario where the clusterNetwork is configured with a specific network range + hostprefix, once the available subnets are assigned to the maximum quantity of nodes, the host-subnets from new nodes are configured considering duplicate information where this causes timeouts and pod-to-pod communication issues in the platform. 

      See the example below:

          clusterNetwork:
          - cidr: 10.xx.0.0/18
            hostPrefix: 22

      A cidr /18 + hostPrefix 22 is able to assign host-subnets to 16 nodes. Once the cluster has 17 nodes, the new node is going to have a duplicate network already assigned to the nodes present in the cluster. In the following example, the cluster has 21 nodes and 5 networks are duplicate: 

       

      $ oc get nodes -oyaml | grep k8s.ovn.org/node-subnets | awk '{print $2}' | sort
      '{"default":["10.xx.0.0/22"]}'
      '{"default":["10.xx.12.0/22"]}'
      '{"default":["10.xx.16.0/22"]}'
      '{"default":["10.xx.20.0/22"]}'
      '{"default":["10.xx.24.0/22"]}'
      '{"default":["10.xx.28.0/22"]}'
      '{"default":["10.xx.32.0/22"]}'
      '{"default":["10.xx.36.0/22"]}'
      '{"default":["10.xx.40.0/22"]}'
      '{"default":["10.xx.4.0/22"]}'
      '{"default":["10.xx.44.0/22"]}'
      '{"default":["10.xx.48.0/22"]}'
      '{"default":["10.xx.48.0/22"]}' <--
      '{"default":["10.xx.52.0/22"]}'
      '{"default":["10.xx.56.0/22"]}' 
      '{"default":["10.xx.56.0/22"]}' <--
      '{"default":["10.xx.56.0/22"]}' <--
      '{"default":["10.xx.56.0/22"]}' <--
      '{"default":["10.xx.56.0/22"]}' <--
      '{"default":["10.xx.60.0/22"]}'
      '{"default":["10.xx.8.0/22"]}'
      

      No error message is reported in the OVN-K's components. The OVN-K should not allow these network assignments. 

      Version-Release number of selected component (if applicable): 4.14.37

      How reproducible: Easily. 

      Steps to Reproduce:

      Configure a small clusternetwork and create a high quantity of nodes to cause the duplicate networks.

      Actual results: No messages are reported  except the following in the northd pod: 

      northd|WARN|No path for static route 10.xxx.56.0/22; next hop 10.xxx.56.2 

      Expected results: It should not allow duplicate networks. 

              rhn-support-arghosh Arnab Ghosh
              rhn-support-bgomes Bruno Gomes
              None
              None
              Anurag Saxena Anurag Saxena
              None
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

                Created:
                Updated:
                Resolved:

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0 minutes
                  0m
                  Logged:
                  Time Spent - 1 day
                  1d