Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-17976

After node reboots, interfaces do not come up with the defined NM configs.

XMLWordPrintable

    • Quality / Stability / Reliability
    • False
    • None
    • None
    • Important
    • No
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Description of problem:

      After rebooting nodes, the networkmanager configuration for interfaces is not being picked up. Instead of the defined static IP and configuration, interfaces come up as dhcp. 

      Version-Release number of selected component (if applicable):

      4.13.4

      How reproducible:

      Every Time

      Steps to Reproduce:

      1. Cluster was deployed and then end customer asked for a second interface to be added and the MTU be increased to 9000. This change was implemented by RH solutions architects. We know changing the MTU post installation is not supported. This MTU change was verified as working and worked for weeks before issues were found. 
      2. For a networking hardware change (new cables) the nodes were cordoned, drained, and rebooted. When the nodes came back up, the interfaces were not picking up their expected configurations. 
      3. To temporarily resolve the issue, the manual commands below were ran:
      ```
      HOST_IP="172.29.135.31"
      HOST_MASK="24"
      HOST_GW="172.29.135.1"
      HOST_INT="eno12399np0"
      
      ip addr add $HOST_IP/$HOST_MASK dev $HOST_INT
      ip route add 0.0.0.0/0 via $HOST_GW dev $HOST_INT
      
      touch /run/resolv-prepender-kni-conf-done
      ln -s /etc/resolv.conf /var/run/NetworkManager/resolv.conf
      
      systemctl restart kubelet
      ```
      

      Actual results:

      Node interfaces do not pick up NMstate defined configurations

      Expected results:

      Node interfaces start and use the NMstate defined configurations

      Additional info:

      Cluster must-gather: https://attachments.access.redhat.com/hydra/rest/cases/03589546/attachments/b652b2ac-5eb2-46cf-8eac-31431a615299?usePresignedUrl=true
      
      Node sosreport: https://gss--c.vf.force.com/apex/Case_View?usePresignedUrl=true
      
      NMstate configuration files: https://attachments.access.redhat.com/hydra/rest/cases/03589546/attachments/61bd9cf5-9e05-456b-aa20-331e47b23bee?usePresignedUrl=true
      
      sample-01 files are previous to the reboot. I also tried using sample-02 files manually once the systems were back online, but network manager still ignores these deployments, fails on reboot, and the eno12399np0 nics are in DHCP mode (in both cases).

              bnemec@redhat.com Benjamin Nemec
              rhn-support-emahoney Evan Mahoney
              None
              None
              Qiong Wang Qiong Wang
              None
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

                Created:
                Updated:
                Resolved: