Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-18716

OVN br-int flows do not get updated on other nodes when a nodes bond MACADDR is changed to other slave interface after reboot.

    XMLWordPrintable

Details

    • Important
    • No
    • False
    • Hide

      None

      Show
      None
    • Hide
      Previously when a MAC address changes on the physical interface being used by OVN-Kubernetes, it would not be be updated correctly within OVN-Kubernetes and may cause traffic disruption and/or kube API outage from the node for a prolonged period of time. This is most common when a bond interface is being used, where the mac address of the bond may swap depending on which slave is the first to come up.

      With the fix OVN-Kubernetes will dynamically detect MAC address changes and update correctly.
      Show
      Previously when a MAC address changes on the physical interface being used by OVN-Kubernetes, it would not be be updated correctly within OVN-Kubernetes and may cause traffic disruption and/or kube API outage from the node for a prolonged period of time. This is most common when a bond interface is being used, where the mac address of the bond may swap depending on which slave is the first to come up. With the fix OVN-Kubernetes will dynamically detect MAC address changes and update correctly.
    • Bug Fix
    • In Progress
    • affects cu's with a bond for br-ex who have not followed the workaround in KCS

    Description

      Description of problem:

      OVN br-int ovs flows do not get updated on other nodes when a nodes bond MACADDR is changed to other slave interface after reboot. This causes network traffic coming for the sdn of one node to get dropped when it hits the node that changed mac addresses on its bond interface.

      Version-Release number of selected component (if applicable): 4.12+

      How reproducible: 100% of the time after rebooting if the mac changes. mac does not always change.

      Steps to Reproduce:

      1. Capture bond0 mac before reboot
2. Reboot host
3. Confirm mac change
4.  oc run --rm -it test-pod-sdn --image=registry.redhat.io/openshift4/network-tools-rhel8  --overrides='{"spec": {"tolerations": [{"operator": "Exists"}],"nodeSelector":{"kubernetes.io/hostname":"nodeb-not-rebooted"}}}' /bin/bash
5. Ping rebooted node

      Actual results:

      ping hits rebooted node but is dropped because the MAC address is of other slave interface and not the one bond is using.

      Expected results:

      OVS flows to update on all nodesafter reboot if MAC changes

      Additional info:

        If we restart NetworkManager a couple times this triggers OVS flows to get updated, not sure why. 

Possible workarounds 
 -  https://access.redhat.com/solutions/6972925
 - Statically set the mac of bond0 to one of the slave interfaces.

      Attachments

        Issue Links

          is duplicated by

          Bug - A problem that impairs or prevents the functions of the product. OCPBUGS-20353 NetworkManager fails on RHEL 8.6 worker configuring br-ex interface post reboot

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Closed
          links to

          Web Link https://github.com/ovn-org/ovn-kubernetes/pull/3984

          Web Link KB6972925: How to make bonds always use the first slave interface's MAC address in Red Hat OpenShift Container Platform 4

          GitHub openshift/ovn-kubernetes#2010: OCPBUGS-18716: [DownstreamMerge] 1-5-24

          Web Link RHEA-2024:0041 OpenShift Container Platform 4.16.z bug fix update

          Activity

            People

              trozet@redhat.com Tim Rozet
              rhn-support-rhowe Ryan Howe
              Ross Brattain Ross Brattain
              Votes:
              0 Vote for this issue
              Watchers:
              16 Start watching this issue

              Dates

                Created:
                Updated: