Uploaded image for project: 'RHEL'
  1. RHEL
  2. RHEL-77167

[balance-slb] br-ex fails to activate after patch-ex-to-phy activation

    • NetworkManager-1.51.90-1.el9
    • No
    • Critical
    • ZStream
    • 1
    • rhel-sst-network-management
    • ssg_networking
    • 8
    • Hide

      [2025-02-17] 

      [2025-02-11] MR merged. Doing the builds.

      [2025-02-11] A MR trying to fix this issue is available upstream for review. It is updating the NM autoconnect-ports mechanism for OVS interface to make sure that during startup, only OVS-interface connections that are ready, are activated. 

      [2025-02-05] The issue affects OpenShift OVN-K deployments using OVS with a balance-slb bond, where br-ex intermittently fails to obtain a DHCP lease after multiple reboots and therefore, leading to complete network loss. First analysis of the logs confirms that br-ex enters the "prepare" state and waits for the controller to be ready but never reattempts activation - causing it to miss its DHCP configuration. The root cause is still under investigation.

      Show
      [2025-02-17]   RHEL-77167 - The 9.4.z stream containing this fix will ship today through this errata: https://errata.engineering.redhat.com/advisory/144737   [2025-02-11] MR merged. Doing the builds. [2025-02-11] A MR trying to fix this issue is available upstream for review. It is updating the NM autoconnect-ports mechanism for OVS interface to make sure that during startup, only OVS-interface connections that are ready, are activated.  [2025-02-05] The issue affects OpenShift OVN-K deployments using OVS with a balance-slb bond, where br-ex intermittently fails to obtain a DHCP lease after multiple reboots and therefore, leading to complete network loss. First analysis of the logs confirms that br-ex enters the "prepare" state and waits for the controller to be ready but never reattempts activation - causing it to miss its DHCP configuration. The root cause is still under investigation.
    • False
    • Hide

      None

      Show
      None
    • None
    • NMT Sprint ending 2025-03-20
    • Approved Blocker
    • Hide

      Given a system administrator is configured an OpenShift OVN-K deployment using OVS with a balance-slb bond,

      When the system is rebooted, 

      Then, the patch-ex-to-phy and patch-phy-to-ex ports must successfully establish connectivity between br-ex and br-phy after each reboot and the system should maintain full network connectivity.

      Definition of Done:

      • The implementation meets the acceptance criteria
      • Integration tests are written and pass 
      • The code is part of a downstream build attached to an errata
      Show
      Given a system administrator is configured an OpenShift OVN-K deployment using OVS with a balance-slb bond, When the system is rebooted,  Then, the patch-ex-to-phy and patch-phy-to-ex ports must successfully establish connectivity between br-ex and br-phy after each reboot and the system should maintain full network connectivity. Definition of Done: The implementation meets the acceptance criteria Integration tests are written and pass  The code is part of a downstream build attached to an errata
    • Pass
    • None
    • None

      What were you trying to do that didn't work?

       
      Balance-slb bond connected to br-phy and br-ex for OpenShift OVN-K

      (eno2, eno3) <-> OVS br-phy (balance-slb) <-> OVS patch <-> OVS br-ex
      

      After rebooting 61 times br-ex fails to activate, system has no network.

      What is the impact of this issue to you?

      No network.

      Please provide the package NVR for which the bug is seen:

      NetworkManager-1.46.0-21.el9_4.x86_64

      How reproducible is this bug?:

      Intermittent, failed after 61 reboots.

      Steps to reproduce

      nmstate

      ovn:
        bridge-mappings:
          - localnet: localnet-network
            bridge: br-ex
            state: present
      interfaces:
        - name: br-ex
          type: ovs-bridge
          state: up
          ipv4:
            enabled: false
            dhcp: false
          ipv6:
            enabled: false
            dhcp: false
          bridge:
            allow-extra-patch-ports: true
            port:
              - name: br-ex
              - name: patch-ex-to-phy
          ovs-db:
            external_ids:
              bridge-uplink: "patch-ex-to-phy"
        - name: br-ex
          type: ovs-interface
          state: up
          copy-mac-from: eno2
          ipv4:
            enabled: true
            dhcp: true
            auto-route-metric: 48
          ipv6:
            enabled: false
            dhcp: false
        - name: br-phy
          type: ovs-bridge
          state: up
          ipv4:
            enabled: false
            dhcp: false
          ipv6:
            enabled: false
            dhcp: false
          bridge:
            allow-extra-patch-ports: true
            port:
              - name: patch-phy-to-ex
              - name: ovs-bond
                link-aggregation:
                  mode: balance-slb
                  port:
                    - name: eno2
                    - name: eno3
        - name: patch-ex-to-phy
          type: ovs-interface
          state: up
          patch:
            peer: patch-phy-to-ex
        - name: patch-phy-to-ex
          type: ovs-interface
          state: up
          patch:
            peer: patch-ex-to-phy
        - name: eno1
          type: ethernet
          state: up
          ipv4:
            enabled: false
          ipv6:
            enabled: false
        - name: eno2
          type: interface
          state: up
          ipv4:
            enabled: false
          ipv6:
            enabled: false
        - name: eno3
          type: interface
          state: up
          ipv4:
            enabled: false
          ipv6:
            enabled: false
        - name: eno4
          type: interface
          state: up
          ipv4:
            enabled: false
          ipv6:
            enabled: false
        - name: ens2f0
          type: interface
          state: down
          ipv4:
            enabled: false
          ipv6:
            enabled: false
        - name: ens2f1
          type: interface
          state: down
          ipv4:
            enabled: false
          ipv6:
            enabled: false
      
      
      

      Expected results

      br-ex activates, system has networking.

      Actual results

      br-ex fails to come up, system has no networking.

              ferferna Fernando Fernandez Mancera
              rbrattai@redhat.com Ross Brattain
              Ross Brattain
              Network Management Team Network Management Team
              Filip Pokryvka Filip Pokryvka
              Votes:
              0 Vote for this issue
              Watchers:
              15 Start watching this issue

                Created:
                Updated: