Uploaded image for project: 'RHEL'
  1. RHEL
  2. RHEL-143578

NetworkManager removes bond OVS ports from OVS bridge after reboot

Linking RHIVOS CVEs to...Migration: Automation ...Sync from "Extern...XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Major Major
    • None
    • rhel-9.7.z
    • NetworkManager
    • None
    • Low
    • rhel-net-mgmt
    • 5
    • False
    • False
    • Hide

      None

      Show
      None
    • None
    • None
    • Hide

      Definition of Done:

      Please mark each item below with ( / ) if completed or ( x ) if incomplete:

      ( ) The acceptance criteria defined below are met.

      Given a RHEL 9.7 KVM host with NetworkManager package containing commit efbfa4d27f and an OVS bridge 'br-overcloud' with:

      • One NM-managed OVS port 'ovs-port-eth0' (created via nmcli/connection profile)
      • One external OVS port 'bond0' (created via 'ovs-vsctl add-port br-overcloud bond0'),

      When the system reboots and NetworkManager completes initialization,

      Then NetworkManager does not remove the bond ovs ports and 'ovs-vsctl list-ports br-overcloud' shows both 'ovs-port-eth0' and 'bond0'

      ( ) Integration test case is available upstream.

      ( ) Code is reviewed and merged upstream.

      ( ) Preliminary testing is done.

      ( ) A demo recorded

      Show
      Definition of Done: Please mark each item below with ( / ) if completed or ( x ) if incomplete: ( ) The acceptance criteria defined below are met. Given a RHEL 9.7 KVM host with NetworkManager package containing commit efbfa4d27f and an OVS bridge 'br-overcloud' with: One NM-managed OVS port 'ovs-port-eth0' (created via nmcli/connection profile) One external OVS port 'bond0' (created via 'ovs-vsctl add-port br-overcloud bond0'), When the system reboots and NetworkManager completes initialization, Then NetworkManager does not remove the bond ovs ports and 'ovs-vsctl list-ports br-overcloud' shows both 'ovs-port-eth0' and 'bond0' ( ) Integration test case is available upstream. ( ) Code is reviewed and merged upstream. ( ) Preliminary testing is done. ( ) A demo recorded
    • None
    • None
    • Unspecified
    • Unspecified
    • Unspecified
    • x86_64
    • None

      Issue Description:

      • The customer attempted to use the updated NetworkManager package included in RHBA-2025:20948 (version 1.54.0-3.el9_7). After upgrading, NetworkManager unexpectedly removed the bond interface OVS ports from the OVS switch following a server reboot, causing the OVS configuration to break.

      Workaround:

      • Downgraded NetworkManager to 1.52.0-5.el9_6, which prevents the issue.
      • With the older version, OVS ports remain intact and the configuration functions as expected.

      Impact:

      • The issue disrupts the OVS networking setup on the KVM host, causing connectivity issues and service downtime.
      • To maintain stability, the customer must remain on the older NetworkManager version, which prevents them from benefiting from the latest fixes, improvements, and security updates.

      Environment:

      • RHEL 9.7 (kernel: 5.14.0-611.11.1.el9_7.x86_64)
      • Standalone Open vSwitch configuration
      • KVM host environment

      Package details:

      openvswitch-selinux-extra-policy-1.0-39.el9fdp.noarch            Fri  Apr  11  10:24:44  2025  1744363484  Red      Hat,
      openvswitch2.17-2.17.0-158.el9fdp.x86_64                         Fri  Apr  11  10:24:56  2025  1744363496  Red      Hat,

      NetworkManager-config-server-1.54.0-3.el9_7.noarch               Mon  Dec  8   10:17:30  2025  1765189050  Red      Hat,
      NetworkManager-libnm-1.54.0-3.el9_7.x86_64                       Fri  Dec  19  10:03:33  2025  1766138613  Red      Hat,
      NetworkManager-1.54.0-3.el9_7.x86_64                             Fri  Dec  19  10:03:33  2025  1766138613  Red      Hat,
      NetworkManager-wwan-1.54.0-3.el9_7.x86_64                        Fri  Dec  19  10:03:34  2025  1766138614  Red      Hat,
      NetworkManager-bluetooth-1.54.0-3.el9_7.x86_64                   Fri  Dec  19  10:03:34  2025  1766138614  Red      Hat,
      NetworkManager-ovs-1.54.0-3.el9_7.x86_64                         Fri  Dec  19  10:03:34  2025  1766138614  Red      Hat,
      NetworkManager-adsl-1.54.0-3.el9_7.x86_64                        Fri  Dec  19  10:03:34  2025  1766138614  Red      Hat,
      NetworkManager-team-1.54.0-3.el9_7.x86_64                        Fri  Dec  19  10:03:34  2025  1766138614  Red      Hat,
      NetworkManager-tui-1.54.0-3.el9_7.x86_64                         Fri  Dec  19  10:03:34  2025  1766138614  Red      Hat,
      NetworkManager-wifi-1.54.0-3.el9_7.x86_64                        Fri  Dec  19  10:03:34  2025  1766138614  Red      Hat,

      Expected results:

      • NetworkManager should keep OVS ports intact and preserve the existing OVS configuration.

      Actual results:

      • NetworkManager removes bond interfaces from the OVS switch after a server reboot, causing OVS networking to break.

      Logs details:

      As per NetworkManager logs, bond0 and bond1 are being removed from ovsdb.

      +++++++++++++++++

      an 20 10:25:28 bkl-mpl-kvmh02 NetworkManager[1747]: <debug> [1768904728.3123] device[6b3e3018d4b161b4] (bond1): remove_pending_action (2): 'queued-state-change-disconnected'

      Jan 20 10:25:28 bkl-mpl-kvmh02 NetworkManager[1747]: <trace> [1768904728.3128] ovsdb: obj[iface:ae8f4e05-e0a6-44ad-a2cb-9dbe64e85b47]: removed an '' interface: bond0
      Jan 20 10:25:28 bkl-mpl-kvmh02 NetworkManager[1747]: <trace> [1768904728.3128] ovsdb: obj[port:5183a278-f611-4985-9778-c0a879e4d68a]: removed a port: bond0

      Jan 20 10:25:28 bkl-mpl-kvmh02 NetworkManager[1747]: <debug> [1768904728.3152] device[b826ca65089344b3] (bond0): remove_pending_action (2): 'autoactivate'
      Jan 20 10:25:28 bkl-mpl-kvmh02 NetworkManager[1747]: <debug> [1768904728.3155] device[6b3e3018d4b161b4] (bond1): remove_pending_action (2): 'autoactivate'
      Jan 20 10:25:28 bkl-mpl-kvmh02 NetworkManager[1747]: <debug> [1768904728.3166] device[a014378e62808cd5] (bond0)[Open vSwitch Port]: remove_pending_action (0): 'autoactivate'

      Jan 20 10:25:28 bkl-mpl-kvmh02 NetworkManager[1747]: <trace> [1768904728.3171] ovsdb: obj[iface:88a202a1-163a-49d2-8b8e-a23aa8cbd660]: removed an '' interface: bond1
      Jan 20 10:25:28 bkl-mpl-kvmh02 NetworkManager[1747]: <trace> [1768904728.3171] ovsdb: obj[port:4a080a00-5016-476f-80de-31e04937fac0]: removed a port: bond1

      Jan 20 10:25:28 bkl-mpl-kvmh02 NetworkManager[1747]: <debug> [1768904728.3219] device[181a5f44cd66c721] (bond1)[Open vSwitch Port]: remove_pending_action (0): 'autoactivate'

      +++++++++++++++++

      • As per ovsswitch logs, A short time later, when the bond devices finally appear, OVS adds them, and Immediately afterward, the interfaces are removed without any administrator action:

      ++++++++++++++

      sosreport-bkl-mpl-kvmh02-04332198-2026-01-20-ezcqywl]$ cat var/log/openvswitch/ovs-vswitchd.log |grep bond

      2026-01-20T10:22:38.141Z|00157|bridge|INFO|bridge br-overcloud: deleted interface bond0 on port 1
      2026-01-20T10:22:38.142Z|00160|bridge|INFO|bridge br-mgmt: deleted interface bond1 on port 4
      2026-01-20T10:25:27.398Z|00034|bridge|WARN|could not open network device bond0 (No such device)
      2026-01-20T10:25:27.409Z|00036|bridge|WARN|could not open network device bond1 (No such device)
      2026-01-20T10:25:27.425Z|00041|bridge|WARN|could not open network device bond0 (No such device)
      2026-01-20T10:25:27.427Z|00042|bridge|WARN|could not open network device bond1 (No such device)
      2026-01-20T10:25:28.047Z|00044|bridge|WARN|could not open network device bond0 (No such device)
      2026-01-20T10:25:28.055Z|00045|bridge|WARN|could not open network device bond1 (No such device)
      2026-01-20T10:25:28.068Z|00046|bridge|WARN|could not open network device bond0 (No such device)
      2026-01-20T10:25:28.072Z|00047|bridge|WARN|could not open network device bond1 (No such device)
      2026-01-20T10:25:28.182Z|00048|bridge|WARN|could not open network device bond0 (No such device)
      2026-01-20T10:25:28.188Z|00049|bridge|WARN|could not open network device bond1 (No such device)
      2026-01-20T10:25:28.196Z|00050|bridge|WARN|could not open network device bond0 (No such device)
      2026-01-20T10:25:28.202Z|00051|bridge|WARN|could not open network device bond1 (No such device)
      2026-01-20T10:25:28.271Z|00052|bridge|INFO|bridge br-overcloud: added interface bond0 on port 1
      2026-01-20T10:25:28.277Z|00053|bridge|WARN|could not open network device bond1 (No such device)
      2026-01-20T10:25:28.289Z|00054|bridge|INFO|bridge br-mgmt: added interface bond1 on port 2
      2026-01-20T10:25:28.302Z|00056|bridge|INFO|bridge br-overcloud: deleted interface bond0 on port 1
      2026-01-20T10:25:28.494Z|00058|bridge|INFO|bridge br-mgmt: deleted interface bond1 on port 2

      ++++++++++++++

      Case Number:  04332198

      Sos report: sosreport-bkl-mpl-kvmh02-04332198-2026-01-20-ezcqywl.tar.xz

              nm-team Network Management Team
              rhn-support-bas Balakrishnan S
              Balakrishnan B
              Network Management Team Network Management Team
              Vladimir Benes Vladimir Benes
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

                Created:
                Updated: