-
Bug
-
Resolution: Unresolved
-
Major
-
None
-
rhos-18.0.z
-
None
-
False
-
-
False
-
?
-
rhos-connectivity-nfv
-
None
-
-
-
-
-
Bugs Pending Verification
-
1
-
Important
To Reproduce Steps to reproduce the behavior:
- Deploy RHOSO18 with ifcfg network method for EDPM nodes and NTP servers are assigned for EDPM nodes
- Reboot the EDPM nodes.
- chrony is not synced with the assigned server.
Expected behavior
- chrony synced with the assigned NTP servers.
Bug impact
- After system rebooted, EDPM node is out of sync with NTP servers and time difference happens with OCP nodes or the rest of EDPM nodes.
Known workaround
- Run chronyc onoffline after reboot
- Disable NetworkManager
Additional context
- If I understand correctly, ifcfg method in os-net-config is still supported and the older method used customer in RHOSO18 are existing.
- This behavior happens the following reasons.
When ifcfg method is used, NetworkManager(NM) is still enabled. In ifcfg-XXX configuratin, ifcfg-lo file is a single device file which is not managed by os-net-config and it doesn't have NM_CONTROLLED=no. Due to that, lo device is managed by NM. As a result NM tries to up lo device then it sends connectivity-change event.Nov 18 05:53:27 edpm-compute-0.rhoso.example.com chronyd[1007]: chronyd version 4.5 starting (+CMDMON +NTP +REFCLOCK +RTC +PRIVDROP +SCFILTER +SIGND +ASYNCDNS +NTS +SECHASH +IPV6 +DEBUG) Nov 18 05:53:27 edpm-compute-0.rhoso.example.com chronyd[1007]: Frequency -13.775 +/- 0.018 ppm read from /var/lib/chrony/drift : Nov 18 05:53:29 edpm-compute-0.rhoso.example.com NetworkManager[1186]: <debug> [1763445209.4225] monotonic timestamp started counting 1.479607127 seconds ago with an offset of 8.0 seconds to CLOCK_BOOTTIME (local time is 2025-11-18 05:53:29) Nov 18 05:53:29 edpm-compute-0.rhoso.example.com NetworkManager[1186]: <info> [1763445209.4229] NetworkManager (version 1.46.0-33.el9_4) is starting... (boot:db1c4e38-58ff-44b7-b14d-3088bf2e9931) Nov 18 05:53:29 edpm-compute-0.rhoso.example.com NetworkManager[1186]: <info> [1763445209.4229] Read config: /etc/NetworkManager/NetworkManager.conf (etc: 95-nm-debug.conf, 99-cloud-init.conf) Nov 18 05:53:29 edpm-compute-0.rhoso.example.com NetworkManager[1186]: <debug> [1763445209.4229] CONFIG: config-data[0x5646b85db020]: 2 groups Nov 18 05:53:29 edpm-compute-0.rhoso.example.com NetworkManager[1186]: <debug> [1763445209.4229] CONFIG: Nov 18 05:53:29 edpm-compute-0.rhoso.example.com NetworkManager[1186]: <debug> [1763445209.4229] CONFIG: [main] Nov 18 05:53:29 edpm-compute-0.rhoso.example.com NetworkManager[1186]: <debug> [1763445209.4230] CONFIG: dns=none Nov 18 05:53:29 edpm-compute-0.rhoso.example.com NetworkManager[1186]: <debug> [1763445209.4230] CONFIG: rc-manager=unmanaged Nov 18 05:53:29 edpm-compute-0.rhoso.example.com NetworkManager[1186]: <debug> [1763445209.4230] CONFIG: no-auto-default=* Nov 18 05:53:29 edpm-compute-0.rhoso.example.com NetworkManager[1186]: <debug> [1763445209.4230] CONFIG: configure-and-quit=no Nov 18 05:53:29 edpm-compute-0.rhoso.example.com NetworkManager[1186]: <debug> [1763445209.4230] CONFIG: Nov 18 05:53:29 edpm-compute-0.rhoso.example.com NetworkManager[1186]: <debug> [1763445209.4230] CONFIG: [logging] Nov 18 05:53:29 edpm-compute-0.rhoso.example.com NetworkManager[1186]: <debug> [1763445209.4230] CONFIG: domains=ALL:TRACE Nov 18 05:53:29 edpm-compute-0.rhoso.example.com NetworkManager[1186]: <debug> [1763445209.4230] CONFIG: Nov 18 05:53:29 edpm-compute-0.rhoso.example.com NetworkManager[1186]: <debug> [1763445209.4230] CONFIG: # no-auto-default file "/var/lib/NetworkManager/no-auto-default.state" Nov 18 05:53:29 edpm-compute-0.rhoso.example.com NetworkManager[1186]: <debug> [1763445209.4232] config: state: successfully read state file "/var/lib/NetworkManager/NetworkManager.state" Nov 18 05:53:29 edpm-compute-0.rhoso.example.com NetworkManager[1186]: <debug> [1763445209.4232] WEXT support is disabled Nov 18 05:53:29 edpm-compute-0.rhoso.example.com NetworkManager[1186]: <debug> [1763445209.4237] create NMDBusManager singleton (67bfecf95168a207) : Nov 18 05:53:29 edpm-compute-0.rhoso.example.com NetworkManager[1186]: <debug> [1763445209.6263] device[d0821320cb7af9be] (lo): connectivity state changed from NONE to LIMITED Nov 18 05:53:29 edpm-compute-0.rhoso.example.com NetworkManager[1186]: <debug> [1763445209.6264] device[d0821320cb7af9be] (lo): connectivity state changed from NONE to LIMITED Nov 18 05:53:29 edpm-compute-0.rhoso.example.com NetworkManager[1186]: <debug> [1763445209.6264] device[d0821320cb7af9be] (lo): remove_pending_action (1): 'in-state-change' Nov 18 05:53:29 edpm-compute-0.rhoso.example.com NetworkManager[1186]: <debug> [1763445209.6265] device[d0821320cb7af9be] (lo): remove_pending_action (0): 'queued-state-change-activated' Nov 18 05:53:29 edpm-compute-0.rhoso.example.com NetworkManager[1186]: <info> [1763445209.6265] manager: startup complete Nov 18 05:53:29 edpm-compute-0.rhoso.example.com NetworkManager[1186]: <trace> [1763445209.6265] settings: [timestamps-keyfile]: prune keyfile of old entries: "/var/lib/NetworkManager/timestamps" Nov 18 05:53:29 edpm-compute-0.rhoso.example.com nm-dispatcher[1193]: req:4 'up' [lo]: new request (4 scripts) Nov 18 05:53:29 edpm-compute-0.rhoso.example.com NetworkManager[1186]: <trace> [1763445209.6285] settings: [timestamps-keyfile]: write keyfile: "/var/lib/NetworkManager/timestamps" Nov 18 05:53:29 edpm-compute-0.rhoso.example.com nm-dispatcher[1193]: req:4 'up' [lo]: environment: CONNECTION_DBUS_PATH=/org/freedesktop/NetworkManager/Settings/1 Nov 18 05:53:29 edpm-compute-0.rhoso.example.com NetworkManager[1186]: <debug> [1763445209.6285] pacrunner: pacrunner not on D-Bus Nov 18 05:53:29 edpm-compute-0.rhoso.example.com nm-dispatcher[1193]: req:4 'up' [lo]: environment: CONNECTION_EXTERNAL=1 Nov 18 05:53:29 edpm-compute-0.rhoso.example.com NetworkManager[1186]: <debug> [1763445209.6285] pacrunner: try D-Bus activating pacrunner... Nov 18 05:53:29 edpm-compute-0.rhoso.example.com nm-dispatcher[1193]: req:4 'up' [lo]: environment: CONNECTION_FILENAME=/run/NetworkManager/system-connections/lo.nmconnection Nov 18 05:53:29 edpm-compute-0.rhoso.example.com nm-dispatcher[1193]: req:4 'up' [lo]: environment: CONNECTION_UUID=bdf13464-cf19-48ae-8152-656ef877f6c5 Nov 18 05:53:29 edpm-compute-0.rhoso.example.com nm-dispatcher[1193]: req:4 'up' [lo]: environment: CONNECTION_ID=lo Nov 18 05:53:29 edpm-compute-0.rhoso.example.com nm-dispatcher[1193]: req:4 'up' [lo]: environment: DEVICE_IFACE=lo Nov 18 05:53:29 edpm-compute-0.rhoso.example.com nm-dispatcher[1193]: req:4 'up' [lo]: environment: DEVICE_IP_IFACE=lo Nov 18 05:53:29 edpm-compute-0.rhoso.example.com nm-dispatcher[1193]: req:4 'up' [lo]: environment: IP4_NUM_ROUTES=0 Nov 18 05:53:29 edpm-compute-0.rhoso.example.com nm-dispatcher[1193]: req:4 'up' [lo]: environment: PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin Nov 18 05:53:29 edpm-compute-0.rhoso.example.com nm-dispatcher[1193]: req:4 'up' [lo]: environment: NM_DISPATCHER_ACTION=up Nov 18 05:53:29 edpm-compute-0.rhoso.example.com nm-dispatcher[1193]: req:1 'hostname', "/usr/lib/NetworkManager/dispatcher.d/11-dhclient": complete: process succeeded Nov 18 05:53:29 edpm-compute-0.rhoso.example.com nm-dispatcher[1193]: req:1 'hostname', "/usr/lib/NetworkManager/dispatcher.d/20-chrony-dhcp": run script Nov 18 05:53:29 edpm-compute-0.rhoso.example.com systemd[1]: Finished Network Manager Wait Online. Nov 18 05:53:29 edpm-compute-0.rhoso.example.com nm-dispatcher[1193]: req:1 'hostname', "/usr/lib/NetworkManager/dispatcher.d/20-chrony-dhcp": complete: process succeeded Nov 18 05:53:29 edpm-compute-0.rhoso.example.com nm-dispatcher[1193]: req:1 'hostname', "/usr/lib/NetworkManager/dispatcher.d/20-chrony-onoffline": run script Nov 18 05:53:29 edpm-compute-0.rhoso.example.com systemd[1]: Starting LSB: Bring up/down networking... Nov 18 05:53:29 edpm-compute-0.rhoso.example.com nm-dispatcher[1193]: req:1 'hostname', "/usr/lib/NetworkManager/dispatcher.d/20-chrony-onoffline": complete: process succeeded Nov 18 05:53:29 edpm-compute-0.rhoso.example.com nm-dispatcher[1193]: req:1 'hostname': completed (4 scripts) Nov 18 05:53:29 edpm-compute-0.rhoso.example.com nm-dispatcher[1193]: req:2 'connectivity-change': start running ordered scripts... Nov 18 05:53:29 edpm-compute-0.rhoso.example.com nm-dispatcher[1193]: req:2 'connectivity-change', "/usr/lib/NetworkManager/dispatcher.d/04-iscsi": run script Nov 18 05:53:29 edpm-compute-0.rhoso.example.com NetworkManager[1186]: <debug> [1763445209.6386] dispatcher: (1) succeeded (after 0.076 sec, 4 scripts invoked) Nov 18 05:53:29 edpm-compute-0.rhoso.example.com NetworkManager[1186]: <debug> [1763445209.6387] dispatcher: (1) /usr/lib/NetworkManager/dispatcher.d/04-iscsi succeeded Nov 18 05:53:29 edpm-compute-0.rhoso.example.com NetworkManager[1186]: <debug> [1763445209.6387] dispatcher: (1) /usr/lib/NetworkManager/dispatcher.d/11-dhclient succeeded Nov 18 05:53:29 edpm-compute-0.rhoso.example.com NetworkManager[1186]: <debug> [1763445209.6387] dispatcher: (1) /usr/lib/NetworkManager/dispatcher.d/20-chrony-dhcp succeeded Nov 18 05:53:29 edpm-compute-0.rhoso.example.com NetworkManager[1186]: <debug> [1763445209.6387] dispatcher: (1) /usr/lib/NetworkManager/dispatcher.d/20-chrony-onoffline succeeded Nov 18 05:53:29 edpm-compute-0.rhoso.example.com nm-dispatcher[1193]: req:2 'connectivity-change', "/usr/lib/NetworkManager/dispatcher.d/04-iscsi": complete: process succeeded Nov 18 05:53:29 edpm-compute-0.rhoso.example.com nm-dispatcher[1193]: req:2 'connectivity-change', "/usr/lib/NetworkManager/dispatcher.d/11-dhclient": run script Nov 18 05:53:29 edpm-compute-0.rhoso.example.com nm-dispatcher[1193]: req:2 'connectivity-change', "/usr/lib/NetworkManager/dispatcher.d/11-dhclient": complete: process succeeded Nov 18 05:53:29 edpm-compute-0.rhoso.example.com nm-dispatcher[1193]: req:2 'connectivity-change', "/usr/lib/NetworkManager/dispatcher.d/20-chrony-dhcp": run script Nov 18 05:53:29 edpm-compute-0.rhoso.example.com nm-dispatcher[1193]: req:2 'connectivity-change', "/usr/lib/NetworkManager/dispatcher.d/20-chrony-dhcp": complete: process succeeded Nov 18 05:53:29 edpm-compute-0.rhoso.example.com nm-dispatcher[1193]: req:2 'connectivity-change', "/usr/lib/NetworkManager/dispatcher.d/20-chrony-onoffline": run script Nov 18 05:53:29 edpm-compute-0.rhoso.example.com chronyd[1007]: Source 192.168.122.1 offline Nov 18 05:53:29 edpm-compute-0.rhoso.example.com network[1205]: WARN : [network] You are using 'network' service provided by 'network-scripts', which are now deprecated.
connectivity-change events triggers NM dispatcher script and the script runs /usr/lib/NetworkManager/dispatcher.d/20-chrony-onoffline
#!/usr/bin/sh # This is a NetworkManager dispatcher / networkd-dispatcher script for # chronyd to set its NTP sources online or offline when a network interface # is configured or removed export LC_ALL=C chronyc=/usr/bin/chronyc # For NetworkManager consider only selected events if [ $# -ge 2 ]; then case "$2" in up|down|connectivity-change) ;; dhcp4-change|dhcp6-change) # Actions "up" and "connectivity-change" in some cases do not # guarantee that the interface has a route (e.g. a bond). # dhcp(x)-change handles at least cases that use DHCP. ;; *) exit 0;; esac fi # Note: for networkd-dispatcher routable.d ~= on and off.d ~= off $chronyc onoffline > /dev/null 2>&1 exit 0
By running chronyc onoffline and the script isn't run again, the chronyc keeps sources are offline.
To prevent this behavior, we should disable NetworkManager for ifcfg method.