Current status and considerations:

Following the solution https://access.redhat.com/solutions/5876771, the custome define several static routes to reach common services (e.g.: DNS servers) and apply them to the Master nodes [1] of first OCP clusters deployed in 4.12 (if not in a more old version).

The UPI OCP clusters node were based on SDN newtorking with IPv4 address and default gateway statically assigned (no DHCP) and static routes attached to the exiting network interface: ens192.

In OCP 4.16, migrating from SDN (ip address assigned to ens192) to OVN (ip address moved to br-ex) the static routes have been lost because not more applicable to the ens192 because this interface was changed to unumbered.

$ oc patch Network.operator.openshift.io cluster --type=merge \
>   --patch '{
>     "spec":{
>       "defaultNetwork":{
>         "ovnKubernetesConfig":{
>           "gatewayConfig":

}}}}'

In this customer context based on a service used to add static routes and defined to work in an SDN environment, 
simply updating the network interface name in MachineConfig [1] (from ens192 to br-ex) is insufficient, as the nmcli-dev-modify.service initiates prematurely, primarily when the br-ex interface is not yet fully active [2].

   ~~~
   Reading applied connection from device 'br-ex' (/org/freedesktop/NetworkManager/Devices/34) failed: Device is not activated
   ~~~ 

Some workarounds could be evaluated and used to address this configuration, but it is unclear whether there could be any side effects behind adopting these approaches:

1) MachineConfig with nmcli-dev-modify.service based on "nmcli dev modify br-ex"

As a result of the evaluation of the thread below (thread used to change the ipv4.never-default on br-ex):
https://redhat-internal.slack.com/archives/CQXKTD8FM/p1740044505173659?thread_ts=1740043759.673619&cid=CQXKTD8FM 
the system service could be patched as described below.

    systemd:
      units:
        # This section defines the systemd service.
        - name: nmcli-dev-modify.service
          enabled: true 
          contents: |
            [Unit]
            Description=Set custom IPv4 routes 
            Wants=network-online.target
            After=network-online.target ignition-firstboot-complete.service ovs-configuration.service
            Before=kubelet.service crio.service
            [Service]
            Type=oneshot
            RemainAfterExit=true
            ExecStart=nmcli dev modify br-ex +ipv4.routes "26.2.11.89/32 26.0.164.1,26.2.11.93/32 26.0.164.1,26.0.131.51/32 26.0.164.1,26.2.197.125/32 26.0.164.1,26.3.17.112/32 26.0.164.1,26.2.11.5/32 26.0.164.1,26.2.124.189/32 26.0.164.1,26.2.218.245/32 26.0.164.1,26.3.147.113/32 26.0.164.1,26.0.167.119/32 26.0.164.1,26.3.147.112/32 26.0.164.1,26.0.141.200/32 26.0.164.1,26.0.157.105/32 26.0.164.1,26.3.147.111/32 26.0.164.1,26.0.132.74/32 26.0.164.1,26.0.41.61/32 26.0.164.1,26.2.169.158/32 26.0.164.1,26.2.169.159/32 26.0.164.1"
            [Install]
            WantedBy=kubelet.service

Could this adjustment be sufficient to ensure a smooth upgrade of the OCP nodes where the rpm-ostree rebase requires access to the DNS service through static routes ?

2) MachineConfig based on ip route add
Even if retired, the solution https://access.redhat.com/solutions/7073996 could be potentially used to reach the same goal of adding static routes to br-ex as in the example below:

   variant: openshift
   version: 4.13.0
   metadata:
     name: 89-worker-dispatcher
     labels:
       machineconfiguration.openshift.io/role: worker 
   storage:
     files:
     - path: /etc/NetworkManager/dispatcher.d/br-ex-up-add-route.sh
       mode: 0744 
       overwrite: true
       contents: 
         inline: |
           #!/usr/bin/bash
           if [ "$1" == "br-ex" ] && { [ "$2" == "up" ] || [ "$2" == "reapply" ]; }; then
             ip route add 26.2.11.89/32 via 26.0.164.1 dev br-ex 
             ip route add 26.2.11.93/32 via 26.0.164.1 dev br-ex 
             ip route add 26.0.131.51/32 via 26.0.164.1 dev br-ex 
             .... # othe static routes to be added
           fi

But also in this case it isn't clear if this adjustment will be sufficient to ensure a smooth upgrade of the OCP nodes where the rpm-ostree rebase requires access to the DNS service through static routes.

OCPBUG scope:

Identify the appropriate approach and syntax (e.g., using the Wants, After, Before, and WantedBy arguments of the service invoked via MachineConfig) to correctly configure static routes on an OpenShift Container Platform 4.16+ cluster based on OVN networking, ensuring proper operation during day two management.