When static routes are applied to worker nodes using a NodeNetworkConfigurationPolicy(NNCP),a Multus secondary network interface may disappear from running Pods.

In the observed case, the affected interface is an ipvlan-based Multus secondary interface, where the Pod interface (eth3) uses a host interface (enp4s0.6) as its master. After the NNCP is applied, the secondary interface is removed from the Pod network namespace. Restarting the Pod recreates the interface and restores connectivity.

According to the journal logs on the worker node, when static routes are configured, NetworkManager seems to deactivate and then reactivate the master interfaces enp4s0.1, enp4s0.2, enp4s0.3, and enp4s0.6.

However for enp4s0.4, enp4s0.5, enp4s0.7, and enp4s0.8, NetworkManager seems to update the corresponding connections, but does not perform a device deactivate/activate.

It is likely that the temporary deactivation and reactivation of the master interface causes the ipvlan-based Multus interface to be removed from the Pod. Since Multus does not perform live reconciliation for existing Pods, the secondary interface is not automatically recreated until the Pod is restarted.

kubernetes-nmstate-operator.4.14.0-202406051038    

Not always

  1. Create a Pod with a Multus secondary network interface of type ipvlan, using a host interface (for example, enp4s0.6) as the master.     2. Add static routes on the existed NNCP.
  3. Observe that the Multus secondary interface disappears from the running Pod.
  4. Restart the Pod and confirm that the secondary interface is recreated and connectivity is restored.

When static routes are applied using NNCP, the Pod’s Multus secondary interface disappears.

Applying static routes using NNCP should not affect the Pod’s Multus secondary interface.