OpenShift opened an issue recently (OCPBUGS-18467) . The issue was traced to OVN as the culprit.

Configuration

The topology for this scenario is roughly

Server (External to OVN)
   |
Public Switch
   |
Gateway Router
   |
Internal Switch
   |
Client

Configuration details:

• The Gateway Router has an SNAT rule that changes the Client IP address to the Gateway Router's "public" IP address.
• The Gateway Router has configured the gateway_mtu on the port connected to the Internal Switch

Scenario

The client sends a UDP packet to the server. The server then responds with a UDP packet that is larger than the client's MTU. Because of the configured gateway_mtu, OVN sends an ICMP "needs fragmentation" message to the server.

The IP header on the ICMP packet has correct information, so the ICMP packet is routed to the server properly. However, the inner ICMP packet headers have the Client's IP address as the source address instead of the SNATted IP address (i.e. the Gateway Router's public IP address). Since the server's network stack does not recognize the source IP address, the ICMP message is ignored. The server continues to send packets that are too large, and OVN continues to respond with ICMP error messages that are ignored by the server. The client never receives data from the server.

Reproducer

imaximet@redhat.com has written an OVN system test that reproduces the scenario: https://github.com/igsilya/ovn/commit/b97fbb0396e53c933e0df0b60ab760ba71130fd1

Diagnosis

The issue occurs because the gateway_mtu is applied on a port that OVN developers did not expect. If the gateway_mtu were on the port connecting the Gateway Router to the Public Switch, then the scenario would likely work as expected.

In order to fix this, OVN needs to ensure that the expected IP addresses always appear in "inner" packets. Whether this applies to more than just ICMP packets, and whether this extends beyond just SNAT transformations is not clear.