Description of problem:

Submariner connectivity tests fail with OCP 4.18.

Version-Release number of selected component (if applicable):

ACM 2.13
Submariner 0.20.0

How reproducible: Always

Steps to Reproduce:

1. Install Submariner 0.20.0 with ACM 2.13 with at least one managedcluster on OCP 4.18
2. Run `subctl verify --only connectivity`
3. ...

Actual results:

Connectivity tests fail with following error:
Basic TCP connectivity tests across overlapping clusters without discovery when a pod connects via TCP to the globalIP of a remote service in reverse direction when the pod is not on a gateway and the remote service is not on a gateway should have sent the expected data from the pod to the other pod [dataplane, globalnet]
github.com/submariner-io/submariner@v0.20.0-rc1/test/e2e/dataplane/tcp_gn_pod_connectivity.go:53
Feb 12 09:57:21.429: Creating namespace objects with basename "dataplane-gn-conn-nd"
Feb 12 09:57:21.696: Generated namespace "e2e-tests-dataplane-gn-conn-nd-6kbzc" in cluster "vthapar-acmhub" to execute the tests in
Feb 12 09:57:21.696: Creating namespace "e2e-tests-dataplane-gn-conn-nd-6kbzc" in cluster "vthapar-red"
Feb 12 09:57:22.221: Creating a listener pod in cluster "vthapar-acmhub", which will wait for a handshake over TCP
Feb 12 09:57:24.543: Pointing a ClusterIP service to the listener pod in cluster "vthapar-acmhub"
Feb 12 09:57:25.367: Creating a connector pod in cluster "vthapar-red", which will attempt the specific UUID handshake over TCP
Feb 12 09:57:28.197: Feb 12 09:57:28.197: INFO: ExecWithOptions &

{Command:[sh -c for j in $(seq 1 50); do echo [dataplane] connector says 56598388-aa40-473b-a8c8-4da095696681; done | for i in $(seq 2); do if nc -v 242.0.255.253 1234 -w 60; then break; else sleep 30; fi; done] Namespace:e2e-tests-dataplane-gn-conn-nd-6kbzc PodName:custombzf7v ContainerName:connector-pod Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:true}

Feb 12 09:57:30.121: Connector pod is scheduled on node "ip-10-1-20-100.us-east-2.compute.internal"
Feb 12 09:57:30.121: Waiting for the listener pod "tcp-check-listener6sqr8" on node "ip-10-0-47-42.us-east-2.compute.internal" to exit, returning what listener sent
Feb 12 09:57:31.385: Feb 12 09:57:31.385: INFO: Pod "tcp-check-listener6sqr8" on node "ip-10-0-47-42.us-east-2.compute.internal" output:
Ncat: Version 7.92 ( https://nmap.org/ncat )
Ncat: Listening on 0.0.0.0:1234
Ncat: Connection from 10.131.2.2.
Ncat: Connection from 10.131.2.2:40398.
[dataplane] connector says 56598388-aa40-473b-a8c8-4da095696681

Feb 12 09:57:31.650: Verifying that the listener got the connector's data and the connector got the listener's data
Feb 12 09:57:31.650: Verifying the output of the listener pod contains a cluster-scoped global IP [242.1.0.1 242.1.0.2 242.1.0.3 242.1.0.4 242.1.0.5 242.1.0.6 242.1.0.7 242.1.0.8] of the connector Pod
[FAILED] in [It] - github.com/submariner-io/submariner@v0.20.0-rc1/test/e2e/framework/dataplane.go:201 @ 02/12/25 09:57:31.651
Feb 12 09:57:31.651: Deleting namespace "e2e-tests-dataplane-gn-conn-nd-6kbzc" on cluster "vthapar-acmhub"
Feb 12 09:57:31.918: Deleting namespace "e2e-tests-dataplane-gn-conn-nd-6kbzc" on cluster "vthapar-red"
• [FAILED] [10.750 seconds]
Basic TCP connectivity tests across overlapping clusters without discovery when a pod connects via TCP to the globalIP of a remote service in reverse direction when the pod is not on a gateway and the remote service is not on a gateway [It] should have sent the expected data from the pod to the other pod [dataplane, globalnet]
github.com/submariner-io/submariner@v0.20.0-rc1/test/e2e/dataplane/tcp_gn_pod_connectivity.go:53

[FAILED] Expected
<string>: Ncat: Version 7.92 ( https://nmap.org/ncat )
Ncat: Listening on 0.0.0.0:1234
Ncat: Connection from 10.131.2.2.
Ncat: Connection from 10.131.2.2:40398.
[dataplane] connector says 56598388-aa40-473b-a8c8-4da095696681
[dataplane] connector says 56598388-aa40-473b-a8c8-4da095696681
[dataplane] connector says 56598388-aa40-473b-a8c8-4da095696681
[dataplane] connector says 56598388-aa40-473b-a8c8-4da095696681
[dataplane] connector says 56598388-aa40-473b-a8c8-4da095696681
[dataplane] connector says 56598388-aa40-473b-a8c8-4da095696681
[dataplane] connector says 56598388-aa40-473b-a8c8-4da095696681
[dataplane] connector says 56598388-aa40-473b-a8c8-4da095696681
[dataplane] connector says 56598388-aa40-473b-a8c8-4da095696681
[dataplane] connector says 56598388-aa40-473b-a8c8-4da095696681
[dataplane] connector says 56598388-aa40-473b-a8c8-4da095696681
[dataplane] connector says 56598388-aa40-473b-a8c8-4da095696681
[dataplane] connector says 56598388-aa40-473b-a8c8-4da095696681
[dataplane] connector says 56598388-aa40-473b-a8c8-4da095696681
[dataplane] connector says 56598388-aa40-473b-a8c8-4da095696681
[dataplane] connector says 56598388-aa40-473b-a8c8-4da095696681
[dataplane] connector says 56598388-aa40-473b-a8c8-4da095696681
[dataplane] connector says 56598388-aa40-473b-a8c8-4da095696681
[dataplane] connector says 56598388-aa40-473b-a8c8-4da095696681
[dataplane] connector says 56598388-aa40-473b-a8c8-4da095696681
[dataplane] connector says 56598388-aa40-473b-a8c8-4da095696681
[dataplane] connector says 56598388-aa40-473b-a8c8-4da095696681
[dataplane] connector says 56598388-aa40-473b-a8c8-4da095696681
[dataplane] connector says 56598388-aa40-473b-a8c8-4da095696681
[dataplane] connector says 56598388-aa40-473b-a8c8-4da095696681
[dataplane] connector says 56598388-aa40-473b-a8c8-4da095696681
[dataplane] connector says 56598388-aa40-473b-a8c8-4da095696681
[dataplane] connector says 56598388-aa40-473b-a8c8-4da095696681
[dataplane] connector says 56598388-aa40-473b-a8c8-4da095696681
[dataplane] connector says 56598388-aa40-473b-a8c8-4da095696681
[dataplane] connector says 56598388-aa40-473b-a8c8-4da095696681
[dataplane] connector says 56598388-aa40-473b-a8c8-4da095696681
[dataplane] connector says 56598388-aa40-473b-a8c8-4da095696681
[dataplane] connector says 56598388-aa40-473b-a8c8-4da095696681
[dataplane] connector says 56598388-aa40-473b-a8c8-4da095696681
[dataplane] connector says 56598388-aa40-473b-a8c8-4da095696681
[dataplane] connector says 56598388-aa40-473b-a8c8-4da095696681
[dataplane] connector says 56598388-aa40-473b-a8c8-4da095696681
[dataplane] connector says 56598388-aa40-473b-a8c8-4da095696681
[dataplane] connector says 56598388-aa40-473b-a8c8-4da095696681
[dataplane] connector says 56598388-aa40-473b-a8c8-4da095696681
[dataplane] connector says 56598388-aa40-473b-a8c8-4da095696681
[dataplane] connector says 56598388-aa40-473b-a8c8-4da095696681
[dataplane] connector says 56598388-aa40-473b-a8c8-4da095696681
[dataplane] connector says 56598388-aa40-473b-a8c8-4da095696681
[dataplane] connector says 56598388-aa40-473b-a8c8-4da095696681
[dataplane] connector says 56598388-aa40-473b-a8c8-4da095696681
[dataplane] connector says 56598388-aa40-473b-a8c8-4da095696681
[dataplane] connector says 56598388-aa40-473b-a8c8-4da095696681
[dataplane] connector says 56598388-aa40-473b-a8c8-4da095696681

To satisfy at least one of these matchers: [%!s(*matchers.OrMatcher=&

{[0xc0008b7d70 0xc0008b7da0] <nil>}

) %!s(*matchers.ContainSubstringMatcher=&

{242.1.0.8 []}

)]
In [It] at: github.com/submariner-io/submariner@v0.20.0-rc1/test/e2e/framework/dataplane.go:201 @ 02/12/25 09:57:31.651
------------------------------
SSSSSSSSSSSSSSSSSSSSSSSSSS

Summarizing 1 Failure:
[FAIL] Basic TCP connectivity tests across overlapping clusters without discovery when a pod connects via TCP to the globalIP of a remote service in reverse direction when the pod is not on a gateway and the remote service is not on a gateway [It] should have sent the expected data from the pod to the other pod [dataplane, globalnet]
github.com/submariner-io/submariner@v0.20.0-rc1/test/e2e/framework/dataplane.go:201

Ran 12 of 48 Specs in 153.982 seconds

Expected results:

Test should pass

Additional info:

Issue is with SNAT on gateway node of destination cluster. Instead of PodIP/PodGlobalnetIP it shows the IP of destination GW CNI interface.