Description of problem:

On a regional DR environment with a RHACM hub cluster and two Openshift Virtualization managed clusters, Submariner fails to install. The hub reports:  "message": "The connection between clusters \"rdr-c2\" and \"rdr-c1\" is not established (status=connecting)"

Version-Release number of selected component (if applicable):

ocp-release:4.16.0-ec.4-x86_64

How reproducible:

100%

Steps to Reproduce:

1. Deploy Openshift ocp-release:4.16.0-ec.4-x86_64 via PSI on Openstack 

2. Deploy two Openshift Virtualization Clusters with 4.16.0-ec.4-x86_64 

3. Import the two Openshift Virtualization clusters into the RHACM hub

Install submariner.v0.17.0 via the RHACM hub >>>>> The installation reports errors:
"message": "The connection between clusters \"rdr-c2\" and \"rdr-c1\" is not established (status=connecting)"

oc logs submariner-addonxxxxxxx
 
2024-04-04T13:39:00.297Z DBG ..reswan/libreswan.go:255 libreswan Connection "submariner-cable-rdr-c1-192-168-2-82-0-0" not found in active connections obtained from whack: map[], map[] 2024-04-04T13:39:00.298Z DBG ..reswan/libreswan.go:269 libreswan Connection "submariner-cable-rdr-c1-192-168-2-82" not found in active connections obtained from whack: map[], map[] 2024/04/04 13:39:00 FATAL: sending packet: write ip4 0.0.0.0->242.0.255.254: sendto: object is remote 2024/04/04 13:39:01 FATAL: sending packet: write ip4 0.0.0.0->242.0.255.254: sendto: object is remote 2024/04/04 13:39:02 FATAL: sending packet: write ip4 0.0.0.0->242.0.255.254: sendto: object is remote 2024/04/04 13:39:03 FATAL: sending packet: write ip4 0.0.0.0->242.0.255.254: sendto: object is remote

To summarize our shared analysis after deploying environments with different versions of ocp and cnv we can say that installing Submariner:
Fails when the hub is ocp-release:4.16.0-ec.4-x86_64 and the managed clusters are CNV-v4.16.0.rhel9-1718
Fails when the hub is ocp-release:4.16.0-ec.4-x86_64 and the managed clusters are CNV 4.15.0 GA
Passes when you tested tested vmware on ocp 4.16 as managed cluster under hub on ocp 4.15.

Actual results:

The submariner installtion fails

Expected results:

The installation should pass

Additional info: