Description of problem:

Version-Release number of selected component (if applicable):

ACM registry:2.8.0-DOWNSTREAM-2023-05-30-03-43-50
Submariner 0.15

How reproducible:

Steps to Reproduce:

1. When issue reported in https://issues.redhat.com/browse/ACM-5779 was hit, it was expected that restarting the gateway pods on C1 would restore connectivity.
But another issue related to submariner connectivity was observed which is being reported in this bug.

Actual results: Race condition in GN pod while processing the endpoint notifications

Related logs shared by Sridhar- http://pastebin.test.redhat.com/1101858

Expected results: This should be avoided to have seemless submariner connectivity

Additional info:

The issue is being discussed here- https://redhat-internal.slack.com/archives/C0134E73VH6/p1685713251676249