Some e2e tests are failing with the job "gcp-custom-dns" (was "gcp-user-provisioned-dns") for featuregate "GCPClusterHostedDNSInstall" which is promoted to GA in 4.20.

For more details about "GCPClusterHostedDNSInstall" please refer to https://github.com/openshift/enhancements/pull/1468/files#diff-230cf161d0e8b0dc075e6760516b1171448adae3e6ee937ac9a97a0ce44862ddR106-R120

In short, the http2 and grpc tests use dedicated ingresscontrollers, and gateway also has separated LB and dnsrecord, so the default wildcard created by the new static CoreDNS won't work for those tests.


And another issue is: grpc/http2 tests use ingressDomain to generate shardFQDN so the shard ingresscontroller domain looks like:

 "domain: e2e-test-router-http2-vtzjw.apps.hongli-gcp20.qe.gcp.devcluster.openshift.com"

If the dnsrecords of shard cannot be published, the "*.e2e-test-router-http2-vtzjw.apps.<baseDomain>" is resolved to "*.apps.<baseDomain>", so the request goes to default router and causes unexpected result.


Here are the failing tests:
[sig-network-edge][Conformance][Area:Networking][Feature:Router][apigroup:route.openshift.io][apigroup:config.openshift.io]
 The HAProxy router should pass the http2 tests 
[apigroup:image.openshift.io][apigroup:operator.openshift.io] 
[Suite:openshift/conformance/parallel/minimal] 
            
[sig-network-edge][Conformance][Area:Networking][Feature:Router] The 
HAProxy router should pass the gRPC interoperability tests 
[apigroup:route.openshift.io][apigroup:operator.openshift.io] 
[Suite:openshift/conformance/parallel/minimal] 

[sig-network-edge][OCPFeatureGate:GatewayAPIController][Feature:Router][apigroup:gateway.networking.k8s.io]
 Ensure LB, service, and dnsRecord are created for a Gateway object 
[Suite:openshift/conformance/parallel]             
              : 
[sig-network-edge][OCPFeatureGate:GatewayAPIController][Feature:Router][apigroup:gateway.networking.k8s.io]
 Ensure HTTPRoute object is created 
[Suite:openshift/conformance/parallel] 

    4.20

    100%

    1. run e2e test with job "e2e-gcp-custom-dns" (was "e2e-gcp-user-provisioned-dns")
    2. 
    3. 
    

    tests fail

    1. the sig-network-edge tests should pass on "gcp-custom-dns" job
    2. shardFQDN should use oc.Namespace() + "." + BaseDomain

    example job link: https://prow.ci.openshift.org/view/gs/test-platform-results/pr-logs/pull/openshift_installer/9792/pull-ci-openshift-installer-main-e2e-gcp-custom-dns/1957526619325206528 

    code of shardFQDN: https://github.com/openshift/origin/blob/main/test/extended/router/http2.go#L257