-
Story
-
Resolution: Done
-
Normal
-
None
-
None
Background
Code in library-go currently uses feature gates to determine if Azure and GCP clusters should be external or not. They have been promoted for at least one release and we do not see ourselves going back.
In 4.17 the code is expected to be deleted completely.
We should remove the reliance on the feature gate from this part of the code and clean up references to feature gate access at the call sites.
Steps
- Update library go to remove reliance on feature gates
- Update callers to no longer rely on feature gate accessor (KCMO, KASO, MCO, CCMO)
- Remove feature gates from API repo
Stakeholders
- Cluster Infra
- MCO team
- Workloads team
- API server team
Definition of Done
- Feature gates for external cloud providers are removed from the product
- Docs
- <Add docs requirements for this card>
- Testing
- <Explain testing that will be added>
- is cloned by
-
-
- Closed
-
- links to
(2 links to)
[OCPCLOUD-2514] External cloud providers should not rely on feature gates
Joel Speed
added a comment - Ack, LGTM then Checked several clusters, template are as below, all install succeed. clusterversion 4.16.0-0.nightly-2024-03-13-061822
template:
ipi-on-azure/versioned-installer-private_cluster-MAG
ipi-on-azure/versioned-installer-sno
ipi-on-gcp/versioned-installer_customer_vpc-http_proxy-private-sts
ipi-on-azure/versioned-installer{}
Run regression for ccm cases and ingress critical and high cases. one case failed "OCP-45971 Author:miyadav [CCM] Implement the in-tree to out-of-tree code owner migration", raised pr to remove it from 4.16. this is related to pr https://github.com/openshift/cluster-cloud-controller-manager-operator/pull/335
/03-14 15:44:19.128 error: 1 fail, 7 pass, 6 skip (12m19s) 03-14 15:44:19.128 get result and parse it 03-14 15:44:19.128 compile result 0 03-14 15:44:19.128 can not get pipeline_type, try to get profile from ci 03-14 15:44:19.128 can not get build_version, treat it as non-arm 03-14 15:44:19.128 junit_e2e_20240314-074418.xml 03-14 15:44:19.128 03-14 15:44:19.128 03-14 15:44:19.128 03-14 15:44:19.128 03-14 15:44:19.128 The Case Execution Summary: 03-14 15:44:19.128 SKIP OCP-42879 Author:zhsun Cloud-config configmap should be copied and kept in sync within the CCCMO namespace [Disruptive] [Serial] 03-14 15:44:19.128 PASS OCP-42927 Author:zhsun [CCM] CCM should honour cluster wide proxy settings 03-14 15:44:19.128 PASS OCP-43307 Author:zhsun [CCM] cloud-controller-manager clusteroperator should be in Available state 03-14 15:44:19.128 PASS OCP-44212 Author:zhsun [CCM] The Kubelet and KCM cloud-provider should be external 03-14 15:44:19.128 FAIL OCP-45971 Author:miyadav [CCM] Implement the in-tree to out-of-tree code owner migration 03-14 15:44:19.128 PASS OCP-63829 Author:miyadav [CCM] Target workload annotation should be present in deployments of ccm 03-14 15:44:19.128 SKIP OCP-64657 Author:miyadav [CCM] Alibaba clusters are TechPreview and should not be upgradeable 03-14 15:44:19.128 SKIP OCP-70019 Author:huliu [CCM]Security Group and rules resource should be deleted when deleting a Ingress Controller 03-14 15:44:19.128 SKIP OCP-70296 Author:huliu [CCM] AWS should not use external-cloud-volume-plugin post CSI migration 03-14 15:44:19.128 PASS OCP-70618 Author:huliu [CCM] The new created nodes should be added to load balancer [Disruptive] [Serial][Slow] 03-14 15:44:19.128 PASS OCP-70620 Author:zhsun [CCM] Region and zone labels should be available on the nodes 03-14 15:44:19.128 PASS OCP-70627 Author:zhsun [CCM] Service of type LoadBalancer can be created successful [Disruptive] [Serial] 03-14 15:44:19.128 SKIP OCP-70744 Author:huliu [CCM] Pull images from ECR repository [Disruptive] [Serial] 03-14 15:44:19.128 SKIP OCP-71492 Author:zhsun [CCM] Create CLB service on aws outposts cluster [Disruptive] [Serial] 03-14 17:08:10.379 55 pass, 9 skip (1h21m57s) 03-14 17:08:10.379 get result and parse it 03-14 17:08:10.379 compile result 0 03-14 17:08:10.379 can not get pipeline_type, try to get profile from ci 03-14 17:08:10.379 can not get build_version, treat it as non-arm 03-14 17:08:10.379 junit_e2e_20240314-090758.xml 03-14 17:08:10.379 03-14 17:08:10.379 03-14 17:08:10.379 03-14 17:08:10.379 03-14 17:08:10.379 The Case Execution Summary: 03-14 17:08:10.379 PASS OCP-38674 Author:mjoseph hard-stop-after annotation can be applied globally on all ingresscontroller [Disruptive] [Serial] 03-14 17:08:10.379 PASS OCP-40821 Author:mjoseph The tune.bufsize and tune.maxwrite values can be defined per haproxy router basis 03-14 17:08:10.379 SKIP OCP-41030 Author:mjoseph preemption strategy for keepalived ipfailover [Disruptive] [Serial] 03-14 17:08:10.379 PASS OCP-41042 Author:aiyengar The Power-of-two balancing features defaults to random LB algorithm instead of leastconn for REEN/Edge/insecure routes 03-14 17:08:10.379 PASS OCP-41187 Author:aiyengar The Power of two balancing honours the per route balancing algorithm defined via haproxy.router.openshift.io/balance annotation 03-14 17:08:10.379 PASS OCP-41206 Author:aiyengar Power-of-two feature allows unsupportedConfigOverrides ingress operator option to enable leastconn balancing algorithm 03-14 17:08:10.379 PASS OCP-41929 Author:mjoseph Haproxy router continues to function normally with the service selector of exposed route gets removed/deleted 03-14 17:08:10.379 PASS OCP-43115 Author:jechen Configmap mounted on router volume after ingresscontroller has spec field HttpErrorCodePage populated with configmap name 03-14 17:08:10.379 PASS OCP-43454 Author:shudili The logEmptyRequests option only gets applied when the access logging is configured for the ingresscontroller 03-14 17:08:10.379 PASS OCP-43475 Author:aiyengar The includeSubDomainsPolicy option can be configured to be permissive with NoOpinion flag [Serial] 03-14 17:08:10.379 PASS OCP-43478 Author:aiyengar The PreloadPolicy option can be configured to be permissive with NoOpinion flag [Serial] 03-14 17:08:10.379 PASS OCP-43479 Author:aiyengar The Maxage HSTS policy strictly adheres to validation of route based based on largestMaxAge and smallestMaxAge parameter [Serial] 03-14 17:08:10.379 PASS OCP-43480 Author:aiyengar The HSTS domain policy can be configure with multiple domainPatterns options [Disruptive] [Serial] 03-14 17:08:10.379 PASS OCP-43884 Author:aiyengar lobal HSTS policy can be enforced strictly on a specific namespace using namespaceSelector for given domain pattern filtering [Serial] 03-14 17:08:10.379 PASS OCP-45399 Author:mjoseph ingress controller continue to function normally with unexpected high timeout value 03-14 17:08:10.379 PASS OCP-46183 Author:hongli DNS operator supports Random, RoundRobin and Sequential policy for servers.forwardPlugin [Disruptive] [Serial] 03-14 17:08:10.379 PASS OCP-46287 Author:mjoseph ingresscontroller supports to update maxlength for syslog message 03-14 17:08:10.379 PASS OCP-46571 Author:mjoseph Setting ROUTER_ENABLE_COMPRESSION and ROUTER_COMPRESSION_MIME in HAProxy 03-14 17:08:10.379 SKIP OCP-48138 Author:hongli Support External DNS on AWS platform 03-14 17:08:10.379 PASS OCP-48139 Author:hongli Support External DNS on Azure DNS provider 03-14 17:08:10.379 SKIP OCP-48140 Author:hongli Support External DNS on GCP DNS provider 03-14 17:08:10.379 PASS OCP-49750 Author:mjoseph After certificate rotation, ingress routers metrics endpoint will auto update certificates [Disruptive] [Serial] 03-14 17:08:10.379 PASS OCP-49802 Author:mjoseph HTTPS redirect happens even if there is a more specific http-only 03-14 17:08:10.379 PASS OCP-50074 Author:shudili Allow Ingress to be modified on the settings of livenessProbe and readinessProbe 03-14 17:08:10.379 SKIP OCP-50405 Author:shudili Multiple routers with hostnetwork endpoint strategy can be deployed on same worker node with different http/https/stat port numbers 03-14 17:08:10.379 PASS OCP-50662 Author:shudili Make ROUTER_BACKEND_CHECK_INTERVAL Configurable 03-14 17:08:10.379 SKIP OCP-50819 Author:shudili Routers with hostnetwork endpoint strategy with same http/https/stat port numbers cannot be deployed on the same worker node 03-14 17:08:10.379 PASS OCP-50926 Author:shudili Support a Configurable ROUTER_MAX_CONNECTIONS in HAproxy 03-14 17:08:10.379 PASS OCP-50928 Author:shudili Negative test of Support a Configurable ROUTER_MAX_CONNECTIONS in HAproxy 03-14 17:08:10.379 SKIP OCP-51189 Author:hongli Support aws-load-balancer-operator [Serial] 03-14 17:08:10.379 PASS OCP-51429 Author:mjoseph different router deployment with same route using subdomain 03-14 17:08:10.379 PASS OCP-51437 Author:mjoseph Router deployment using different shard with same subdomain 03-14 17:08:10.379 PASS OCP-52077 Author:mjoseph CoreDNS forwarding DNS requests over TLS with CLEAR TEXT [Disruptive] [Serial] 03-14 17:08:10.380 PASS OCP-52497 Author:mjoseph Support CoreDNS forwarding DNS requests over TLS - using system CA [Disruptive] [Serial] 03-14 17:08:10.380 PASS OCP-52738 Author:aiyengar The Power-of-two balancing features switches to source algorithm for passthrough routes 03-14 17:08:10.380 SKIP OCP-52837 Author:hongli switching of AWS CLB to NLB without deletion of ingresscontroller 03-14 17:08:10.380 PASS OCP-53605 Author:shudili Expose a Configurable Reload Interval in HAproxy 03-14 17:08:10.380 SKIP OCP-54868 Author:shudili Configurable dns Management for LoadBalancerService Ingress Controllers on AWS 03-14 17:08:10.380 PASS OCP-54998 Author:shudili Set Cookie2 by an application in a route should not kill all router pods 03-14 17:08:10.380 PASS OCP-55341 Author:mjoseph configuring list of IP address ranges using load-balancer-source-ranges annotation 03-14 17:08:10.380 PASS OCP-55367 Author:shudili Default HAProxy maxconn value to 50000 for OCP 4.12 03-14 17:08:10.380 PASS OCP-55821 Author:mjoseph Check CoreDNS default bufsize, readinessProbe path and policy 03-14 17:08:10.380 PASS OCP-55825 Author:shudili the 503 Error page should not contain license for a vulnerable release of Bootstrap 03-14 17:08:10.380 PASS OCP-55895 Author:mjoseph When canary route is not available, Ingress should be in degarded state [Disruptive] [Serial] 03-14 17:08:10.380 PASS OCP-56228 Author:shudili Deletion of default router service under the openshift ingress namespace hangs flag [Disruptive] [Serial] 03-14 17:08:10.380 PASS OCP-56240 Author:mjoseph Canary daemonset can schedule pods to both worker and infra nodes [Disruptive] [Serial] 03-14 17:08:10.380 PASS OCP-56325 Author:mjoseph DNS pod should not work on nodes with taint configured [Disruptive] [Serial] 03-14 17:08:10.380 PASS OCP-56539 Author:mjoseph Disabling internal registry should not corrupt /etc/hosts [Disruptive] [Serial] 03-14 17:08:10.380 PASS OCP-56898 Author:mjoseph Accessing the route should wake up the idled resources 03-14 17:08:10.380 PASS OCP-57001 Author:shudili edge terminated h2 (gRPC) connections need a haproxy template change to work correctly 03-14 17:08:10.380 PASS OCP-57012 Author:shudili Forwarded header includes empty quoted proto-version parameter 03-14 17:08:10.380 PASS OCP-57089 Author:shudili Error syncing load balancer and failed to parse the VMAS ID on Azure platform 03-14 17:08:10.380 PASS OCP-57370 Author:mjoseph hostname of componentRoutes should be RFC compliant 03-14 17:08:10.380 PASS OCP-60136 Author:mjoseph reencrypt route using Ingress resource for Microshift with destination CA certificate 03-14 17:08:10.380 PASS OCP-62528 Author:shudili adding/deleting http headers to an edge route by a router owner 03-14 17:08:10.380 SKIP OCP-62926 Author:shudili Ingress controller stats port is not set according to endpointPublishingStrategy 03-14 17:08:10.380 PASS OCP-63512 Author:mjoseph Enbaling force_tcp for protocolStrategy field to allow DNS queries to send on TCP to upstream server [Disruptive] [Serial] 03-14 17:08:10.380 PASS OCP-65827 Author:mjoseph allow Ingress to modify the HAProxy Log Length when using a Sidecar 03-14 17:08:10.380 PASS OCP-66560 Author:shudili adding/deleting http headers to a http route by a router owner 03-14 17:08:10.380 PASS OCP-66572 Author:shudili adding/deleting http headers to a http route by an ingress-controller as a cluster administrator 03-14 17:08:10.380 PASS OCP-66662 Author:shudili adding/deleting http headers to a reen route by a router owner 03-14 17:08:10.380 PASS OCP-67009 Author:shudili adding/deleting http headers to an edge route by an ingress-controller as a cluster administrator 03-14 17:08:10.380 PASS OCP-67010 Author:shudili adding/deleting http headers to a reen route by an ingress-controller as a cluster administrator 03-14 17:08:10.380 PASS OCP-72126 Author:hongli Support multiple cidr blocks for one NSG rule in the IngressController 03-14 16:22:50.450 38 pass, 5 skip (37m11s) 03-14 16:22:50.451 get result and parse it 03-14 16:22:50.451 compile result 0 03-14 16:22:50.451 can not get pipeline_type, try to get profile from ci 03-14 16:22:50.451 can not get build_version, treat it as non-arm 03-14 16:22:50.451 junit_e2e_20240314-082247.xml 03-14 16:22:50.451 03-14 16:22:50.451 03-14 16:22:50.451 03-14 16:22:50.451 03-14 16:22:50.451 The Case Execution Summary: 03-14 16:22:50.451 PASS OCP-40675 Author:aiyengar Ingresscontroller with endpointPublishingStrategy of hostNetwork allows PROXY protocol for source forwarding [Flaky] 03-14 16:22:50.451 PASS OCP-40677 Author:aiyengar Ingresscontroller with endpointPublishingStrategy of nodePort allows PROXY protocol for source forwarding 03-14 16:22:50.451 PASS OCP-40747 Author:mjoseph The tune.maxrewrite value can be modified with headerBufferMaxRewriteBytes parameter 03-14 16:22:50.451 PASS OCP-40748 Author:mjoseph The tune.bufsize value can be modified with headerBufferBytes parameter 03-14 16:22:50.451 SKIP OCP-41025 Author:hongli support to deploy ipfailover [Serial] 03-14 16:22:50.451 PASS OCP-41049 Author:mjoseph DNS controlls pod placement by node selector [Disruptive] [Serial] 03-14 16:22:50.451 PASS OCP-41050 Author:mjoseph DNS controll pod placement by tolerations [Disruptive] [Serial] 03-14 16:22:50.451 PASS OCP-41109 Author:hongli use IngressClass controller for ingress-to-route 03-14 16:22:50.451 PASS OCP-41110 Author:shudili The threadCount ingresscontroller parameter controls the nbthread option for the haproxy router 03-14 16:22:50.451 PASS OCP-41117 Author:hongli ingress operator manages the IngressClass for each ingresscontroller 03-14 16:22:50.451 PASS OCP-41186 Author:aiyengar The Power-of-two balancing features switches to roundrobin mode for REEN/Edge/insecure/passthrough routes with multiple backends configured with weights 03-14 16:22:50.451 PASS OCP-42276 Author:hongli enable annotation traffic-policy.network.alpha.openshift.io/local-with-fallback on LB and nodePort service 03-14 16:22:50.451 PASS OCP-43105 Author:aiyengar The tcp client/server fin and default timeout for the ingresscontroller can be modified via tuningOptions parameterss 03-14 16:22:50.451 PASS OCP-43112 Author:aiyengar timeout tunnel parameter for the haproxy pods an be modified with TuningOptions option in the ingresscontroller 03-14 16:22:50.451 PASS OCP-43113 Author:aiyengar Tcp inspect-delay for the haproxy pod can be modified via the TuningOptions parameters in the ingresscontroller 03-14 16:22:50.451 PASS OCP-43284 Author:mjoseph setting tlssecurityprofile to TLSv1.3 03-14 16:22:50.451 PASS OCP-43300 Author:hongli enable client certificate with optional policy 03-14 16:22:50.451 PASS OCP-43414 Author:aiyengar The logEmptyRequests ingresscontroller parameter set to Ignore add the dontlognull option in the haproxy configuration 03-14 16:22:50.451 PASS OCP-43416 Author:aiyengar httpEmptyRequestsPolicy ingresscontroller parameter set to ignore adds the http-ignore-probes option in the haproxy configuration 03-14 16:22:50.451 PASS OCP-43474 Author:aiyengar The includeSubDomainsPolicy parameter can configure subdomain policy to inherit the HSTS policy of parent domain [Serial] 03-14 16:22:50.451 PASS OCP-43476 Author:aiyengar The PreloadPolicy option can be set to be enforced strictly to be present or absent in HSTS preload header checks [Serial] 03-14 16:22:50.451 PASS OCP-46867 Author:shudili Configure upstream resolvers for CoreDNS flag [Disruptive] [Serial] 03-14 16:22:50.451 PASS OCP-46868 Author:shudili Configure forward policy for CoreDNS flag [Disruptive] [Serial] 03-14 16:22:50.451 PASS OCP-46872 Author:shudili Configure logLevel for CoreDNS under DNS operator flag [Disruptive] [Serial] 03-14 16:22:50.451 PASS OCP-50842 Author:mjoseph destination-ca-certificate-secret annotation for destination CA Opaque certifcate 03-14 16:22:50.451 PASS OCP-51148 Author:mjoseph host name of the route depends on the subdomain if provided 03-14 16:22:50.451 SKIP OCP-51255 Author:mjoseph cluster-ingress-operator can set AWS ELB idle Timeout on per controller basis 03-14 16:22:50.451 PASS OCP-51536 Author:mjoseph Support CoreDNS forwarding DNS requests over TLS using ForwardPlugin [Disruptive] [Serial] 03-14 16:22:50.451 PASS OCP-51946 Author:mjoseph Support CoreDNS forwarding DNS requests over TLS using UpstreamResolvers [Disruptive] [Serial] 03-14 16:22:50.451 PASS OCP-51980 Author:mjoseph destination-ca-certificate-secret annotation for destination CA TLS certifcate 03-14 16:22:50.451 PASS OCP-53696 Author:mjoseph Route status should updates accordingly when ingress routes cleaned up [Disruptive] [Serial] 03-14 16:22:50.451 PASS OCP-54042 Author:mjoseph Configuring CoreDNS caching and TTL parameters [Disruptive] [Serial] 03-14 16:22:50.451 PASS OCP-55223 Author:mjoseph Configuring list of IP address ranges using allowedSourceRanges in LoadBalancerService 03-14 16:22:50.451 SKIP OCP-59951 Author:mjoseph IngressController option to use PROXY protocol with IBM Cloud load-balancers - TCP, PROXY and omitted 03-14 16:22:50.451 PASS OCP-60149 Author:mjoseph http route using Ingress resource for Microshift 03-14 16:22:50.451 PASS OCP-60266 Author:mjoseph creation of edge and passthrough routes for Microshift 03-14 16:22:50.451 PASS OCP-60283 Author:mjoseph creation of http and re-encrypt routes for Microshift 03-14 16:22:50.451 PASS OCP-60350 Author:mjoseph Check the max number of domains in the search path list of any pod 03-14 16:22:50.451 PASS OCP-60492 Author:mjoseph Check the max number of characters in the search path of any pod 03-14 16:22:50.451 PASS OCP-62530 Author:shudili openshift ingress operator is failing to update router-certs [Serial] 03-14 16:22:50.451 SKIP OCP-64611 Author:mjoseph Ingress operator support for private hosted zones in Shared VPC clusters 03-14 16:22:50.451 PASS OCP-67093 Author:shudili Alternate Backends and Weights for a route work well 03-14 16:22:50.451 SKIP OCP-68826 Author:mjoseph External DNS support for preexisting Route53 for Shared VPC clusters
Joel Speed
added a comment - For the purpose of QE testing this, the feature gates no longer take effect, but, we cannot yet remove the feature gates due to some external release tooling changes.
I will merge the final PR at some later date, but for now, could we please test that, even on a cluster where the feature gates are disabled, the CCM is still enabled correctly.
jrouth@redhat.com This will also need to be documented. The CCM related feature gates are now enabled by default and cannot be disabled.
Joel Speed
added a comment - For the purpose of QE testing this, the feature gates no longer take effect, but, we cannot yet remove the feature gates due to some external release tooling changes.
I will merge the final PR at some later date, but for now, could we please test that, even on a cluster where the feature gates are disabled, the CCM is still enabled correctly.
jrouth@redhat.com This will also need to be documented. The CCM related feature gates are now enabled by default and cannot be disabled. 
Moved the remaining work into
OCPCLOUD-2561