Support Network Endpoint Groups on GCP

User Story

As a developer I want Network Endpoint Groups (NEGs) supported on GCP so that problems with Instance Group-backed load balancers are avoided.

<Describes high level purpose and goal for this story. Answers the questions: Who is impacted, what is it and why do we need it?>

Background

OpenShift on GCP currently uses Instance Groups to place nodes behind a load balancer. This works, but has led to issues that have been difficult to trace and debug. Examples include OCPBUGS-26583 and OCPBUGS-5755.

Additionally, GCP only allows instances to participate in one instance group; OpenShift currently assumes instances can participate in multiple instances groups to get different load balancer groupings.

Our fix for OCPBUGS-5755 was discussed upstream, and is unlikely to be accepted. Upstream maintainers have instead suggested using network endpoint groups (NEGs).

Instances are able to be members of multiple NEGs for load balancer grouping on GCP.

Given that OpenShift installation is moving to using Cluster API instead of Terraform (CORS-3196), implementing support for NEGs in the Terraform-based installer is not considered a viable solution at this time. Instead, the Cluster API Provider for GCP (CAPG) will be used to create cloud resources.

<Describes the context or background related to this story>

Steps

• Determine if CAPG supports creating load balancers with NEGs. Code for this is at https://github.com/kubernetes-sigs/cluster-api-provider-gcp/tree/main/cloud/services/compute/loadbalancers
• If not, provide an NEG-based implementation that OpenShift can consume.

Stakeholders

• OpenShift developers
• OpenShift end users seeking to use multiple load balancers within GCP.

Definition of Done

• CAPI-based OpenShift installations create NEG-based load balancers rather than instance group-based ones.

• Docs

• This should, mostly likely, be considered an implementation detail that most customers will not be concerned with.

• Testing

• Unit and integration tests in the upstream project
• Validation that Kubernetes `Service` load balancers created with a CAPI-based OpenShift installer

  cloud.google.com/neg

  annotation

• gcloud compute network-endpoint-groups list

  for the GCP Project associated with the OpenShift cluster returns entries.