User Story:
As an OpenShift Engineer I want implement support of creating a service load balancer NLB with support of Security Group so load balancer controller has an opt-in flow to enhance the security posture and best practices.

Description:

This story focuses on implementing the functionality within the AWS Cloud Controller Manager (CCM) to allow users to optionally provision Network Load Balancers (NLBs) with associated Security Groups. Currently, the CCM does not directly support attaching Security Groups to NLBs during provisioning.

This enhancement will introduce a new Kubernetes service annotation that users can apply to their LoadBalancer type services to indicate they want the NLB to be created with a Security Group managed by CCM. This will provide a more granular and AWS-native approach to securing NLBs compared to managing security rules solely on the worker nodes.

While the AWS Load Balancer Controller (https://github.com/kubernetes-sigs/aws-load-balancer-controller) already offers this capability, the goal of this story is to add minimal, opt-in support directly to the CCM as per the user's request. This approach aims to provide the necessary functionality without requiring to perform significant changes in the OpenShift Ingress Controller, and other OpenShift components, such as installer, ROSA, etc.

Acceptance Criteria:

• A new Kubernetes service annotation, service.beta.kubernetes.io/aws-load-balancer-managed-security-group, is introduced.
• When a Kubernetes service of type LoadBalancer with the annotation service.beta.kubernetes.io/aws-load-balancer-type: nlb includes the service.beta.kubernetes.io/aws-load-balancer-managed-security-group: true  annotation, the CCM will create the Security Group, and provision the NLB associating the Security Groups during creation.
• If the service.beta.kubernetes.io/aws-load-balancer-managed-security-group annotation is not present on an NLB service, the CCM will continue with its existing default NLB provisioning behavior without associating any specific Security Groups at the NLB level.
• If the annotation service.beta.kubernetes.io/aws-load-balancer-managed-security-group is added to a service type loadBalancer with CLB (Classic load Balancer, annotation aws-load-balancer-type different than nlb ), the controller must fail, or an webhook must be added to prevent incorrect configurations
• The implementation ensures that the new functionality is strictly opt-in and does not alter the default behavior for NLB provisioning when the new annotation is absent.
• The CCM should log appropriate messages indicating whether Security Group are being associated with the NLB based on the presence of the annotation.
• The documentation for the AWS CCM in OpenShift is updated to include information about the new annotations service.beta.kubernetes.io/aws-load-balancer-managed-security-group annotation and how to use it.

Other Information:

• Related Research: The research conducted ([link to the research document/conversation history if easily available]) explored various options and concluded that introducing a new annotation for opt-in support is the most suitable approach for minimal changes.
• Potential Code Location: The primary code changes are expected in pkg/providers/v1/aws.go and pkg/providers/v1/aws_loadbalancer.go, specifically within the EnsureLoadBalancer and ensureLoadBalancerv2 functions.
• Considerations:
  • Error handling for invalid Security Group IDs provided in the annotation.
  • User experience and clarity of the annotation name.
  • Potential future enhancements, such as allowing Security Group names instead of just IDs.