-
Epic
-
Resolution: Unresolved
-
Critical
-
None
-
None
-
None
-
AWS Private Link Association of Subnets discovered by Karpenter
-
None
-
-
100% To Do, 0% In Progress, 0% Done
-
False
-
-
False
-
None
-
None
-
None
TL;DR The AWS Private Link that establishes connectivity between the worker nodes deployed in the hosted cluster's VPC subnets and the API Server hosted in the ROSA service account does not add or remove the discovered subnets where Karpenter deploys the worker nodes.
The default EC2NodeClass uses cluster ID as Karpenter subnet discovery Tag. Only subnets that are already part of CAPI machine pool created through OCM are associated with the AWS Private Link VPC Endpoint (VPCE). Those subnets in the VPC (especially from a different AZ where there is no existing OCM Machine pools) are not associated with the VPC Endpoint. These ENIs placed as part of subnet association are also life cycled as part of life cycling machine pool; this leads to situations where Karpenter managed worker nodes are deployed on subnets/AZ where there's no Private Link.
The life cycle of Karpenter managed worker nodes needs to happen independent of CAPI-owned node pools life cycled through OCM. Because:
1) AWS Private Link HA: CAPI node pools are limited to single subnet (AZ). A hosted cluster that has single CAPI node pool (OCM Machine pool) will have AWS Private Link with single AZ. However, Karpenter EC2NodeClass can allow node creation in additional subnets (AZs) in the VPC, to increase the worker node HA. This will lead to AWS Private Link with subnet associated with CAPI node pool subnet and not with subnets associated with Karpenter-managed node pools. [AWS SLA for AWS Private Link: https://aws.amazon.com/privatelink/sla/]
2) Cross-AZ Traffic: In a cluster wherein there are Karpenter-managed worker nodes deployed in 2 or more AZs, the traffic between worker nodes and api-server happen via ENIs from fewer subnets/AZs, requiring cross-AZ traffic and thus leads to EC2 networking costs. [AWS EC2 Pricing for intra-region transfers: https://aws.amazon.com/ec2/pricing/on-demand/]
3) Zero Worker Nodes: OCM Machine pool and the ENIs placed in the subnets where these machine pools are deployed may not exist. OCM allows deleting machine pools for use cases like changing the EC2 instance type during which the subnet association and the ENIs are removed. ROSA HCP will offer zero worker nodes.
Acceptance Criteria:
1. Network connectivity managed by HyperShift is managed for Karpenter discovered worker nodes independent of CAPI managed node pools.
- is cloned by
-
CNTRLPLANE-1816 Worker Node Defaults & Validations per Service Provider
-
- New
-