Our managed services offerings e.g ROSA HCP have a maximum Node limit on the size of clusters that customers can provision. For example: At time of writing, the maximum number of Nodes allowed with a ROSA HCP cluster is 500.

The maximum Node limits for clusters are adjusted periodically over-time, typically in response to performance and scale testing, and new developments in OCP. 

We require a mechanism that will allow us to configure Karpenter to respect the maximum number of Nodes that can be provisioned in a cluster. This is to ensure that Karpenter does not scale the cluster beyond what is supported by our managed services.

When calculating Node limits, Karpenter must take into account all Nodes in the cluster, not just the Nodes it has provisioned. For example, this includes:

• Nodes provisioned by managed NodePools
• Nodes provisioned by Cluster Autoscaler
• Nodes that Karpenter has provisioned

Acceptance Criteria

• Via Hypershift API, Managed Services can configure the maximum total number of Nodes that can be provisioned in a cluster
• The limit can be adjusted over-time in response to increased cluster size limits
• Karpenter will respect this total and not scale the cluster beyond this limit
• Karpenter will include all nodes when performing calculations and not just nodes it has provisioned