For Private Preview customers work directly with the OpenshiftEC2NodeClass API. Creation of an OpenshiftEC2NodeClass creates a corresponding EC2NodeClass instance which is the API for the upstream Karpenter project. This allows Red Hat to combine platform-level defaults with customer configuration options defined via the OpenshiftEC2NodeClass API.

Customers are prevented from editing the EC2NodeClass directly through the use of ValidatingAdmissionPolicy (VAP). Customers can still see and inspect the EC2NodeClass instances. Additionally, customers must reference the EC2NodeClass instance when defining their Karpenter Nodepool and not the OpenshiftEC2NodeClass instance (a minor UX inconvenience).

There are additional concerns that under certain circumstances, customers may be able to work-around the VAP designed to prevent direct modification of the EC2NodeClass instance and "break" their cluster or circumvent platform defaults. 

We are proposing the following update to the Karpenter Controller behaviour to reduce the risk of customers working around platform default behaviour:

• Customers are interacting with OpenShiftEC2NodeClass. This CRD eliminates any of the dangerous fields from EC2NodeClass.
• EC2NodeClasses are supposed to only be created by the OpenShiftEC2NodeClass controller as a result of reconciling a OpenShiftEC2NodeClass object
• EC2NodeClasses are protected from addition/modification/deletion from any user that is not the the OpenShiftEC2NodeClass controller by a VAP
• In the event a VAP is modified or deleted, there is a controller watching and that will replace them in a timely fashion (seconds to minutes, but not hours+)
• In the event that a VAP is deleted, and before it can be recreated, a user creates a EC2NodeClass that doesn't have a matching OpenShiftEC2NodeClass owner, there is a controller that will delete the offending EC2NodeClass object
• In the event that a VAP is deleted, and before it can be recreated, a user modifies an existing EC2NodeClass that such that it no longer matches the expected values as we reconcile the OpenShiftEC2NodeClass that owns it, there is a controller that will revert the modifications to expected values

Other options that have been discussed:

• Offering-specific Karpenter providers e.g. ROSA HCP Karpenter provider, ARO HCP Karpenter Provider
• Proxying: https://docs.google.com/presentation/d/1AAkFblVqSAahfOqiJKP9GoL5YPNrKxhbckK_tBOD1fI/edit?slide=id.p#slide=id.p

Acceptance Criteria

• The UX for customer interaction with the Karpenter API is defined for GA
• The UX is implemented and tested and available to customers in production.