This is to spike on the new scope to support IngressController subnet selection at installation time so we can cover the ask coming from RFE-4738

Goal:

• Create AWS resources validating the LB and scenario of deploying two subnets in same AZ and scheme to isolate LB from subnets than nodes without disrupting the routing operation and LB health check states

Description:

RFE-4738 requests to install API's LBs to a different subnets from worker nodes for IP restrictions/regulations. Currently the API's LBs are created by installer. The EP 1634 proposes to introduce "roles" to compose granular deployments when adding subnets, BYO VPC/unmanaged.

The scope of this spike is to create a cluster to validate the requested scenario prior implementation of proposed in EP, so we can get confidence of any nuance that would exists in that environment.

To reproduce that in the existin codebase, prior EP, it is required to install using UPI-method (create network, LBs, and nodes). Currently the approach of perform UPI installs is using CloudFormation template.

Requirements:

•  create VPC and dependencies, duplicating subnets in existing zones, two subnets in the same zone
• validate the nuances of deployment, routing limitations, and health check states is transictioning as expected

Nice to have: