For larger clusters with (4 nodes or more), a new feature has been introduced that allows users, as part of installation setup, to schedule workloads so that they run on master (control plane) nodes as well as worker nodes. Before this functionality was introduced, the workloads ran on worker nodes only and you had to wait for the installation to finish to be able to configure the scheduling of workloads on master nodes

In the UI, the feature is controlled by a new on/off toggle field called 'Run workloads on control plane nodes' on the Host Discovery page of the Assisted Installer wizard. For the API, the feature is defined using the property "schedulable_masters": true / false during cluster definition.

For smaller clusters of three or less nodes, this feature is enabled automatically and cannot be disabled. The feature is grayed out in the UI and considered undefined behavior if changed through the API.

For a more comprehensive explanation, see the last section of this description.

Note: We currently left this as one story but the writer handling it is welcome to divide it into two.

Document Impact: 

1. Generic GUI Updates -  Chapter 3: Installing with the Assisted Installer UI:

• Add the new Run workloads on control plane nodes (see images below) as step 1 of the procedure:  https://access.redhat.com/documentation/en-us/assisted_installer_for_openshift_container_platform/2022/html/assisted_installer_for_openshift_container_platform/installing-with-ui#adding-hosts-to-the-cluster_installing-with-ui 
• Refer them to the new section below.

2. New Section for both UI and API Updates:

Do one of the following:

• Create a new chapter between chapters 6 and 7, called Enabling Schedulable Masters, that has a similar format to the Configuring the Discovery Image chapter.  In this new chapter, we would have a step for both the UI where we mention the new toggle field (referring them to chapter three for more info), and a step for the API by running this command: https://gist.github.com/omertuc/daba710c3967599c3bceadeeb444b002. This option may be preferable.

• Add a new subsection in chapter 9 - Assigning Roles to Hosts. If you go with this option you need to change the chapter title to make it more comprehensive, as currently it is about user assignment of which physical host will have which role, and is an AI feature relating to the installation.  

The updates should include the following:

• This cross-reference:
  For additional information about the feature, including instructions on how to disable it, refer them to this section in the OCP documentation: https://docs.openshift.com/container-platform/4.13/nodes/nodes/nodes-nodes-managing.html#nodes-nodes-working-master-schedulable_nodes-nodes-managing
• The following warning copied from the OCP docs > Working with Nodes > Managing Nodes: 

  Important: When you configure control plane nodes from the default unschedulable to schedulable, additional subscriptions are required. This is because control plane nodes then become worker nodes.

Day 0, 1 and 2 Implications

Day 1 Implications:

• For smaller clusters, this feature is enabled automatically and cannot be disabled.
• For larger clusters, the feature is turned off by default and the user can choose to enable or disable it.

 Day 2 Implications:

• You can enable and disable the feature for clusters with more than three nodes. Refer them to this section in the OCP documentation (already included above): https://docs.openshift.com/container-platform/4.13/nodes/nodes/nodes-nodes-managing.html#nodes-nodes-working-master-schedulable_nodes-nodes-managing 
  Disabling the option for clusters with three nodes or less would render the system dysfunctional. 
• It is currently possible to add worker nodes in day 2 but adding master nodes is planned for the future as an OpenShift feature. If the feature is enabled, jobs would be automatically run on the newly added master nodes.

Affected Components:
Assisted Installer Saas

Contact People:

Dev: Omer Tuchfeld

QE: Mykhailo Chernyk

Target release:
Technical debt

Fix versions:
Assisted Installer Latest

Release Notes Needed:
No. Feature was developed a while back.

Additional resources and links:
No

Ready for Documenting?

Yes

Priority

2

Additional Information

Red Hat OpenShift is a platform that users install on their servers to help run containers on those servers. OpenShift requires a large number of its own containers that run on itself, so that it can manage itself. These containers are called the control plane.

In an ideal OpenShift cluster, out of the many servers forming the cluster, users need to dedicate 3 fairly large machines to run nothing but our control plane. We call these machines "control plane nodes" (historically "master" nodes). These 3 servers are not used for anything else and are considered overhead. Users run their own containers on the remainder of the cluster machines, which are typically much more powerful. These powerful machines are what we call "worker" nodes.

Therefore, "dedicating" 3 machines for the control plane is the default OpenShift behavior for clusters spanning many servers. There would be 3 "pretty good" machines for the control plane, and about half a dozen, dozens, hundreds or even thousands of powerful worker machines, all managed by those 3 control plane machines. Dedicating just 3 servers for the control plane when you have many more workers to manage makes sense, and is a healthy way to run a cluster. However, in a smaller system of between 5-10 servers for example, it is far more efficient to use the control plane servers for running workloads as well. 

When a user installs a SNO cluster (just one node and nothing else), or when a user installs a compact cluster (just 3 servers), the system automatically: 

• Makes all servers the control plane nodes

• Enables schedulable masters

This is why OpenShift will automatically enable schedulable masters for clusters when there's just 1 or 3 control plane nodes only (2 control plane nodes is not an option). There are additional scenarios where we also automatically enable schedulable masters, for example, when there are 3 control plane nodes and just 1 worker.