We need to document how to scale and control which nodes the CRO can run on.

After this feature, the CRO CustomResourceDefintion is updated to include several deploymentOverrides fields in the spec. E.g.

apiVersion: operator.autoscaling.openshift.io/v1
kind: ClusterResourceOverride
metadata:
  name: cluster
spec:
  podResourceOverride:
    spec:
      memoryRequestToLimitPercent: 50
      cpuRequestToLimitPercent: 25
      limitCPUToMemoryPercent: 200
  deploymentOverrides:
    replicas: 1
    nodeSelector:
      node-role.kubernetes.io/worker: ""
    tolerations:
    - key: "key"
      operator: "Equal"
      value: "value"
      effect: "NoSchedule"

In this example, CRO pods would be able to schedule onto non-master nodes and be able to function correctly on Hypershift-like environments. The change also makes sure two or more CRO pods do not schedule on the same node similar to a DaemonSet.

Replicas (optional) - overrides the number of CRO pods that run on nodes. If empty, defaults to 2.

NodeSelector (optional) - overrides the nodeSelector of the pods. This controls which nodes pods can schedule on according to the label. If empty, defaults to 

node-role.kubernetes.io/master: ""

Tolerations (optional) - overrides the tolerations of the pods to allow pods to schedule on nodes with matching taints. If empty, defaults to

    - key: node-role.kubernetes.io/master
      operator: Exists
      effect: NoSchedule
    - key: node.kubernetes.io/unreachable
      operator: Exists
      effect: NoExecute
      tolerationSeconds: 120
    - key: node.kubernetes.io/not-ready
      operator: Exists
      effect: NoExecute
      tolerationSeconds: 120