-
Feature
-
Resolution: Done
-
Normal
-
ACM 2.11.0
-
False
-
None
-
False
-
Not Selected
Currently to configure an operator or such to run on Infra nodes you should perform a check if there are actually infra nodes in the cluster. Since Infra nodes are generally a day-2 operation, nor do all clusters get configured with then, checking first ensures a Policy won't configure the operator to run on nodes that don't exist. A check like this in lots of Policies adds complexity.
{{- $infraCount := (len (lookup "v1" "Node" "" "" "node-role.kubernetes.io/infra").items) }}
{{- if ne $infraCount 0 }}
nodePlacement:
nodeSelector:
matchLabels:
node-role.kubernetes.io/infra: ""
tolerations:
- operator: Exists
key: node-role.kubernetes.io/infra
{{- end }}
replicas: {{ ($infraCount | default 2) | toInt }}
Since things like ODF generally recommend having dedicated nodes, most recommend those nodes get both an Infra and Storage role label. (this makes them easier to identify in the output of oc get nodes). But this adds additional complexity to the policies checking as above because now it must also check multiple labels. Across 30+ policies this can be easily missed resulting in workloads not running correctly.
{{- $infraCount := (len (lookup "v1" "Node" "" "" "node-role.kubernetes.io/infra" "!node-role.kubernetes.io/storage").items) }}
{{- if ne $infraCount 0 }}
nodePlacement:
nodeSelector:
matchLabels:
node-role.kubernetes.io/infra: ""
tolerations:
- operator: Exists
key: node-role.kubernetes.io/infra
{{- end }}
replicas: {{ ($infraCount | default 2) | toInt }}
To address this add two new template functions "getInfraNodes" and "hasInfraNodes" which checks that a node only has the Infra role and/or Worker role, but would ignore nodes with any other roles assigned. This would change the above to:
{{- if hasInfraNodes }}
nodePlacement:
nodeSelector:
matchLabels:
node-role.kubernetes.io/infra: ""
tolerations:
- operator: Exists
key: node-role.kubernetes.io/infra
{{- end }}
replicas: {{ ((len getInfraNodes.items) | default 2) | toInt }}
- is documented by
-
ACM-12798 Document the getNodesWithExactRoles and hasNodesWithExactRoles policy template functions
- Closed