-
Task
-
Resolution: Done
-
Undefined
-
None
-
None
-
None
-
None
-
5
-
False
-
None
-
False
-
Yes
-
MGDSRVS-346 - Autoscaling of OpenShift Streams Data Plane
-
---
-
---
-
MK - Sprint 220
WHAT
The Control Plane needs to know the maximum number of nodes configured for each supported instance so that it can create appropriate machine pools with the configuration needed by the autoscaler. This information has to be communicated to the fleetshard so that it can report back the maximum units that be handled in the configured nodes.
The control plane will obtain this configuration via a dynamic scaling ConfigMap (set via the Parameter in the service template). The ConfigMap needs to contain the max nodes limits for each instance type. There are some considerations to take:
- Consider the expansion of Kafka. It is quite possible that in the future we might want to allow customer to change the size of their kafka from x1 to x2 i.e expand their current kafka. The setting of this max value has to take this into account so that we do not set it to the max physcial OSD node limit (known value is
200180) - Consider OSD limit as indicated above
Once the final value known for each machine pool, we need to store this in the database so that we can easily send it to the fleetshard. Also, any future change of the configuration won't afect already configured clusters (as no reconciliation of autoscaling configuration is planned for the moment)
The configuration may look like
```
—
maximum_node_limits:
standard:
limit: 50
developer:
limit: 30
```
WHY
This is need to:
- configure the machine pool for each supported instance type
- to communicate the value to the Fleetshard
DONE
Include the following where applicable:
- Configuration exposed
- Splitting logic applied when the cluster supports more instances (Staging)
- The resulting value after the split is stored in the database for each supported instance type
- blocks
-
MGDSTRM-8741 Communicate maxNodes info to the Fleetshard via the ManagedKafkaAgentCR for each supported instance type
- Closed