Feature Overview

Goals

With the planned upcoming support in OpenShift for Kueue (RHBoK) we also propose to enable RHACM to be able to utilise Kueue constructs for a multicluster environment via MultiKueue. By adding Kueue and using the management abilities they are already familiar with and using in RHACM our customers will be able take advantage of MultiKueue deployments using existing workflows.

TLDR Goals:

1. Ability to deploy kueue as an add-on
2. Ability to create Placement that then creates multikueue configs

RHACM hub as multikueue "Manager Cluster" and RHACM managed clusters as multikueue "Worker Clusters". This allows the RHACM hub to utilise Placement to accept a MultiKueue supported job and target a ClusterQueue running in OCP (kueue via operator) or conformant Kubernetes (add-on).

PRD: https://docs.google.com/document/d/1Xm6An8SzoZp8Gw39op1665Ly_FINi5PSm8FEe-ynYME/edit?usp=sharing 

Eng planning: https://docs.google.com/document/d/1iFbzKlHRxTVXTC619ImF1Z49IMDQAAwVvyUCpAf4k5A/edit?usp=sharing

Requirements

This Section: A list of specific needs or objectives that a Feature must
deliver to satisfy the Feature.. Some requirements will be flagged as MVP.
If an MVP gets shifted, the feature shifts. If a non MVP requirement slips,
it does not shift the feature.

Use Cases

• As an admin I want to manage as much of my multicluster environment from a single pane of glass as is humanly possible.
• As an admin, I want to use OpenShift’s support for Kueue for managing workload scheduling across multiple clusters managed by RHACM.
• As an admin I want to interact seamlessly with Kueue through RHACM constructs such as Placement and ApplicationSets (and others).
• As an Admin I want to be able to create RHACM Placement decisions and have RHACM generate corresponding MultiKueueConfig and MultiKueueCluster resources.
• Additionally, RHACM should be able to update MultiKueue dynamically.
• As an Admin I want to be able to manage and monitor my Kueue resources through RHACM APIs as well as the RHACM UI so I can automate the process as well as manage it visually.
• As an admin I want to be able to take advantage of Kueue integration with RHOAI for AI workloads via RHACM.
• As an OPP admin I want to get the most value I can out of integrations provided by my extended platform product.

Questions to answer

• Where is the intersection of RHOAI, Kueue, MultiKueue, and RHACM?
• Will RHACM be responsible for configuring Kueue on its spokes?
• What if spokes are NOT OCP? We cannot rely on Kueue/Operator support so RHACM will need to have w ay to deploy
  • RHACM Add-on should be investigated

Out of Scope

• …

Background, and strategic fit

Allows easy use of Kueue functionality for multicluster via RHACM which will be available in OpenShift.

OCM: https://github.com/open-cluster-management-io/ocm/tree/main/solutions/kueue-admission-check

Assumptions

• Kueue in OCP (see dependent links)

Customer Considerations

• TBA

Documentation Considerations

• New documentation will be required.