-
Feature
-
Resolution: Unresolved
-
Undefined
-
None
-
None
-
False
-
-
False
-
Not Selected
Feature Overview (mandatory - Complete while in New status)
The RHEL AI Llama Stack server needs to be set up and initialized when a user starts the RHEL AI appliance.
Goals (mandatory - Complete while in New status)
- Users get a default stack set up with the RHEL AI providers configured, according to their hardware and their choice of models. Once configured, they see a list of API endpoints.
Requirements (mandatory -_ Complete while in Refinement status):
A list of specific needs, capabilities, or objectives that a Feature must deliver to satisfy the Feature. Some requirements will be flagged as MVP. If an MVP gets shifted, the Feature shifts. If a non MVP requirement slips, it does not shift the feature.
| Requirement | Notes | isMVP? |
|---|---|---|
| Sets up - the hardware config, provider & distribution configs | Yes | |
| Provider and distro configs will need to be readable by RHOAI | ||
| Initializes and sets up the default (InstructLab) stack of providers |
Yes | |
| Works for existing supported hardware configurations |
Yes | |
| Works for models (Granite and third-party supported models) | Yes |
Use Cases - i.e. User Experience & Workflow: (Initial completion while in Refinement status):
Include use case diagrams, main success scenarios, alternative flow scenarios.
- Proposed flow:
- Step 1: Hardware detection -> Build hw config
- Step 2: Default provider and distro configs provided, built with arguments based on hw config (i.e. with default providers and models)
- Step 3: Check for user selection of: student and teacher models, providers, ports, self-signed certificates
- Step 4: Update provider and distro configs
- Step 5: Implicit model download (auth?)
- Step 6: Init RHEL AI container
Done - Acceptance Criteria (mandatory - Complete while in Refinement status):
Acceptance Criteria articulates and defines the value proposition - what is required to meet the goal and intent of this Feature. The Acceptance Criteria provides a detailed definition of scope and the expected outcomes - from a users point of view
…
<your text here>
Out of Scope __(Initial completion while in Refinement status):
High-level list of items or persona’s that are out of scope.
1. For 2.0, it is OK to force a restart if defaults are changed. It is ok to assume that this is only a server-side action.
-
- However, design should be extensible to accommodate client CLI/SDK triggered changes.
Documentation Considerations __(Initial completion while in Refinement status):
Provide information that needs to be considered and planned so that documentation will meet customer needs. If the feature extends existing functionality, provide a link to its current documentation..
- Clearly document what a server CLI user will have to do and what will be available out of the box - especially consider how they will change the models they want to use as student and teacher models
- Document the RHEL AI port that the client CLI will use as a URL to connect to.
Questions to Answer __(Initial completion while in Refinement status):
Include a list of refinement / architectural questions that may need to be answered before coding can begin.
- Are we building a container with an entry-point for some initializations vs persistent systemd? How else can we implement this?
- Is vLLM in a separate container?
- Yes, it’s a stand-alone container, per William
- Hardware detection
- How interactive can initialization be?
- RHEL AI Inference provider
- Are each of the providers in separate containers?
- vLLM and Model Management:
- Where are models downloaded?
- How are they served?
- How are providers reconfigured/updated based on models being swapped?
- User needs to have an API key at this step?
- CLI vs SDK workflows for set up and init?
Background and Strategic Fit (Initial completion while in Refinement status):
Provide any additional context is needed to frame the feature.
<your text here>
Customer Considerations __(Initial completion while in Refinement status):
Provide any additional customer-specific considerations that must be made when designing and delivering the Feature.
<your text here>
Team Sign Off (Completion while in Planning status)
- All required Epics (known at the time) are linked to the this Feature
- All required Stories, Tasks (known at the time) for the most immediate Epics have been created and estimated
- Add - Reviewers name, Team Name
- Acceptance == Feature as “Ready” - well understood and scope is clear - Acceptance Criteria (scope) is elaborated, well defined, and understood
- Note: Only set FixVersion/s: on a Feature if the delivery team agrees they have the capacity and have committed that capability for that milestone
| Reviewed By | Team Name | Accepted | Notes |
- …
- depends on
-
RHELAI-3617 Model management
-
- New
-
- relates to
-
RHELAI-3630 llama-stack-client CLI user - ability to change models
-
- New
-