Uploaded image for project: 'Red Hat Enterprise Linux AI'
  1. Red Hat Enterprise Linux AI
  2. RHELAI-3527

Set up and initialization of RHEL AI 2.0 Components

XMLWordPrintable

    • Icon: Feature Feature
    • Resolution: Unresolved
    • Icon: Undefined Undefined
    • None
    • None
    • Engine/Runtime
    • False
    • Hide

      None

      Show
      None
    • False
    • Not Selected

      Feature Overview (mandatory - Complete while in New status)
      The RHEL AI Llama Stack server needs to be set up and initialized when a user starts the RHEL AI appliance.

      Goals (mandatory - Complete while in New status)

      • Users get a default stack set up with the RHEL AI providers configured, according to their hardware and their choice of models. Once configured, they see a list of API endpoints. 

      Requirements (mandatory -_ Complete while in Refinement status):
      A list of specific needs, capabilities, or objectives that a Feature must deliver to satisfy the Feature. Some requirements will be flagged as MVP. If an MVP gets shifted, the Feature shifts. If a non MVP requirement slips, it does not shift the feature.

      Requirement Notes isMVP?
      Sets up - the hardware config, provider & distribution configs    Yes
      Provider and distro configs will need to be readable by RHOAI    
      Initializes and sets up the default (InstructLab) stack of providers
       
        Yes
      Works for existing supported hardware configurations 
       
        Yes
      Works for models (Granite and third-party supported models)    Yes

      Use Cases - i.e. User Experience & Workflow: (Initial completion while in Refinement status):
      Include use case diagrams, main success scenarios, alternative flow scenarios.

      • Proposed flow: 
      • Step 1: Hardware detection -> Build hw config
      • Step 2: Default provider and distro configs provided, built with arguments based on hw config (i.e. with default providers and models) 
      • Step 3: Check for user selection of: student and teacher models, providers, ports, self-signed certificates
      • Step 4: Update provider and distro configs 
      • Step 5: Implicit model download (auth?) 
      • Step 6: Init RHEL AI container

      Done - Acceptance Criteria (mandatory - Complete while in Refinement status):
      Acceptance Criteria articulates and defines the value proposition - what is required to meet the goal and intent of this Feature. The Acceptance Criteria provides a detailed definition of scope and the expected outcomes - from a users point of view

      <your text here>

      Out of Scope __(Initial completion while in Refinement status):
      High-level list of items or persona’s that are out of scope.
      1. For 2.0, it is OK to force a restart if defaults are changed. It is ok to assume that this is only a server-side action.

        • However, design should be extensible to accommodate client CLI/SDK triggered changes.

      Documentation Considerations __(Initial completion while in Refinement status):
      Provide information that needs to be considered and planned so that documentation will meet customer needs. If the feature extends existing functionality, provide a link to its current documentation..

      1. Clearly document what a server CLI user will have to do and what will be available out of the box - especially consider how they will change the models they want to use as student and teacher models
      2. Document the RHEL AI port that the client CLI will use as a URL to connect to. 
      3.  

      Questions to Answer __(Initial completion while in Refinement status):
      Include a list of refinement / architectural questions that may need to be answered before coding can begin. 

      1. Are we building a container with an entry-point for some initializations vs persistent systemd? How else can we implement this? 
      2. Is vLLM in a separate container? 
      3. Yes, it’s a stand-alone container, per William 
      1. Hardware detection
      2. How interactive can initialization be? 
      3. RHEL AI Inference provider       
      4. Are each of the providers in separate containers? 
      5. vLLM and Model Management: 
      6. Where are models downloaded? 
      7. How are they served? 
      8. How are providers reconfigured/updated based on models being swapped?
      1. User needs to have an API key at this step?
      2. CLI vs SDK workflows for set up and init?

      Background and Strategic Fit (Initial completion while in Refinement status):
      Provide any additional context is needed to frame the feature.
      <your text here>

      Customer Considerations __(Initial completion while in Refinement status):
      Provide any additional customer-specific considerations that must be made when designing and delivering the Feature.
      <your text here>

      Team Sign Off (Completion while in Planning status)

      • All required Epics (known at the time) are linked to the this Feature
      • All required Stories, Tasks (known at the time) for the most immediate Epics have been created and estimated
      • Add - Reviewers name, Team Name
      • Acceptance == Feature as “Ready” - well understood and scope is clear - Acceptance Criteria (scope) is elaborated, well defined, and understood
      • Note: Only set FixVersion/s: on a Feature if the delivery team agrees they have the capacity and have committed that capability for that milestone
      Reviewed By Team Name Accepted Notes
             
             
             
             

       

              jepandit@redhat.com Jehlum Vitasta Pandit
              jepandit@redhat.com Jehlum Vitasta Pandit
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Created:
                Updated: