Uploaded image for project: 'Red Hat Internal Developer Platform'
  1. Red Hat Internal Developer Platform
  2. RHIDP-10740

implement gitops template using vLLM for the AI templates

    • Icon: Task Task
    • Resolution: Done
    • Icon: Major Major
    • None
    • None
    • None
    • HAS Sprint 2264

      Task Description (Required)

       the implementation of gitops template in ai-lab-apps

      to support both vllm and llama.cpp

       

      the vllm image to use is: quay.io/rh-aiservices-bu/vllm-openai-ubi9:0.4.2

       

      The resources definitions should still be general for both chatbot and codegen.

      use condition for vllm and llama.cpp specific configuration

       

      If this requires Change Management, complete sections below: 

      Change Request 

       

      <Select which item is being changed>

       

      [ ]  Add New Tokens

      [ ]  Rotate Tokens

      [ ]  Remove Tokens

      [ ] Others: (specify)

       

        Environment

      <Select which environment the change is being made on.  If both, open a separate issue so changes are tracked in each environment>

       

      [ ]  Stage OR

      [ ]  Prod

       

        Backout Plan

      <State what steps are needed to roll back in case something goes wrong>

       

        Downtime

      <Is there any downtime for these changes?  If so, for how long>

       

        Risk Level

      <How risky is this change?>

       

        Testing

      <How are changes verified?>

       

        Communication

      <How are service owners or consumers notified of these changes?>

              yangcao Stephanie Cao
              eyuen@redhat.com Elson Yuen
              RHIDP - AI
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Created:
                Updated:
                Resolved: