-
Task
-
Resolution: Done
-
Major
-
None
-
None
-
None
Task Description (Required)
the implementation of gitops template in ai-lab-apps
to support both vllm and llama.cpp
the vllm image to use is: quay.io/rh-aiservices-bu/vllm-openai-ubi9:0.4.2
The resources definitions should still be general for both chatbot and codegen.
use condition for vllm and llama.cpp specific configuration
If this requires Change Management, complete sections below:
Change Request
<Select which item is being changed>
[ ] Add New Tokens
[ ] Rotate Tokens
[ ] Remove Tokens
[ ] Others: (specify)
Environment
<Select which environment the change is being made on. If both, open a separate issue so changes are tracked in each environment>
[ ] Stage OR
[ ] Prod
Backout Plan
<State what steps are needed to roll back in case something goes wrong>
Downtime
<Is there any downtime for these changes? If so, for how long>
Risk Level
<How risky is this change?>
Testing
<How are changes verified?>
Communication
<How are service owners or consumers notified of these changes?>
- clones
-
RHIDP-10605 spike: Investigate using vLLM for the AI templates
-
- Closed
-
- is cloned by
-
RHIDP-10523 implement software template using vLLM for the AI templates
-
- Closed
-
-
RHIDP-10621 update software template to triger a pipeline build as last step
-
- Closed
-