Story (Required)

As a developer using the Software Templates, I would like a little bit more context between the various model servers like llama.cpp and vLLM. 

Since I am new to using LLMs, I have no clue what model servers I need to use and what either of llama.cpp or vLLM mean or does. 

Background (Required)

As someone who has not used any LLMs before, they would probably not know what either of llama.cpp or a vLLM model server even mean. We should give more contextual help for users regarding these model servers.

Think about the models too. We can specify some of the model descriptions from the Hugging Face pages.

Out of scope

<Defines what is not included in this story>

Approach (Required)

Give a bit more context for the model servers and models that are available in the dropdown or the text field, so new users who have not used any LLMs before know what they are going to use.

We can scrape some of the description from these model servers and models Github or Hugging Face pages.

Dependencies

<Describes what this story depends on. Dependent Stories and EPICs should be linked to the story.>

Acceptance Criteria (Required)

<Describe edge cases to consider when implementing the story and defining tests>

<Provides a required and minimum list of acceptance tests for this story. More is expected as the engineer implements this story>

documentation updates (design docs, release notes etc)
demo needed
SOP required
education module update (Filled by DEVHAS team only)
R&D label required (Filled by DEVHAS team only)

Done Checklist

Code is completed, reviewed, documented and checked in
Unit and integration test automation have been delivered and running cleanly in continuous integration/staging/canary environment
Continuous Delivery pipeline(s) is able to proceed with new code included
Customer facing documentation, API docs, design docs etc. are produced/updated, reviewed and published
Acceptance criteria are met
If the Grafana dashboard is updated, ensure the corresponding SOP is updated as well