Task Description (Required)

Create Data Science Project in RHOAI for LLM + RAG use case (Small, Medium, Large setup)

https://issues.redhat.com/browse/RHOAISTRAT-139

What is RAG? https://research.ibm.com/blog/retrieval-augmented-generation-RAG 

A RAG NLP recipe is available here https://github.com/containers/ai-lab-recipes/tree/main/recipes/natural_language_processing/rag 

Seems similar to a Chatbot template like needing a LLM like vLLM or llama.cpp.

The only thing that maybe different and require more research is we need to build and deploy the Vector Database and require a BGE model https://huggingface.co/BAAI/bge-base-en-v1.5 for converting documents between natural language and vector representations.

We have couple of options on what to use for a Vector Database like ChromaDB or Milvus. We have to figure out which one is better for our Template.

If this requires Change Management, complete sections below: 

Change Request 

<Select which item is being changed>

[ ]  Add New Tokens

[ ]  Rotate Tokens

[ ]  Remove Tokens

[ ] Others: (specify)

  Environment

<Select which environment the change is being made on.  If both, open a separate issue so changes are tracked in each environment>

[ ]  Stage OR

[ ]  Prod

  Backout Plan

<State what steps are needed to roll back in case something goes wrong>

  Downtime

<Is there any downtime for these changes?  If so, for how long>

  Risk Level

<How risky is this change?>

  Testing

<How are changes verified?>

  Communication

<How are service owners or consumers notified of these changes?>