-
Epic
-
Resolution: Unresolved
-
Undefined
-
None
-
None
-
RAG Eval Implementation
-
False
-
-
False
-
Not Selected
-
To Do
-
RHELAI-2397 - [eval] Downstream RAGAS as RAG Evaluation framework
-
-
100% To Do, 0% In Progress, 0% Done
Goal:
We want to implement the ability for InstructLab/RHEL AI to have the capacity to evaluate how well our supported models are performing in RAG settings.
Acceptance Criteria:
The acceptance criteria is a working implementation of a RAG evaluation suite which can be readily used within RHEL AI pipeline through which a model can be passed along with any other necessary values, and a result is returned indicating the model's performance.
Open questions:
- How well does Ragas perform against the existing evaluation notebook?
