-
Epic
-
Resolution: Unresolved
-
Undefined
-
None
-
QE: Add Model Evaluation to Release QE Testing
-
False
-
-
False
-
-
To Do
-
RHDHPLAN-261 - [Lightspeed] Evaluations - testing accuracy and efficacy across models
-
QE Needed
-
-
The testing scenarios should include:
- Lightspeed Core integration with Gemini API:
- Evaluation against Lightspeed core configured to integrate with the Gemini API. While Gemini's use in products requires testing/validation, the evaluation tool supports configurable API Integration.
- Lightspeed Core integration with Open Weight Models:
- Evaluation against Lightspeed core configured with at least two popular open-weight LLMs, such as Llama, Mistral, or Gemma. The evaluation framework supports assessing compatibility with other small language models. Open-weight models like Llama and Mistral are currently being investigated and considered for Lightspeed integration.
- is cloned by
-
RHIDP-9993 Doc: Developer Lightspeed evalution tool
-
- New
-