• QE: Add Model Evaluation to Release QE Testing
    • False
    • Hide

      None

      Show
      None
    • False
    • RHDHPLAN-261[Lightspeed] Evaluations - testing accuracy and efficacy across models
    • To Do
    • RHDHPLAN-261 - [Lightspeed] Evaluations - testing accuracy and efficacy across models
    • QE Needed

      The testing scenarios should include:

      1. Lightspeed Core integration with Gemini API:
        • Evaluation against Lightspeed core configured to integrate with the Gemini API. While Gemini's use in products requires testing/validation, the evaluation tool supports configurable API Integration.
      2. Lightspeed Core integration with Open Weight Models:
        • Evaluation against Lightspeed core configured with at least two popular open-weight LLMs, such as Llama, Mistral, or Gemma. The evaluation framework supports assessing compatibility with other small language models. Open-weight models like Llama and Mistral are currently being investigated and considered for Lightspeed integration.

              rh-ee-hushaik Husne Shabbir Shaik
              yangcao Stephanie Cao
              RHIDP - Frontend Plugins & UI
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Created:
                Updated: