Uploaded image for project: 'Red Hat Internal Developer Platform'
  1. Red Hat Internal Developer Platform
  2. RHIDP-11529

Run lightspeed eval against developer lightspeed 1.9

XMLWordPrintable

    • Run lightspeed eval against developer lightspeed 1.9
    • False
    • Hide

      None

      Show
      None
    • False
    • RHDHPLAN-930Lightspeed Evaluation Data Update and Public Consumption
    • In Progress
    • RHDHPLAN-930 - Lightspeed Evaluation Data Update and Public Consumption
    • QE Needed, Docs Needed, TE Needed, Customer Facing, PX Needed
    • 50% To Do, 50% In Progress, 0% Done

      EPIC Goal

      with https://issues.redhat.com/browse/RHDHPLAN-261 being done, we consider the 1.8 Eval result as a dry run. in the 1.10 timeframe, we will run the dataset generate & lightspeed evaluation against Developer Lightspeed 1.9 RAG.

      Acceptance Criteria

      500+ single-run Q&A dataset need to be generated against RHDH 1.9 docs

      run the evaluation with 2-3 large/medium models, 2 small models

              yangcao Stephanie Cao
              yangcao Stephanie Cao
              RHDH AI
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated: