EPIC Goal

The evaluation of the Lightspeed plugin is essential to understand the accuracy of the responses of the Lightspeed plugin as well as provide a way for us to understand the effectiveness of the models being used. With the evaluation framework in 1.9 (https://issues.redhat.com/browse/RHDHPLAN-261), we want the user to be able to reproduce the evaluation using the our provided data as well as view the evaluation data that we have produced in each release. These information will help the user to select the model of their choice when running the RHDH Lightspeed.

Dependencies (internal and external)

https://issues.redhat.com/browse/RHIDP-11529 needs to be done

Acceptance Criteria

Documentation: Provides the documentation to help users to understand:
1. the evaluation framework
2. instructions on how to download, set up, and run the RHDH Lightspeed evaluation tools.
3. instructions on how to download the Q&A data set for each release
4. link to the reports on the evaluation result report on each RHDH release
5. include information on how to interpret the results and the information on the evaluation result categories.

Allow off-release cycle data release: The documentation should provide link to the centralized location where user can find the Q&A data set and evaluation result report for each release. This allows those data can be published post release after the RAG content has been updated.