Different backend LLMs lead to different experiences with OLS. We need to provide a quantitative score about how well OLS performs with one LLM over another as a guide for the customers choosing one over the other.

The scoring system should be based on OLS functionalities or capabilities, not as a general-purpose score.

• Scoring system to describe the quality of answers when using the model
• Scoring system to evaluate and measure relevancy and correctness of answers