-
Bug
-
Resolution: Unresolved
-
Normal
-
None
-
None
-
None
-
False
-
-
False
-
-
Section "1.2.4.3. Benchmark evaluation" of RHEL AI 1.3 Release notes:
https://docs.redhat.com/en/documentation/red_hat_enterprise_linux_ai/1.3/html-single/release_notes/index#benchmark-evaluation
The second sentence creates confusion about what's being evaluated:
"Red Hat Enterprise Linux AI includes the ability to run benchmark evaluations on the newly trained models. On your trained model, you can evaluate how well the model knows the model you added with the MMLU_BRANCH benchmark. For more details on benchmark evaluation, see Evaluating your new model."
The illogical part is in the second sentence: "On your trained model, you can evaluate how well the model knows the model you added". It sounds somewhat circular and implies model does "know" itself. It might misrepresent what MMLU benchmarks actually test.
Something like this could bring more clarity:
"On your trained model, you can evaluate how well the model knows the knowledge/skills you added"