-
Initiative
-
Resolution: Duplicate
-
Undefined
-
None
-
None
-
None
-
False
-
-
False
Add support to the JBenchmark system for validating official releases of Red Hat OpenShift AI (RHOAI).
The goal is to ensure that each RHOAI version is tested using automated benchmarking workflows to validate model serving performance, accuracy, and stability across key configurations.
This includes:
- Automatically triggering or manually launching benchmarks aligned with each RHOAI release
- Running tests on relevant vLLM versions, engine arguments, and hardware backends (NVIDIA, AMD, etc.)
- Capturing key performance metrics such as TTFT, throughput, and memory efficiency
- Comparing against benchmarks from previous RHOAI releases to detect regressions
- Publishing results with clear version tagging for analysis
Acceptance Criteria:
- Ability to specify RHOAI version in benchmark configuration
- Benchmark results include version metadata for filtering and comparison
- Historical deltas are calculated and flagged when regressions are detected
- Results are shared with internal consumers (e.g., PSAP, product teams)