Uploaded image for project: 'AI Platform Core Components'
  1. AI Platform Core Components
  2. AIPCC-3166

Validate RHOAI Releases via RH Benchmark

    • Icon: Initiative Initiative
    • Resolution: Duplicate
    • Icon: Undefined Undefined
    • None
    • None
    • Model Validation
    • None
    • False
    • Hide

      None

      Show
      None
    • False

      Add support to the JBenchmark system for validating official releases of Red Hat OpenShift AI (RHOAI).

       

      The goal is to ensure that each RHOAI version is tested using automated benchmarking workflows to validate model serving performance, accuracy, and stability across key configurations.

       

      This includes:

      • Automatically triggering or manually launching benchmarks aligned with each RHOAI release
      • Running tests on relevant vLLM versions, engine arguments, and hardware backends (NVIDIA, AMD, etc.)
      • Capturing key performance metrics such as TTFT, throughput, and memory efficiency
      • Comparing against benchmarks from previous RHOAI releases to detect regressions
      • Publishing results with clear version tagging for analysis 

       

      Acceptance Criteria:

      • Ability to specify RHOAI version in benchmark configuration
      • Benchmark results include version metadata for filtering and comparison
      • Historical deltas are calculated and flagged when regressions are detected
      • Results are shared with internal consumers (e.g., PSAP, product teams)

       

              rh-ee-abadli Aviran Badli (Inactive)
              rh-ee-abadli Aviran Badli (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated:
                Resolved: