Uploaded image for project: 'Performance and Scale for AI Platforms'
  1. Performance and Scale for AI Platforms
  2. PSAP-1582

Standardize LLM inference benchmarking practices across team

XMLWordPrintable

    • Icon: Story Story
    • Resolution: Obsolete
    • Icon: Undefined Undefined
    • Jan 13
    • None
    • None
    • None

      iUser Story:
      As a PSAP engineer working on LLM inference performance I want to standardize on:

      • Set of models we are focused on evaluating
      • How we present the results (shared visualization tooling)
      • What metrics we look at and how we measure them. For example, 95th vs 99th percentile ITL. 

      So that we can compare the different performance results we have gathered across the team

      Acceptance criteria:

              akamra8979 Ashish Kamra
              ccamacho@redhat.com Carlos Camacho
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Created:
                Updated:
                Resolved: