Uploaded image for project: 'Performance and Scale for AI Platforms'
  1. Performance and Scale for AI Platforms
  2. PSAP-1606

Try Nvidia's Genai-perf

XMLWordPrintable

    • Icon: Story Story
    • Resolution: Done
    • Icon: Normal Normal
    • Jan 13
    • None
    • None
    • None
    • Product / Portfolio Work
    • Hide

      Review on usability of genai-perf and differences noted between genai-perf and llm-load-test if any identified. 

      Show
      Review on usability of genai-perf and differences noted between genai-perf and llm-load-test if any identified. 
    • Inference, Tools
    • False
    • False
    • Hide

      None

      Show
      None
    • 1
    • PSAP - General-12, PSAP - General-13

      User Story:
      As a PSAP engineer, I want to keep on top of all the available load-testing tools. Nvidia's genai-perf is recently in rapid development. 

      Acceptance criteria:

      Review on usability of genai-perf and differences noted between genai-perf and llm-load-test if any identified. 

        1. config.yaml
          1 kB
        2. distribution_of_input_sequence_lengths_to_output_sequence_lengths.jpeg
          distribution_of_input_sequence_lengths_to_output_sequence_lengths.jpeg
          35 kB
        3. request_latency.jpeg
          request_latency.jpeg
          25 kB
        4. time_to_first_token_vs_input_sequence_lengths.jpeg
          time_to_first_token_vs_input_sequence_lengths.jpeg
          40 kB
        5. time_to_first_token.jpeg
          time_to_first_token.jpeg
          22 kB
        6. token-to-token_latency_vs_output_token_position.jpeg
          token-to-token_latency_vs_output_token_position.jpeg
          31 kB

              rh-ee-thibrahi Thameem Abbas Ibrahim Bathusha
              rh-ee-thibrahi Thameem Abbas Ibrahim Bathusha
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated:
                Resolved: