Uploaded image for project: 'Performance and Scale for AI Platforms'
  1. Performance and Scale for AI Platforms
  2. PSAP-1524

Develop an internal dataset for RAG and fine-tuning evaluation

XMLWordPrintable

    • Icon: Story Story
    • Resolution: Unresolved
    • Icon: Major Major
    • None
    • None
    • AI/ML
    • None
    • Inference, RHELAI
    • False
    • False
    • None
    • 12
    • PSAP - General-10

      User Story:
      As a PSAP engineer,

      I want a in-house evaluation dataset,

      So that we can validate the RAG and fine-tuning methodologies we use in performance testing.

      Acceptance criteria:

      • A dataset of PDFs converted to Markdown with clear guidelines on data cleaning and format.
      • QNA files derived from the dataset for use in the RHEL AI SDG process.
      • A set of questions for benchmarking.

              rh-ee-smonson Samuel Monson
              rh-ee-smonson Samuel Monson
              Ashish Kamra
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Created:
                Updated: