Feature Overview:

The goal is to provide a new image as part of RHAIIS with model optimization components. We will start with llm-compressor.

Product(s) associated:

RHAIIS
**

Goals:

• Build the packages and image for model optimizations tools as part of RHAIIS for CUDA
• quay.io (stage) image name: rhaiis-model-opt/cuda-ubi9
• Pyxis (production) image name: rhaiis/model-opt-cuda-rhel9

Requirements:

• A new image, rhaiis-model-opt, is created.
• The new image follows the same versioning as the main rhaiis image containing vllm.
• The new image includes llm-compressor and its dependencies.
• The packages to include are documented in https://issues.redhat.com/browse/INFERSTRAT-72

Done - Acceptance Criteria:
Acceptance Criteria articulates and defines the value proposition - what is required to meet the goal and intent of this Feature. The Acceptance Criteria provides a detailed definition of scope and the expected outcomes - from a users point of view

Use Cases - i.e. User Experience & Workflow:
Include use case diagrams, main success scenarios, alternative flow scenarios.

Out of Scope:

Details of this feature were copied from: https://issues.redhat.com/browse/INFERSTRAT-68 . GuideLLM is not in scope for 3.2.2, but will be looked into in future releases.

Documentation Considerations :

The RHAIIS team is responsible for product documentation.