Goal

*With the growing adoption of AI/ML applications integrating a range of open-source and proprietary software, achieving high resource utilization while balancing performance and cost becomes increasingly challenging. Setting up dedicated instances for each individual model is often costly and inefficient, leading to over-provisioning and under-utilization. This project aims to explore a transparent hot-swapping technique for dynamically swapping GPU workloads that allows for more efficient inference and training with a large set of models on fewer machines.

Acceptance criteria

A list of verification conditions, successful functional tests, or expected outcomes in order to declare this story/task successfully completed.

• Verify X
• Verify Y
• Verify Z