-
Task
-
Resolution: Unresolved
-
Undefined
-
None
-
None
-
None
-
False
-
-
False
-
Release Notes
-
-
RHEL AI can support NVIDIA MIG (Multi-Instance GPU) devices when containers are run using podman and the nvidia-container-runtime. This setup does not require Podman to be natively CDI-enabled, as MIG instances can be specified explicitly using environment variables.
We should document this capability and provide guidance for end-users.
Scope:
Document how to use NVIDIA_VISIBLE_DEVICES with MIG UUIDs or CDI device names for inferencing in Podman-based RHEL AI environments.
Include examples like:
podman run --runtime=nvidia \ -e NVIDIA_VISIBLE_DEVICES=nvidia.com/gpu=mig1:0 \ <image> nvidia-smi -L
Reference official NVIDIA documentation:
https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/cdi-support.html#using-cdi-with-non-cdi-enabled-runtimes
Note that
CUDA_VISIBLE_DEVICES
will also respect MIG UUIDs or CDI names inside the container, enabling compatible CUDA-based applications.
Consider updating ilab to preserve the user-specified NVIDIA_VISIBLE_DEVICES environment variable when launching inference jobs.