Uploaded image for project: 'Red Hat Enterprise Linux AI'
  1. Red Hat Enterprise Linux AI
  2. RHELAI-4085

`ilab model serve` fails with Triton only support CUDA 10.0 or higher, but got CUDA version: 12.8

XMLWordPrintable

    • False
    • Hide

      None

      Show
      None
    • False
    • Critical
    • Proposed

      To Reproduce Steps to reproduce the behavior:

      • Execute `ilab model serve` on RHEL AI 1.5-4 drop

      Expected behavior

      • `ilab model serve` starts correctly

      Device Info (please complete the following information):

      • Hardware Specs: AWS p5 (8xH100)
      • OS Version: RHEL AI 1.5
      • InstructLab Version: 0.26.0

      Bug impact

      • Ilab model serving does not work

      Known workaround

      • N/A

      Additional context

      • see attached logs

              fdupont@redhat.com Fabien Dupont
              jskladan@redhat.com Josef Skladanka
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

                Created:
                Updated:
                Resolved: