-
Bug
-
Resolution: Done
-
Major
-
RHELAI 1.3 GA
-
False
-
-
False
-
Release Notes
-
Impediment
-
Known Issue
-
-
-
Important
-
Approved
To Reproduce Steps to reproduce the behavior:
- boot system with registry.stage.redhat.io/rhelai1/bootc-intel-rhel9:1.3-1732661719
- run ilab config init
- Check the config.yaml
[root@dhcp-10-111-212-61 devcloud]# cat .config/instructlab/config.yaml | grep gpus gpus: 8 gpus: 8 gpus: 8
If the config remains like this, `ilab model serve` will fail with:
AssertionError: GPUExecutor only supports single GPU. Exception ignored in: <function HabanaExecutor.__del__ at 0x7f37f90eab60> Traceback (most recent call last): File "/opt/app-root/lib64/python3.11/site-packages/vllm/executor/habana_executor.py", line 197, in __del__ File "/opt/app-root/lib64/python3.11/site-packages/vllm/executor/habana_executor.py", line 194, in shutdown AttributeError: 'HabanaExecutorAsync' object has no attribute 'driver_worker'
As per cheimes@redhat.com the config should be created with gpus set to 1 ( slack thread )
Also by updating the gpus and tensor values in the serve section to 1, the serve and chat works just fine:
[root@dhcp-10-111-212-61 devcloud]# ILAB_HOME=/var/home/devcloud ilab chat ╭────────────────────────────────────────────────────────────────────────────────────────── system ──────────────────────────────────────────────────────────────────────────────────────────╮ │ Welcome to InstructLab Chat w/ GRANITE-7B-REDHAT-LAB (type /h for help) │ ╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯ >>> hello [S][default] ╭────────────────────────────────────────────────────────────────────────────────── granite-7b-redhat-lab ───────────────────────────────────────────────────────────────────────────────────╮ │ Hello! How can I assist you today? │ ╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── elapsed 0.207 seconds ─╯ >>> are you alive ? [S][default] ╭────────────────────────────────────────────────────────────────────────────────── granite-7b-redhat-lab ───────────────────────────────────────────────────────────────────────────────────╮ │ Yes, I am an AI language model designed to help answer your questions and provide information as best I can. │ ╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── elapsed 0.513 seconds ─╯ >>>
Expected behavior
- <your text here>
Screenshots
- Attached Image
Device Info (please complete the following information):
- Hardware Specs: [e.g. Apple M2 Pro Chip, 16 GB Memory, etc.]
- OS Version: [e.g. Mac OS 14.4.1, Fedora Linux 40]
- Python Version: [output of \\{{{}python --version{}}}]
- InstructLab Version: [output of \\{{{}ilab --version{}}}]
Additional context
- <your text here>
- ...
- mentioned on
(3 mentioned on)