Uploaded image for project: 'Red Hat Enterprise Linux AI'
  1. Red Hat Enterprise Linux AI
  2. RHELAI-3884

ilab can't download models (using 'hf_transfer')

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Undefined Undefined
    • rhelai-1.4.4
    • rhelai-1.4
    • InstructLab - CLI
    • None
    • False
    • Hide

      None

      Show
      None
    • False

      On an EC2 image (rhelai-Mar-13-04h-51m),
      problem hit while downloading mistralai/Mistral-7B-Instruct-v0.1

      System info

      [cloud-user@ip-172-31-14-222 ~]$ grep -e AI -e 202 /etc/os-release 
      VERSION="9.20250108.0.4 (Plow)"
      PRETTY_NAME="Red Hat Enterprise Linux 9.20250108.0.4 (Plow)"
      OSTREE_VERSION='9.20250108.0'
      VARIANT="RHEL AI"
      RHEL_AI_VERSION_ID='1.4.0'
      
      $ grep IMAGE_NAME= $(which ilab)
      IMAGE_NAME="registry.redhat.io/rhelai1/instructlab-nvidia-rhel9:1.4-1738905416"
      

      Error

      looks like we're missing a dependency

      $ ilab model download --repository mistralai/Mistral-7B-Instruct-v0.1 --hf-token XXX
      INFO 2025-04-03 12:28:08,522 instructlab.model.download:77: Downloading model from Hugging Face:
          Model: mistralai/Mistral-7B-Instruct-v0.1@main
          Destination: /var/home/cloud-user/.cache/instructlab/models
      Removing incomplete file '/var/home/cloud-user/.cache/instructlab/models/mistralai/Mistral-7B-Instruct-v0.1/.cache/huggingface/download/.gitattributes.a6344aac8c09253b3b630fb776ae94478aa0275b.incomplete' (hf_transfer=True)
      INFO 2025-04-03 12:28:08,670 huggingface_hub.file_download:1528: Removing incomplete file '/var/home/cloud-user/.cache/instructlab/models/mistralai/Mistral-7B-Instruct-v0.1/.cache/huggingface/download/.gitattributes.a6344aac8c09253b3b630fb776ae94478aa0275b.incomplete' (hf_transfer=True)
      Downloading '.gitattributes' to '/var/home/cloud-user/.cache/instructlab/models/mistralai/Mistral-7B-Instruct-v0.1/.cache/huggingface/download/.gitattributes.a6344aac8c09253b3b630fb776ae94478aa0275b.incomplete'
      INFO 2025-04-03 12:28:08,670 huggingface_hub.file_download:1536: Downloading '.gitattributes' to '/var/home/cloud-user/.cache/instructlab/models/mistralai/Mistral-7B-Instruct-v0.1/.cache/huggingface/download/.gitattributes.a6344aac8c09253b3b630fb776ae94478aa0275b.incomplete'
      Downloading failed with the following exception: 
      Downloading model failed with the following error: 
      Downloading model failed with the following Hugging Face Hub error:
          
      Downloading safetensors model failed with the following Hugging Face Hub error:
          Fast download using 'hf_transfer' is enabled (HF_HUB_ENABLE_HF_TRANSFER=1) but 'hf_transfer' package is not available in your environment. Try `pip install hf_transfer`.
      
      

      workaround
      as a workaround manually setting "-E HF_HUB_ENABLE_HF_TRANSFER=0" in the podman run command allows the download to proceed

              Unassigned Unassigned
              dhiggins@redhat.com Derek Higgins
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated:
                Resolved: