Uploaded image for project: 'Red Hat Enterprise Linux AI'
  1. Red Hat Enterprise Linux AI
  2. RHELAI-2668

Granite 8B inference on Gaudi cards, vLLM >=0.6.2

XMLWordPrintable

    • Icon: Spike Spike
    • Resolution: Unresolved
    • Icon: Undefined Undefined
    • None
    • RHELAI 1.4 GA
    • InstructLab - Training
    • None
    • False
    • Hide

      None

      Show
      None
    • False

      rh-ee-thibrahi tested Granite-8B on upstream vLLM running on Gaudi cards- this seems to have worked.

       

      Investigate this result further.

              rhn-support-jkunstle James Kunstle
              rhn-support-jkunstle James Kunstle
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Created:
                Updated: