Uploaded image for project: 'Red Hat Enterprise Linux AI'
  1. Red Hat Enterprise Linux AI
  2. RHELAI-2668

Granite 8B inference on Gaudi cards, vLLM >=0.6.2

XMLWordPrintable

    • Sprint 1

      rh-ee-thibrahi tested Granite-8B on upstream vLLM running on Gaudi cards- this seems to have worked.

       

      Investigate this result further.

              rhn-support-jkunstle James Kunstle (Inactive)
              rhn-support-jkunstle James Kunstle (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Created:
                Updated:
                Resolved: