Uploaded image for project: 'Red Hat Internal Developer Platform'
  1. Red Hat Internal Developer Platform
  2. RHIDP-10113

Investigate slowness with Ollama v0.5.5 on team cluster

XMLWordPrintable

    • 8
    • False
    • Hide

      None

      Show
      None
    • False

      Task Description (Required)

      In RHDHPAI-513, we recently upgraded the Ollama image used on our team cluster to v0.5.5 (https://github.com/redhat-ai-dev/ollama-ubi/pull/7), however it seems to be responding to requests much slower than previous versions. It seems that GPU acceleration could almost be disabled, based on how slow the requests are.

      We should investigate why it's slow, and try to fix it. For the time being, the instance on the team cluster has been downgraded to v0.4.7.

              Unassigned Unassigned
              johnmcollier John Collier
              RHIDP - AI
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Created:
                Updated: