Uploaded image for project: 'AI Platform Core Components'
  1. AI Platform Core Components
  2. AIPCC-3159

RHAIIS: set VLLM_USE_V1 env var for ROCm

XMLWordPrintable

    • False
    • Hide

      None

      Show
      None
    • False
    • AIPCC Accelerators 9, AIPCC Accelerators 10

      vLLM v0.9.1 defaults on V0 which has pretty bad accuracy regressions on ROCm.

      https://github.com/neuralmagic/nm-cicd/actions/runs/15590090771

      Setting VLLM_USE_V1=1 to force using the V1 engine solves the issue

              rh-ee-dtrifiro Daniele Trifirò
              rh-ee-dtrifiro Daniele Trifirò
              Selbi Nuryyeva
              Frank's Team
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Created:
                Updated:
                Resolved: