Uploaded image for project: 'AI Platform Core Components'
  1. AI Platform Core Components
  2. AIPCC-10018

Adding new ENV variables for prefix caching

      Adding

      VLLM_SPYRE_USE_CHUNKED_PREFILL=1  
      VLLM_DT_CHUNK_LEN=1024
      

      variables to enable prefix caching in vLLM

              rimsaha@redhat.com Rimpi Saha
              autobot-jira-api pme bot
              Frank's Team
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Created:
                Updated:
                Resolved: