Uploaded image for project: 'AI Platform Core Components'
  1. AI Platform Core Components
  2. AIPCC-1254

build nixl for llm-d in multi-node scenarios

    • llm-d: build nixl
    • False
    • Hide

      None

      Show
      None
    • False
    • AIPCC-3181 - Support for llm-d
    • AIPCC-3181Support for llm-d

      Feature Overview (mandatory - Complete while in New status)

      NVIDIA Inference Xfer Library (NIXL) is a tool for point-to-point communication in AI inference frameworks. We expect to see it added to vllm / vllm-d for sharing workloads in multi-node scenarios.

      Goals (mandatory - Complete while in New status)

      • Build nixl as a wheel

      Requirements (mandatory -_ Complete while in Refinement status):
      A list of specific needs, capabilities, or objectives that a Feature must deliver to satisfy the Feature. Some requirements will be flagged as MVP. If an MVP gets shifted, the Feature shifts. If a non MVP requirement slips, it does not shift the feature.

      Requirement Notes isMVP?
      Wheel build   Yes
           

       

      Done - Acceptance Criteria (mandatory - Complete while in Refinement status):

      A wheel collection owner can add a supported version of nixl to their collection.

      Use Cases - i.e. User Experience & Workflow: (Initial completion while in Refinement status):

      vllm-d builds will include this package

      Out of Scope _{}(Initial completion while in Refinement status):{_}

      • We do not need to add the packages to any collections.

      Documentation Considerations _{}(Initial completion while in Refinement status):{_}

      N/A

      Questions to Answer _{}(Initial completion while in Refinement status):{_}

      Which version? Always the latest.

      Background and Strategic Fit (Initial completion while in Refinement status):
      https://github.com/ai-dynamo/nixl

      Customer Considerations _{}(Initial completion while in Refinement status):{_}

      N/A

      Team Sign Off (Completion while in Refinement status)

      • All required Epics (known at the time) are linked to the this Feature
      • All required Stories, Tasks (known at the time) for the most immediate Epics have been created and estimated
      • Add - Reviewers name, Team Name
      • Acceptance == Feature as “Ready” - well understood and scope is clear - Acceptance Criteria (scope) is elaborated, well defined, and understood
      • Note: Only set FixVersion/s: on a Feature if the delivery team agrees they have the capacity and have committed that capability for that milestone

      *An engineer or tech lead from the product requesting this feature is required for the signoff below.

      Reviewed By Team Name Accepted Notes
      Doug Hellmann AIPCC yes  
             
             
             

       

              mdean@redhat.com Meirav Dean
              dhellman@redhat.com Doug Hellmann
              Antonio's Team
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

                Created:
                Updated:
                Resolved: