Uploaded image for project: 'AI Platform Core Components'
  1. AI Platform Core Components
  2. AIPCC-3661

builder next PoC: build multiple versions and identify new releases of critical wheels earlier

    • False
    • True
    • 67% To Do, 33% In Progress, 0% Done
    • Hide

      08/Oct/25

      Green

      The priorities for this ticket are CUDA, ROCm, and CPU. CUDA is in progress, while CPU and ROCm were deprioritized in favor of the RHAIIS 3.2.3 release. Once those deliverables are stable and resources are available, CPU and ROCm will resume. 

      Show
      08/Oct/25 Green The priorities for this ticket are CUDA, ROCm, and CPU. CUDA is in progress, while CPU and ROCm were deprioritized in favor of the RHAIIS 3.2.3 release. Once those deliverables are stable and resources are available, CPU and ROCm will resume. 

      One of the issues we have is that we are building on-demand vs building as packages are updated or released. We should establish a way to identify new releases of critical wheels and start to build them earlier.

      We also need to be able to simultaneously build multiple versions of the same package, so we can publish them to the user-facing package index. To do that we need to remove some constraints from the wheel builder image. As an example, consider the pytorch/torch package. As of writing this pytorch-2.8.0 is available, however, due to the constraints we have, we cannot build torch-2.8.0 and are stuck at pytorch-2.7.0 or pytorch-2.7.1. Building torch-2.8.0 in advance would allow us to complete that work prior to our customers and partners requesting an upgrade.

      As a starting point we could create new collections for  next-cpu-ubi9 that initially include an unconstrained torch, and then expand the collection to include all packages with constraints in the accelerated collection, which is the default collection for test builds.  We would expand the collections for each variant – rocm, cuda, spyre, and tpu.

      When we remove the constraints, we may have to update fromager to allow separate build constraints for packages, to ensure that different versions of torch are built with the right build dependencies.

       

              fjansen@redhat.com Frank Jansen
              prarit@redhat.com Prarit Bhargava
              Prarit Bhargava
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

                Created:
                Updated: