Uploaded image for project: 'AI Platform Core Components'
  1. AI Platform Core Components
  2. AIPCC-3661

builder next PoC: build multiple versions and identify new releases of critical wheels earlier

    • False
    • True
    • 0% To Do, 0% In Progress, 100% Done
    • Hide

      14/Oct/25

      This Initiative is being re-designed using Test Collection and should no longer be targeted for 3.0. The re-design will be managed by Martin Prpic and Andre Lustosa and a new Initiative will be created/this will be resolved.

      Show
      14/Oct/25 This Initiative is being re-designed using Test Collection and should no longer be targeted for 3.0. The re-design will be managed by Martin Prpic and Andre Lustosa and a new Initiative will be created/this will be resolved.

      One of the issues we have is that we are building on-demand vs building as packages are updated or released. We should establish a way to identify new releases of critical wheels and start to build them earlier.

      We also need to be able to simultaneously build multiple versions of the same package, so we can publish them to the user-facing package index. To do that we need to remove some constraints from the wheel builder image. As an example, consider the pytorch/torch package. As of writing this pytorch-2.8.0 is available, however, due to the constraints we have, we cannot build torch-2.8.0 and are stuck at pytorch-2.7.0 or pytorch-2.7.1. Building torch-2.8.0 in advance would allow us to complete that work prior to our customers and partners requesting an upgrade.

      As a starting point we could create new collections for  next-cpu-ubi9 that initially include an unconstrained torch, and then expand the collection to include all packages with constraints in the accelerated collection, which is the default collection for test builds.  We would expand the collections for each variant – rocm, cuda, spyre, and tpu.

      When we remove the constraints, we may have to update fromager to allow separate build constraints for packages, to ensure that different versions of torch are built with the right build dependencies.

       

              fjansen@redhat.com Frank Jansen
              prarit@redhat.com Prarit Bhargava
              Prarit Bhargava
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

                Created:
                Updated:
                Resolved: