-
Bug
-
Resolution: Done
-
Critical
-
RHODS_1.1_GA
Description of problem: The cuda build chain breaks at the minimal-gpu image build step
Prerequisites (if any, like setup, operators/versions): install RHODS on OSD
Steps to Reproduce
- Install RHODS on OSD
- Wait for the cuda image builds to complete successfully
- Notice that the chain dies at gpu-minimal image
Actual results: The build chain fails at the gpu-minimal image
Expected results: All notebook image builds succeed
Reproducibility (Always/Intermittent/Only Once): Always (after July 23 at least). The problem seems to be that one of the images in the s2i build chain does a curl to fetch an s2i assemble script. It looks like at some point, the script being fetched has changed and causes us to break.
Build Details: v1.0.17
Additional info: Likely caused by the s2i build chain using curl to grab s2i assemble scripts dynamically. When those curled scripts change, we are then subject to breakage if they do not work for our builds.
- relates to
-
RHODS-1526 PyTorch and Tensorflow build pods timing out
-
- Closed
-
-
RHODS-1536 Unable to finish CUDA Builds in Open Shift Dedicated due to unschedulable pod
-
- Closed
-