Goal: 

• Stop depending on DeepSeed for instructlab.training[cuda]

Acceptance Criteria:

• AIPCC build pipeline no longer has to build DeepSpeed for RHELAI

Upstream issue: https://github.com/instructlab/training/issues/429

 
Could you please make DeepSpeed an optional feature, so it does no longer get installed by default with instructlab[cuda] and instructlab-training[cuda]? FSDP has become the default training method a while ago. In downstream we would like to stop maintaining and building DeepSpeed. The package needs several patches and customizations to build in downstream. We need to create new patches every time we have to update DeepSpeed or PyTorch.

Suggestion:

• Remove deepspeed from requirements-cuda.txt and document that people have to manually install the package with pip install deepspeed if they would like to use the feature.
• Introduce a new dependency deepspeed / requirements-deepspeed.txt, so people can do pip install instructlab-training[cuda,deepspeed].