Feature Overview (mandatory - Complete while in New status)

With RHEL AI 1.5, we are exposing capabilities for AI engineers/Data Scientists that are only available by direct interaction with the InstructLab SDK and will require Jupyter notebooks to modify, configure, and trigger those. 

Creating a dedicated container for these will require all the packages already in the ilab container image. For this reason, this is for adding two additional packages:

• ipython
• notebook

Goal
Enable advanced users like AI Engineers and Data Scientists to use advanced flows for:

• Custom document pre-processing flows
• SDG with subset selection
• SDG with mixin ratios
• Custom SDG pipelines
• Custom fine-tuning pipelines (for training instruct student models)

Requirements

Advanced users can use the advanced capabilities to customize the flow for their use case.

They should be able to execute Jupyter Notebooks in the ilab CLI container `jupyter execute my_notebook.ipynb`

1. User onboard Jupyter Notebooks to the home directory in RHEL AI box (e.g. ~/notebooks/myselection_algo.ipynb, ~/notebooks/mycustom_pipeline.ipynb)
2. User should be able to do `ilab shell` and execute the notebooks in the order they need. Example

ilab shell

> cd /var/home/cloud-user/notebooks

> jupyter execute ./myselection_algo.ipynb ./mycustom_pipeline.ipynb

NOTE: THIS IS NOT SUPPORTING JUPYTER HUB, only the execution of Jupyter notebooks using unmodified InstructLab libraries

• Users requiring Jupyter Hub should use RHOAI built-in capabilities
• Users requiring customizations or changes to InstructLab libraries should use the "research fork" and follow the steps determined by the consulting engagement.

Done:

The ilab container image includes ipython and notebook (Jupyter notebook) as part of the image.

Use Cases 

See goals and requirements

Out of Scope 

Documenting how to use these is outside the scope of this Jira card. Documentation will be provided to AI specialists, AI consultants, and customers requiring advanced capabilities.

Documentation Considerations

N/A

Questions to Answer {}{}(Initial completion while in Refinement status):
Include a list of refinement / architectural questions that may need to be answered before coding can begin.
<your text here>

Background and Strategic Fit (Initial completion while in Refinement status):
Provide any additional context is needed to frame the feature.
<your text here>

Customer Considerations {}{}(Initial completion while in Refinement status):
Provide any additional customer-specific considerations that must be made when designing and delivering the Feature.
<your text here>