Feature Overview

The current Mistral teacher model supports a subset of languages when compared to what the new Granite-3.0 family of models supports.

This Jira card is for the work to study the viability of using the Granite-3.0-Instruct as a future teacher model in RHEL AI.

This would bring several benefits:

• The possibility of native multi-lingual support with InstructLab
• Having a teacher model with well-known data provenance (a condition required by public sector customers and other highly regulated industries)
• Reducing the requirement of 3rd-party models

The study should include

• Identify a new prompt required for a new reference language
• Validate SDG overall quality remains within a minimum margin of error
• Validate that the resulting trained model performance is within a minimum margin of error
• Identify if further fine-tuning should be done to the model for better translation of a particular target language

Goals:

• Determine the viability of using Granite-3.0-Instruct as the default Teacher Model in RHEL AI.
• Identify required prompt updates.
• Identify if further fine-tuning is required to support a particular target language

Requirements:

• Identify required prompts updates
• Validate generated SDG quality remains within a margin of error
• Validate that the resulting trained model performance is within a margin of error
• Provide recommendations for rejecting, using or tuning Granite-3.0-Instruct as a Teacher Model

Done:

[ ] Prompt updates identified

[ ] Validation of SDG quality

[ ] Validation of quality & performance of the fine-tuned model

[ ] Recommendation of next steps

Use Cases:

Native support multi-lingual flow with Instructlab.

Out of Scope:

Considerations for any language outside the published Granite-3.0 languages

Documentation Considerations:

No need. This research should not have an impact on RHEL AI docs until productization.

Questions to Answer:

• Do we need to maintain a prompt per language, or could the multi-lingual model be used to translate the prompt when needed?
• What would be the expected model degradation using pre-defined/translated prompts versus auto-translating the prompts?

Background and Strategic Fit:

This effort is to identify a model with full data provenance and indemnification as a teacher model.

Customer Considerations: