Feature Overview (aka. Goal Summary)  

OpenShift Lightspeed (OLS) wants to introduce the ability for customer bring in non OLS supported LLM Provider and Model Integration in OLS by enabling customers to try new LLM providers and models seamlessly without modifying code or resorting to unsupported workarounds.

Currently, OLS restricts the use of specific LLM providers and models. However, telemetry data has revealed that customers often attempt to use un supported LLM providers (e.g., "fake provider") or new models (e.g., LLaMA) by hacking the system. To address this we want to offer an intuitive way to configure and use custom providers or models.

Brining new model provider to OLS is not supported but we still want to give the capability to user so that we can gather telemetry on what new LLM provider and/model customer are using

Example of how customer is using work around to add new provider and models 

fake_provider 

Customer using LLAMA and granite with OpenAI