Feature Overview (aka. Goal Summary)

The current behavior of OpenShift Lightspeed (OLS) waits for the entire response from the large language model (LLM) before displaying it to the user, leading to a perception that OLS is slow. This feature will address the problem by introducing real-time streaming of responses, allowing users to see output progressively as the LLM generates it. This will significantly reduce perceived wait time and enhance user satisfaction by providing faster feedback.