Task

As an engineer working on RHDH, I want to get some metrics around the perfomance of the Deep Context Retrieval PoC so that is easier for all stakeholders to understand the limitations of this feature.

Background

This is the final step of the parent epic (https://issues.redhat.com/browse/RHIDP-11566) and will try to measure all metrics from the PoC created in https://issues.redhat.com/browse/RHIDP-11567 

The important part is to make sure we have test the PoC against multiple models.

Dependencies and Blockers

QE impacted work
Documentation impacted work 

Acceptance Criteria

• Performance metrics have been shared here in the Jira issue
• Compare the metrics taken from lightspeed when the feature is disabled with those when the feature is enabled.
• Experiment with various AI Providers to understand the importance and role of the model and its capabilities