Proposed title of this feature request

OpenTelemetry OpenLLmetry integration

What is the nature and description of the request?

OpenLLMetry is an open source project that allows instrumentation for LLM apps. Since Tracing it's built on top of OpenTelemetry. Users can export the generated telemetry data to any observability stack via an OpenTelemetry collector.

It provides instrumentation to:

• Python
• Node.js
• Next.js
• Go
• Ruby

With Elixir and Java In development.

Why does the customer need this? (List the business requirements)

Data scientists who run workloads in the cloud, need to obtain workload telemetry to observe their LLM and ML applications. This is implemented as part of the integrations framework to ensure that users have options, so they can leverage on OpenShift to send data to many platforms, on prem, self-supported and/or observability solutions.

The business requirements of this feature are:

• Ease instrumentation of workloads in the above mentioned languages.
• Explore providing a dashboard in the OpenShift console related to relevant LLM info as defined in the OpenTelemetry standard.
  • Relevant info includes detecting hallucinations, text quality, length, word count... Read more here
• Document this integration as part of the integrations framework of this outcome: OBSDA-914 
• Document Traceloop integration (can be created as a separated Epic) this integration as part of the integrations framework of this outcome: OBSDA-914

List any affected packages or components.

• Red Hat build of OpenTelemetry

Further reading

• https://www.traceloop.com/docs/introduction
• https://docs.dynatrace.com/docs/observe-and-explore/dynatrace-for-ai-observability/traceloop-openllmetry
• https://docs.newrelic.com/docs/opentelemetry/get-started/traceloop-llm-observability/traceloop-llm-observability-intro/
• A demo on how this works can be found in the presentation rhn-support-iblancas and rh-ee-jgomezse did on the MLOps-CoP: Slides and Recording . First part is an Observability and OpenTelemetry intro, and the AI bits start at 36:20. Demo at 40'

Future work

• OBSDA-921 is a placeholder for spillovers of this Feature which may also entail discovery of further instrumentation framework