Instrumentation of OLS service and components to cover:
 

• Export metrics for latency invoking internal components (e.g., invoking memory, invoking RAG, invoking agents)
• Export metrics from LLM backend: tokens, request latency, rate of fail request
• Export metrics of the number of concurrent requests, requests per second, histogram metric of latency per session (distributed)
• Expose TLS-enabled metrics endpoint (e.g., for Prometheus)
• Create an OLS monitoring dashboard
• Collect metrics of statistics of documents retrieved (e.g., how many times a document is retrieved for answering customer questions)
  • The intent is to have an understanding of which documents are the most useful, to serve as indirect anonymized metrics of the type of topic areas the end-users are using OLS to get assistance with, etc