When https://github.com/openshift/sippy/pull/2327 merges we should have very insightful metrics into slow and large api responses and overall rate of traffic.

Add charts to the sippy dashboard showing interesting data: https://grafana-loki.ci.openshift.org/d/WzzGWoiSk/sippy?orgId=1

The library we use has some good queries to start with: https://github.com/slok/go-http-metrics?tab=readme-ov-file#prometheus-query-examples

Just have to watch out for the fact we now scrape two pods as of this morning, so the data needs to be aggregated if you want to show one global value per line. I think this is typically done using things like "sum without(instance)" or similar. Chatgpt may be able to help for simple promql like this, but it falls apart on complex stuff.