-
Epic
-
Resolution: Unresolved
-
Undefined
-
None
-
None
-
None
-
None
Related: https://issues.redhat.com/browse/ROX-26926 to implement Pyroscope in ACS
Pyroscope is a continuous profiling tool available from Grafana. We have persistent performance issues with XL tenants and collecting existing profiles adhoc when memory usage goes too high isn't sufficient. Let's use a tool like Pyroscope to get continuous profiling data to dig deeper into Central's performance characteristics and try to reduce resource usage, instead of throwing more resources at the problem.
Related:
- https://redhat-internal.slack.com/archives/C073ZQQN6HE/p1759915326589419
- https://www.datadoghq.com/blog/go-memory-metrics/#virtual-go-memory
- https://www.datadoghq.com/blog/go-memory-metrics/#how-to-analyze-go-memory-usage
- https://grafana.com/grafana/dashboards/14061-go-runtime-metrics/
- https://grafana.com/docs/pyroscope/latest/deploy-kubernetes/helm/#query-profiles-in-grafana
* Here is doc page for go pull mode: https://grafana.com/docs/pyroscope/latest/configure-client/grafana-alloy/go_pull/