When I set up Tempo tracing according to the documentation with the default values, with some traces (~100), the `tempo-sample-query-frontend` pod (tempo container inside) is constantly crashing with OOMKilled.
Due to that, the tracing page in Kiali is unable to fetch traces till the tempo-sample-query-frontend pod is restarted. ( with the next crashes, that restore period, during which the UI is unavailable, increases, so also Kiali traces unavalability increases )
//edit:
TempoStack CR can be configured to increase memory for that pod as follows:
apiVersion: tempo.grafana.com/v1alpha1 kind: TempoStack ... spec: resources: total: limits: cpu: 2000m memory: 8Gi
Mentioned in issues: https://github.com/grafana/tempo-operator/issues/716 , https://github.com/grafana/tempo/issues/1540#issuecomment-1743276065
- is related to
-
OSSM-5444 Support Distributed Tracing 3 (Tempo)
- Closed
-
TRACING-4009 Tempo OOM with query-frontend
- Closed