Uploaded image for project: 'OpenShift Service Mesh'
  1. OpenShift Service Mesh
  2. OSSM-5995

OOMKilled in tempo-sample-query-frontend

XMLWordPrintable

    • Icon: Risk Risk
    • Resolution: Not a Bug
    • Icon: Critical Critical
    • None
    • OSSM 2.5.0
    • Kiali, Maistra
    • None
    • False
    • None
    • False

      When I set up Tempo tracing according to the documentation with the default values, with some traces (~100), the `tempo-sample-query-frontend` pod (tempo container inside) is constantly crashing with OOMKilled.
      Due to that, the tracing page in Kiali is unable to fetch traces till the tempo-sample-query-frontend pod is restarted. ( with the next crashes, that restore period, during which the UI is unavailable, increases, so also Kiali traces unavalability increases )

      //edit:

      TempoStack CR can be configured to increase memory for that pod as follows:

      apiVersion: tempo.grafana.com/v1alpha1
      kind: TempoStack
      ...
      spec:
        resources:
          total:
            limits:
              cpu: 2000m
              memory: 8Gi
      

      Mentioned in issues: https://github.com/grafana/tempo-operator/issues/716 , https://github.com/grafana/tempo/issues/1540#issuecomment-1743276065

              Unassigned Unassigned
              mkralik@redhat.com Matej Kralik
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Created:
                Updated:
                Resolved: