I followed "Configuring the distributed tracing platform (Tempo)" section from the documentation to use Tempo with Istio and Kiali. After deploying everything, I'm able to see the traces generated by Envoy in the Jaeger UI and Kiali.

When deploying a workload that needs to use OTLP HTTP exporter, I continue seeing the traces generated by Envoy but not the ones generated by my application.

When I exclude the tracing-system from the ServiceMeshMemberRoll object, I can get the traces from my application but not from the Envoy proxy.

The usage of the OTLP HTTP exporter is a very common use case and we should document how customers need to configure OSSM to make it work when using Tempo. This works when using Jaeger.

I think maybe there is some kind of object or setting that needs to be enabled to make this work but I was not able to get any log or error to get any kind of hint about what is happening.

For reference, I'm instrumenting a Python application with this Instrumentation object:

apiVersion: opentelemetry.io/v1alpha1
kind: Instrumentation
metadata:
  name: python
  namespace: demo
spec:
  exporter:
    # Python autoinstrumentation uses http/proto by default
    # so data must be sent to 4318 instead of 4317.
    endpoint: http://tempo-sample-distributor.tracing-system.svc.cluster.local:4318
  propagators:
    - tracecontext
    - baggage
    - b3multi
  sampler:
    type: parentbased_traceidratio
    argument: "1"
  resource:
    addK8sUIDAttributes: true

And this is the application:

import flask
from time import sleep
import requests

app = flask.Flask(__name__)

@app.route('/no-header-forwarding')
def no_forwarding():
    # Simulate some work
    sleep(0.1)
    response = requests.get('http://app2:9080')
    # Simulate some work
    sleep(0.1)
    return flask.Response(response.text)