-
Task
-
Resolution: Done
-
Major
-
None
-
None
-
None
Implement Elasticsearch writer that accepts OTEL data model (pdata.Traces).
Requirements:
- accept OTEL data model pdata.Traces and directly convert it to Jaeger ES domain model
- use ES bulk API but always submit the request at the end of processing. This differs from the current implementation which batches requests in the background. The new design should alllow us to leverage OTEL batch processor https://github.com/open-telemetry/opentelemetry-collector/tree/master/processor#batch.
However, Jaeger ES Bulk configuration exposes a different configuration that OTEL batch processing:
```
--es.bulk.actions int The number of requests that can be enqueued before the bulk processor decides to commit (default 1000)
--es.bulk.flush-interval duration A time.Duration after which bulk requests are committed, regardless of other thresholds. Set to zero to disable. By default, this is disabled. (default 200ms)
--es.bulk.size int The number of bytes that the bulk requests can take up before the bulk processor decides to commit (default 5000000)
```
OTEL batch is able to configure only time based bulking and size - number of spans. We should do perf tests to assure that the new implementation has similar characteristics than the current one.
Related to https://github.com/jaegertracing/jaeger/issues/2163