Goal: Enable tracing data to be collected via a streaming platform.

Problem: Dependency on a suitably provisioned Kafka cluster.

Why is this important: Kafka provides a buffer between collectors and backend storage (Elasticsearch) to help cope with spikes in throughput. It also provides a way for adhoc realtime analysis to be performed on the tracing data without having to poll the storage.

Dependencies (internal and external):
Strimzi/AMQ

Prioritized epics + deliverables (in scope / not in scope):

Estimate (XS, S, M, L, XL, XXL):
M

Previous Work: Functionality has been implemented upstream, including adding Kerberos support to the Kafka client (sarama) in support of a customer use case. Main outstanding work is to setup QE environment.

Customers: OSSM early access customers USAA and Sabre have asked for Kafka support.

Open questions: