Story

As a user of cluster log forwarding,
I want to forward logs to a RH managed lokistack

Acceptance Criteria

• Verify a user can forward all logs types to a RH managed Loki Stack that has no predetermined name
• Verify logs are forwarded to a separate tenant based upon the log type

Note

"May" Need to add separate task to support plain "loki" implementation

Currently we set up a relatively complicated trio of plain loki outputs behind the scenes of the "default" output.
This would not be easy to explain to users, or to maintain if lokistack changes its tenancy model.

Differences between LokiStack and "plain loki"

• The Loki and LokiStack APIs use the same trailing path parts for request URL but have different bases.
• LokiStack includes the tenant in the URL path, plain Loki  passes it as a HTTP Header.
• For lokistack, sending a stream of mixed log types requires separating it into single-type batches and making multiple requests.
• LokiStack requres 3 separate queries, the results need to be collated by timestamp to make the result look like a single query.

The Loki and LokiStack outputs can use mostly the same code, the LokiStack output has some additional complications:

• creating the 3 separate loki sinks (one per tenant) for a single user-configured CLF output
• separate logs arriving at the output and forward them to  the correct sinks.
• ensure all 3 sinks participate together in flow control, acknowledgements, performance tuning and other "per output" configuration. The result should "as if" there were a single sink, or as close as is reasonably possible.