Story

The LokiStack operator includes a proxy to enforce multi-tennacy. To connect via the proxy, the Loki forwarder needs to present a bearer token. This story is for the vector collector, the fluentd collector is a separate story.

Design

The forwarder requires a bearer-token and CA to communicate securely.

It be able to use credentials from the cluster (for default LokiStack in same cluster) or from a user-provided Secret (for forwarding to LokiStack instances outside the cluster)

Credentials are searched in the following order (first one wins)

Bearer token:

1. If the output Secret contains the key "token", use the value as the token.
2. If the output Secret contains the key "token" with an empty value, then don't use any token (to allow the user to force no-token in unusual situations)
3. If the output secret does not contain a "token" key and the Loki URL is an in-cluster service address then use the collector's service-account token: /var/run/secrets/kubernetes.io/serviceaccount/token
4. If none of the above, don't use any token.

Certificate Authority (same as used for TLS)

1. If the "ca-bundle.crt" is present in the output secret, use that.
2. If there is a CA injected into the Collector as a config map via inject-ca-bundle annotation use that
3. Else use the system default CA.

The field ClusterLogForwarder.spec.outputs[].loki.tenentKey identifies a field in the message record who's value is used as the tenent-key. The default value is "log_type" which yields the tenants ["application", "infrastructure", "audit"]. This is described in detail in the design document

Accpetance Criteria

• The Loki forwarder can connect and forward logs via the LokiStack proxy
• Logs are separated by tenant  as expected.
• Uses credentials from the output Secret if present.
• Uses credentials from SA and ca-bundle if present.
• tenetKey behaves as documented (note LOG-2356 this is buggy on fluentd, make sure it is correct for vector)

Open Questions

• Should the  use of injected CA be extended to all outputs that use TLS?
• Do we need a way to prevent the use of any token?
  • If so should use of a token be opt-in or opt-out?
  • SA token is always present, is it safe/reasonable to always use it?