Uploaded image for project: 'Docs for Red Hat Developers'
  1. Docs for Red Hat Developers
  2. RHDEVDOCS-3228

Cluster LogForwarder Loki support

XMLWordPrintable

      Goals

      • Provide a new output option to forward any logs to an existing Loki cluster with the following configuration options:
        • Tenant as endpoint URI path. One tenant per OutputRef
        • Server-side TLS support only
        • Translation of fluentd fields to Loki labels..

      Non-Goals

      • Changing or adopting the new Loki output into our "DEFAULT" pipeline.
      • Us supporting Loki.
      • No custom field translation
      • No basic authentication

      Motivation

      Loki is becoming a hot topic pretty quick. We already have a handful (and more) customers that are really interested in adopting it themselves and we also plan to adopt it as an alternative to our current storage engine - the Elastic stack. We need to make sure that we let users easily integrate Loki into our log pipeline as well as our ability to switch the "DEFAULT" pipeline to Loki in the future.

      Alternatives

      Acceptance Criteria

      • Verify that any log messages from a particular input source (infra, app, and audit) are forwarded and present inside a remote Loki (can run locally, but the important part is that we standup a standalone Loki cluster).
      • Verify that we translate Loki configuration into a valid Fluentd config.
      • Verify that instead of using fields for docker., kubernetes., pipeline_metadata.*, we translate them into labels.

      Risk and Assumptions

      Documentation Considerations

      • Documentation describing what is (is not ) configurable.

      Open Questions

      Additional Notes

      • With the new log forwarding API, we are now have the ability to introduce other endpoints to which we forward logs without compromising the supportability of OpenShift Logging.
      • Loki is another promising endpoint

            rdlugyhe Rolfe Dlugy-Hegwer
            rdlugyhe Rolfe Dlugy-Hegwer
            Kabir Bharti Kabir Bharti
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated:
              Resolved: