Uploaded image for project: 'OpenShift Logging'
  1. OpenShift Logging
  2. LOG-2571

Cluster-Logging Loki Integration

    XMLWordPrintable

Details

    • CLO Loki Integration
    • False
    • None
    • False
    • OBSDA-7Adopting Loki as an alternative to Elasticsearch to support more lightweight, easier to manage/operate storage scenarios
    • Green
    • NEW
    • To Do
    • OBSDA-7 - Adopting Loki as an alternative to Elasticsearch to support more lightweight, easier to manage/operate storage scenarios
    • Impediment
    • QE Needed, Docs Needed, TE Needed, Customer Facing, PX Needed
    • VERIFIED
    • 100
    • 100% 100%
    • Feature Freeze

    Description

      Goals

      • Provide single-entrypoint log forwarding support for LokiStack as a new default store from cluster-logging collectors.
      • Ensure the log collectors/forwarders have end-to-end access to LokiStack pushing logs based on the current tenancy model (application, infrastructure, audit)

      Non-Goals

      • Provide single-entrypoint configuration for LokiStack custom resources via the ClusterLogging custom resource.

      Motivation

      The current approach to use LokiStack as the new storage solution for ClusterLogging requires to address manual steps provided by documentation. These steps include (Ref: docs):

      • Create RBAC roles and bindings for the cluster logging log collector serviceaccount to access the LokiStack resource
      • Define a ClusterLogForwarder configuration that ensures one pipeline/output to the LokiStack resource per tenant (application, infrastructure, audit).
      • Ensure that ClusterLogForwarder is using the LokiStack gateway endpoint so that proper authentication/authorization control is applied on collector requests.
      • Ensure that ClusterLogging collector configuration is use the serviceaccount oauth token mounted in the collector pods.

      All the above can and should be automated by the cluster-logging-operator upon using LokiStack as new type of log store. The above bits describe basically steps that require automation for a seamless connectivity between log collectors and LokiStack as a store.

      Alternatives

      Continue to provide the above steps only per documentation.

      Acceptance Criteria

      • Verify that log collectors (fluentd, vector) can push logs to LokiStack using the serviceaccount token.
      • Verify that log collectors push logs based on the cuCLO Loki Integrationrent tenancy model.

      Risk and Assumptions

      • We want to keep the configuration bits in ClusterLogging/ClusterLogForwarder limited to connectivity and ensure at the same time that users don't need to figure them out.
      • We want to make it easy to identify the LokiStack instance by name instead of reconciling a LokiStack custom resource from ClusterLogging Operator. On the one hand this ensures users can tune LokiStack independently from ClusterLogging. On the other hand this requires users to properly size and install a LokiStack custom resource upfront.

      Documentation Considerations

      Given that creating a LokiStack resource is a per-requisite for using ClusterLogging with a log store type LokiStack, we need to reverse the order of installation (First LokiStack, Next ClusterLogging). In addition we need to inform the user that ClusterLogForwarder is used by default in this case.

      Open Questions

      Additional Notes

      Attachments

        Issue Links

          Activity

            People

              ptsiraki@redhat.com Periklis Tsirakidis
              ptsiraki@redhat.com Periklis Tsirakidis
              Kabir Bharti Kabir Bharti
              Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: