Uploaded image for project: 'OpenShift Logging'
  1. OpenShift Logging
  2. LOG-6582

[release-5.8] Emit stream labels following OTel Semantic Conventions as a forward compatibility measure

XMLWordPrintable

    • Icon: Story Story
    • Resolution: Done
    • Icon: Normal Normal
    • Logging 5.8.17
    • Logging 5.8.z
    • Log Collection
    • None
    • 2
    • False
    • None
    • False
    • NEW
    • OBSDA-849 - Enable Full In-Cluster OpenTelemetry Support In OpenShift Logging
    • VERIFIED
    • This enhancement adds OTel semantic stream labels to the loki/lokistack output so that logs can be queried using both ViaQ and OTel stream labels.
    • Enhancement
    • Log Collection - Sprint 265, Log Collection - Sprint 266

      Story

      As a user of ClusterLogForwarder,
      I want releases of Logging to attach the "OTel semantics" stream labels to logs output using the LokiStack sink
      so that I can use both the ViaQ and the OTel stream labels to query for logs.

      Acceptance Criteria

      • Verify ingested logs include the following stream labels:
      openshift_log_type (same value as log_type)
      k8s_namespace_name (same value as kubernetes_namespace_name)
      k8s_pod_name       (same value as kubernetes_pod_name)
      k8s_container_name (same value as kubernetes_container_name)
      k8s_node_name      (same value as kubernetes_host)
      
      • Verify logs are returned in the console UI without error using any of the drop-down controls
      • Make sure that customers having a custom LabelKeys configuration will only get the compatibility labels matching their set of "ViaQ labels".

      Notes

      Logging 6.1 has introduced an alternative data model ("Otel") to be sent to OTLP capable receivers, including our own LokiStack-based storage. The new data model is following the OTel Semantic Conventions for naming the attributes and so uses a different set of "attributes" for the metadata identifying a stream of logs ("stream labels").

      Because these stream labels are used by our own log visualization tool, a compatibility layer was built into the new data model, so that it also emits the old attribute names, so that the visualization can be used unmodified with the new data model.

      This story proposes to do the same compatibility, but in the other direction, adding the new attributes to the old data-model, so that a query searching for either set of attributes will return both old and new data (LogQL does not have an "or" operator in the stream-label selector).

      The change should enable customers to query across both data-models with a single query, allowing easy migration from the old data model to the new one. For this to work properly, this change needs to be backported to all maintained releases of Logging.

              rh-ee-calee Calvin Lee
              rojacob@redhat.com Robert Jacob
              Kabir Bharti Kabir Bharti
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

                Created:
                Updated:
                Resolved: