Uploaded image for project: 'OpenShift Logging'
  1. OpenShift Logging
  2. LOG-4460

Loki custom labelKeys is causing vector to enter CrashLoopBackOff state

    XMLWordPrintable

Details

    • False
    • None
    • False
    • NEW
    • VERIFIED
    • Hide
      Before this change vector forwarding to loki using custom labels was not usable when switching from fluentd which caused the collector to error. This change fixes vector configuration to sanitize labels in the same way as fluentd loki plugin to ensure the collector starts and correctly processes labels.
      Show
      Before this change vector forwarding to loki using custom labels was not usable when switching from fluentd which caused the collector to error. This change fixes vector configuration to sanitize labels in the same way as fluentd loki plugin to ensure the collector starts and correctly processes labels.
    • Bug Fix
    • Proposed
    • Log Collection - Sprint 241
    • Moderate

    Description

      Description of problem:

      OpenShift Container Platform 4 - Cluster Logging is currently configured with fluentd and ClusterLogForwarder to send logs to default elasticsearch and in parallel LokiStack. This all works fine and to enhance LokiStack capabilities, labelKeys were added to the Loki ClusterLogForwarder output specification.

      Now, when switching from fluentd to vector as collector, vector immediately enters the CrashLoopBackOff state with the below reported error message.

      $ oc logs collector-9q6pn
      Defaulted container "collector" out of: collector, logfilesmetricexporter
      2023-08-18T14:38:29.580435Z ERROR vector::cli: Configuration error. error=Invalid field path in template "kubernetes.namespace_labels.foo-bar_com_platform-stage" (see https://vector.dev/docs/reference/configuration/template-syntax/)
      in `sinks.loki_output_application`
      

      It's though expected that transition between Fluentd and vector can be triggered without such an error being raised and to have vector support and work custom labelKeys

      Version-Release number of selected component (if applicable):

      OpenShift Container Platform 4 - Cluster Logging 5.7.4

      How reproducible:

      Always

      Steps to Reproduce:

      1. Install OpenShift Container Platform 4 - Cluster Logging with default sink configured for elasticsearch and ClusterLogForwarder with Loki and add labelKeys as mentioned above (for example kubernetes.namespace_labels.foo-bar_com_platform-stage).

      Actual results:

      $ oc logs collector-9q6pn
      Defaulted container "collector" out of: collector, logfilesmetricexporter
      2023-08-18T14:38:29.580435Z ERROR vector::cli: Configuration error. error=Invalid field path in template "kubernetes.namespace_labels.foo-bar_com_platform-stage" (see https://vector.dev/docs/reference/configuration/template-syntax/)
      in `sinks.loki_output_application`
      

      Expected results:

      No error being reported and collector to work with vector as it did with fluentd

      Additional info:

      Attachments

        Activity

          People

            jcantril@redhat.com Jeffrey Cantrill
            rhn-support-sreber Simon Reber
            Kabir Bharti Kabir Bharti
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: