Uploaded image for project: 'OpenShift Logging'
  1. OpenShift Logging
  2. LOG-3945

[Vector] Collector pods in CrashLoopBackOff when ClusterLogForwarder pipeline has space in between the pipeline name.

    XMLWordPrintable

Details

    • False
    • None
    • False
    • NEW
    • VERIFIED
    • Hide
      Before this update, a ClusterLogForwarder pipeline with a space in it's name caused the collector pods to continuously crash. With this update, pipeline names are sanitized so that all spaces, dashes (-), and dots (.) are replaced with underscores (_).
      Show
      Before this update, a ClusterLogForwarder pipeline with a space in it's name caused the collector pods to continuously crash. With this update, pipeline names are sanitized so that all spaces, dashes (-), and dots (.) are replaced with underscores (_).
    • Bug Fix
    • Log Collection - Sprint 235, Log Collection - Sprint 236

    Description

      Description of problem:

      When forwarding logs to the default log store and using ClusterLogForwarder which has space in the pipeline name, the collector pods run into CrashLoopBackOff and the collector logs throws configuration error.

      ERROR vector::cli: Configuration error. error=expected a right bracket, found an identifier at line 283 column 21

      Version-Release number of selected component (if applicable):

      Cluster-logging.v5.7.0

      Elasticsearch-operator.v5.7.0

      Server Version: 4.11.0-0.nightly-2023-04-10-100905

      How reproducible:

      Always

      Steps to Reproduce:

      *Create Cluster Logging instance with Vector as collector and Elasticsearch as default log store.

      apiVersion: "logging.openshift.io/v1"
kind: "ClusterLogging"
metadata:
  name: "instance" 
  namespace: "openshift-logging"
spec:
  managementState: "Managed"  
  logStore:
    type: "elasticsearch"  
    retentionPolicy: 
      application:
        maxAge: 10h
      infra:
        maxAge: 10h
      audit:
        maxAge: 10h
    elasticsearch:
      nodeCount: 1 
      storage: {} 
      resources: 
          limits:
            memory: "4Gi"
          requests:
            memory: "1Gi"
      proxy: 
        resources:
          limits:
            memory: 256Mi
          requests:
            memory: 256Mi
      redundancyPolicy: "ZeroRedundancy"
  visualization:
    type: "kibana"  
    kibana:
      replicas: 1
  collection:
    type: vector

      *Create a Cluster Log Forwarder instance to forward all log types to the default log store.

      apiVersion: logging.openshift.io/v1
kind: ClusterLogForwarder
metadata:
  name: instance 
  namespace: openshift-logging 
spec:
  pipelines:
   - name: forward all logs
     inputRefs:  
     - audit
     - infrastructure
     - application
     outputRefs:
     - default

      *Check the collector pod status, logs and config.

      $ oc get pods
NAME                                            READY   STATUS             RESTARTS      AGE
cluster-logging-operator-7f569b4dc9-7tj89       1/1     Running            0             67m
collector-dqq7n                                 1/2     CrashLoopBackOff   6 (50s ago)   6m27s
collector-f5tg4                                 1/2     CrashLoopBackOff   6 (39s ago)   6m27s
collector-l9s2d                                 1/2     CrashLoopBackOff   6 (43s ago)   6m27s
collector-m9c25                                 1/2     CrashLoopBackOff   6 (54s ago)   6m30s
collector-z5vkl                                 1/2     CrashLoopBackOff   6 (32s ago)   6m30s
elasticsearch-cdm-n3v0u28g-1-78b8d46d9f-xrwlj   2/2     Running            0             19m
elasticsearch-im-app-28020150-zxqbz             0/1     Completed          0             4m9s
elasticsearch-im-audit-28020150-d7jqz           0/1     Completed          0             4m9s
elasticsearch-im-infra-28020150-8tsd4           0/1     Completed          0             4m9s
kibana-69bdd87787-5f4gv                         2/2     Running            0             19m
 

      $ oc logs collector-f5tg4
Defaulted container "collector" out of: collector, logfilesmetricexporter
2023-04-11T10:33:30.064435Z ERROR vector::cli: Configuration error. error=expected a right bracket, found an identifier at line 283 column 21

      Transform name in the generated vector config where the error is occurring.

      [transforms.forward all log types]
type = "remap"
inputs = ["audit","application","infrastructure"]
source = '''

      The transform name should add a separator in the name.

      Attachments

        Activity

          People

            rh-ee-calee Calvin Lee
            rhn-support-ikanse Ishwar Kanse
            Ishwar Kanse Ishwar Kanse
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: