Uploaded image for project: 'OpenShift Logging'
  1. OpenShift Logging
  2. LOG-3945

[Vector] Collector pods in CrashLoopBackOff when ClusterLogForwarder pipeline has space in between the pipeline name.

XMLWordPrintable

    • False
    • None
    • False
    • NEW
    • VERIFIED
    • Hide
      Before this update, a ClusterLogForwarder pipeline with a space in it's name caused the collector pods to continuously crash. With this update, pipeline names are sanitized so that all spaces, dashes (-), and dots (.) are replaced with underscores (_).
      Show
      Before this update, a ClusterLogForwarder pipeline with a space in it's name caused the collector pods to continuously crash. With this update, pipeline names are sanitized so that all spaces, dashes (-), and dots (.) are replaced with underscores (_).
    • Bug Fix
    • Log Collection - Sprint 235, Log Collection - Sprint 236

      Description of problem:

      When forwarding logs to the default log store and using ClusterLogForwarder which has space in the pipeline name, the collector pods run into CrashLoopBackOff and the collector logs throws configuration error.

      ERROR vector::cli: Configuration error. error=expected a right bracket, found an identifier at line 283 column 21

      Version-Release number of selected component (if applicable):

      Cluster-logging.v5.7.0

      Elasticsearch-operator.v5.7.0

      Server Version: 4.11.0-0.nightly-2023-04-10-100905

      How reproducible:

      Always

      Steps to Reproduce:

      *Create Cluster Logging instance with Vector as collector and Elasticsearch as default log store.

      apiVersion: "logging.openshift.io/v1"
      kind: "ClusterLogging"
      metadata:
        name: "instance" 
        namespace: "openshift-logging"
      spec:
        managementState: "Managed"  
        logStore:
          type: "elasticsearch"  
          retentionPolicy: 
            application:
              maxAge: 10h
            infra:
              maxAge: 10h
            audit:
              maxAge: 10h
          elasticsearch:
            nodeCount: 1 
            storage: {} 
            resources: 
                limits:
                  memory: "4Gi"
                requests:
                  memory: "1Gi"
            proxy: 
              resources:
                limits:
                  memory: 256Mi
                requests:
                  memory: 256Mi
            redundancyPolicy: "ZeroRedundancy"
        visualization:
          type: "kibana"  
          kibana:
            replicas: 1
        collection:
          type: vector
      

      *Create a Cluster Log Forwarder instance to forward all log types to the default log store.

      apiVersion: logging.openshift.io/v1
      kind: ClusterLogForwarder
      metadata:
        name: instance 
        namespace: openshift-logging 
      spec:
        pipelines:
         - name: forward all logs
           inputRefs:  
           - audit
           - infrastructure
           - application
           outputRefs:
           - default
      

      *Check the collector pod status, logs and config.

      $ oc get pods
      NAME                                            READY   STATUS             RESTARTS      AGE
      cluster-logging-operator-7f569b4dc9-7tj89       1/1     Running            0             67m
      collector-dqq7n                                 1/2     CrashLoopBackOff   6 (50s ago)   6m27s
      collector-f5tg4                                 1/2     CrashLoopBackOff   6 (39s ago)   6m27s
      collector-l9s2d                                 1/2     CrashLoopBackOff   6 (43s ago)   6m27s
      collector-m9c25                                 1/2     CrashLoopBackOff   6 (54s ago)   6m30s
      collector-z5vkl                                 1/2     CrashLoopBackOff   6 (32s ago)   6m30s
      elasticsearch-cdm-n3v0u28g-1-78b8d46d9f-xrwlj   2/2     Running            0             19m
      elasticsearch-im-app-28020150-zxqbz             0/1     Completed          0             4m9s
      elasticsearch-im-audit-28020150-d7jqz           0/1     Completed          0             4m9s
      elasticsearch-im-infra-28020150-8tsd4           0/1     Completed          0             4m9s
      kibana-69bdd87787-5f4gv                         2/2     Running            0             19m
       
      
      $ oc logs collector-f5tg4
      Defaulted container "collector" out of: collector, logfilesmetricexporter
      2023-04-11T10:33:30.064435Z ERROR vector::cli: Configuration error. error=expected a right bracket, found an identifier at line 283 column 21
      

      Transform name in the generated vector config where the error is occurring.

      [transforms.forward all log types]
      type = "remap"
      inputs = ["audit","application","infrastructure"]
      source = '''
      

      The transform name should add a separator in the name.

              rh-ee-calee Calvin Lee
              rhn-support-ikanse Ishwar Kanse
              Ishwar Kanse Ishwar Kanse
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

                Created:
                Updated:
                Resolved: