Uploaded image for project: 'OpenShift Logging'
  1. OpenShift Logging
  2. LOG-4558

Vector configuration for splunk doesn't take into consideration the fields defined

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Obsolete
    • Icon: Minor Minor
    • None
    • Logging 5.7.z
    • Log Collection
    • False
    • None
    • False
    • NEW
    • NEW
    • Log Collection - Sprint 243, Log Collection - Sprint 244, Log Collection - Sprint 245, Log Collection - Sprint 246, Log Collection - Sprint 247, Log Collection - Sprint 248
    • Important

      Description of problem:

      When configured Vector for log forwarding to splunk defining fields in the clusterlogforwarder as:

      $ oc get clusterlogforwarder instance -o yaml 
      ...
      spec:
        outputs:
        - name: splunk-application
          splunk:
            fields:
            - foo
            - bar
          type: splunk
          url: http://splunk.example.com
      

      The fields foo and bar are never added to the vector configuration generated by the operator.

      A second thing is that the API  [1] and also Upstream documentation [2] indicates that fields can be an array or string, but the Splunk documentation indicates that it must be a valid JSON [3]

      Version-Release number of selected component (if applicable):

      // Using Vector as collector
      
      $ oc get csv
      NAME                            DISPLAY                                          VERSION    REPLACES                                   PHASE
      cluster-logging.v5.7.6          Red Hat OpenShift Logging                        5.7.6      cluster-logging.v5.7.5                     Succeeded
      elasticsearch-operator.v5.7.6   OpenShift Elasticsearch Operator                 5.7.6      elasticsearch-operator.v5.7.5              Succeeded 

      How reproducible:

      Always

      Steps to Reproduce:

      1. Deploy CLO with latest 5.7
      2. Setup clusterLogforwarder sending to splunk and configuring fields as below:
      $ oc get clusterlogforwarder instance -o yaml 
      ...
      spec:
        outputs:
        - name: splunk-application
          splunk:
            fields:
            - foo
            - bar
          type: splunk
          url: http://splunk.example.com
      
      1. Verify that vector doesn't include the fields added in the clusterlogforwarder for the splunk output

      Actual results:

      The fields foo and bar are not added to the vector configuration to the indexed_fields, then, never sent to splunk.

      Expected results:

      The fields foo and bar are added to the vector configuration and sent to splunk. Needed to observe that splunk is expecting a JSON as per for indexing a key - value

      Additional info:

      [1]https://github.com/openshift/cluster-logging-operator/blob/master/bundle/manifests/logging.openshift.io_clusterlogforwarders.yaml#L419
      [2] https://vector.dev/docs/reference/configuration/sinks/splunk_hec_logs/#indexed_fields
       [3] https://docs.splunk.com/Documentation/Splunk/8.0.0/Data/IFXandHEC

              Unassigned Unassigned
              rhn-support-ocasalsa Oscar Casal Sanchez
              Votes:
              2 Vote for this issue
              Watchers:
              4 Start watching this issue

                Created:
                Updated:
                Resolved: