Uploaded image for project: 'OpenShift Logging'
  1. OpenShift Logging
  2. LOG-2554

[Vector] [master] Fallback index is not used when structuredTypeKey is missing from JSON log data

XMLWordPrintable

    • False
    • None
    • False
    • NEW
    • VERIFIED
    • Logging (Core) - Sprint 218

      Version-Release number of selected component (if applicable):

      Logging 5.5

      Server Version: 4.11.0-0.nightly-2022-04-26-181148

      Kubernetes Version: v1.23.3+d464c70

       

      Description of problem:

      When using structuredTypeKey and structuredTypeName together in a ClusterLogForwarder instance, the logs are not sent to the fallback index when structuredTypeKey is missing from JSON logs.

       

      How reproducible:

      Always

       

      Steps to reproduce:

      1 Deploy Cluster Logging and Elasticsearch 5.5 operators.

      2 Create a new project with the following log generator application.

      oc new-project project-qa
      oc new-app https://gitlab.cee.redhat.com/aosqe/aosqe-tools/-/raw/master/logging/log_gen/container_json_log_template.json 

      3 Create a ClusterLogForwarder instance with an invalid structuredTypeKey.

      oc create -f https://gitlab.cee.redhat.com/aosqe/aosqe-tools/-/raw/master/logging/log_template/vector/clf_structuredTypeKey_invalid.yaml 
      
        outputDefaults:
          elasticsearch:
            structuredTypeKey: kubernetes.labels.foo
            structuredTypeName: qa-index-name

      4 Create the ClusterLogging instance.

      oc create -f https://gitlab.cee.redhat.com/aosqe/aosqe-tools/-/raw/master/logging/log_template/vector/cli_zeroredundancy.yaml 

      6 Check the indices in the default Elasticsearch log store. The logs are sent to the app index instead of app-qa-index-name-00* index.

      $ oc rsh elasticsearch-cdm-uo4a0lox-1-677b4db655-5bhx2
      Defaulted container "elasticsearch" out of: elasticsearch, proxy
      sh-4.4$ indices
      Fri Apr 29 10:01:35 UTC 2022
      health status index        uuid                   pri rep docs.count docs.deleted store.size pri.store.size
      green  open   app-000001   HYCnvH26SsG-q0ELMx2Jsg   1   0        783            0          0              0
      green  open   infra-000001 5n_IDYFJSJGiB_gxVaOd-w   1   0          0            0          0              0
      green  open   .kibana_1    6EAoMeY0TYOW4NwFidTITQ   1   0          0            0          0              0
      green  open   audit-000001 fiEghAi-QLSeYUInp9io7w   1   0          0            0          0              0
      green  open   .security    nYgC7l9qSS6esNwWIhIoHw   1   0          6            0          0              0 

            vimalkum@redhat.com Vimal Kumar
            rhn-support-ikanse Ishwar Kanse
            Ishwar Kanse Ishwar Kanse
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: