Uploaded image for project: 'OpenShift Logging'
  1. OpenShift Logging
  2. LOG-1387

The structured index can not be managed by index management cronjob

    XMLWordPrintable

Details

    • False
    • False
    • NEW
    • NEW
    • Undefined

    Description

      Description of problem:
      No sequence number suffix for the structured index(qa-openshift-label), No alias for the structured index(qa-openshift-label)

      + oc exec c elasticsearch elasticsearch-cdm-wd1u5qo4-1-5c6b7b8f65-5gfrj – es_util '-query=_cat/indices?v'
      health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
      green open audit-000008 4a9ncM3LTgW5z5S-HgTg7A 3 1 0 0 1.5kb 783b
      green open .kibana_111578566_user1 QeieudG0TteFMwAKc5dvJA 3 1 1 0 8.4kb 4.2kb
      green open app-000009 HHuCYo8TR5q4_emChJBsqQ 3 1 0 0 1.5kb 783b
      green open app-000008 Wxy_sjgBTg2LOHT6PbVPEQ 3 1 0 0 1.5kb 783b
      green open infra-000008 U6ccBvwARUma_pN0Sm9RzA 3 1 0 0 1.5kb 783b
      green open .security KtDicwIaTMCsVQg09IW3rA 1 1 5 0 58.2kb 29.1kb
      green open audit-000009 jEVxZuxYSiewM0zs-R6FEw 3 1 0 0 1.5kb 783b
      green open .kibana_-377444158_kubeadmin EYm_kdoYRr2fsPB0CQlitw 3 1 2 0 26.3kb 13.1kb
      green open infra-000009 RCgYxcC-ReWvKkAB1OXhtQ 3 1 0 0 1.5kb 783b
      green open qa-openshift-label ejciKxFbQFGuwk20Sebfog 5 1 8176 0 9.6mb 4.9mb
      green open .kibana_1 p2Wy5WrJT8yo5QV3nsfWfQ 1 1 0 0 522b 261b

      Version:
      clusterserviceversion.operators.coreos.com/cluster-logging.5.1.0-26
      clusterserviceversion.operators.coreos.com/elasticsearch-operator.5.1.0-34
      Plus
      elasticsearch with openshift-ingest-plugin
      $ oc exec c elasticsearch elasticsearch-cdm-wd1u5qo4-1-5c6b7b8f65-5gfrj – es_util '-query=_cat/plugins'
      elasticsearch-cdm-wd1u5qo4-1 opendistro_security 0.10.0.4
      elasticsearch-cdm-wd1u5qo4-1 openshift-ingest-plugin 6.8.1.0
      elasticsearch-cdm-wd1u5qo4-1 prometheus-exporter 6.8.1.0

      severity
      U/U

      Steps to Reproduce:

      1. Deploy clusterlogging
      2. user1 create projects
      oc new-project project-qa-1
      oc new-app https://raw.githubusercontent.com/openshift/verification-tests/master/testdata/logging/loggen/container_json_log_template.json

      3. forward to defaut ES using structuredInde index

      Create ClusterLogForwarder instance
      cat <<EOF|oc create -f -
      apiVersion: logging.openshift.io/v1
      kind: ClusterLogForwarder
      metadata:
        name: instance
      spec:
        inputs:
        - application:
            namespaces:
            - project-qa-1
          name: input-qa
        outputDefaults:
          elasticsearch:
            structuredIndexKey: openshift.labels.team
            structuredIndexName: qa
        pipelines:
        - inputRefs:
          - input-qa
          labels:
            team: qa-openshift-label
          name: pipeline-a
          outputRefs:
          - default
          parse: json
      EOF
      

      4. Check the index and alias

      
      + oc exec -c elasticsearch elasticsearch-cdm-wd1u5qo4-1-5c6b7b8f65-5gfrj -- es_util '--query=_cat/indices?v'
      health status index                        uuid                   pri rep docs.count docs.deleted store.size pri.store.size
      green  open   audit-000008                 4a9ncM3LTgW5z5S-HgTg7A   3   1          0            0      1.5kb           783b
      green  open   .kibana_111578566_user1      QeieudG0TteFMwAKc5dvJA   3   1          1            0      8.4kb          4.2kb
      green  open   app-000009                   HHuCYo8TR5q4_emChJBsqQ   3   1          0            0      1.5kb           783b
      green  open   app-000008                   Wxy_sjgBTg2LOHT6PbVPEQ   3   1          0            0      1.5kb           783b
      green  open   infra-000008                 U6ccBvwARUma_pN0Sm9RzA   3   1          0            0      1.5kb           783b
      green  open   .security                    KtDicwIaTMCsVQg09IW3rA   1   1          5            0     58.2kb         29.1kb
      green  open   audit-000009                 jEVxZuxYSiewM0zs-R6FEw   3   1          0            0      1.5kb           783b
      green  open   .kibana_-377444158_kubeadmin EYm_kdoYRr2fsPB0CQlitw   3   1          2            0     26.3kb         13.1kb
      green  open   infra-000009                 RCgYxcC-ReWvKkAB1OXhtQ   3   1          0            0      1.5kb           783b
      green  open   qa-openshift-label           ejciKxFbQFGuwk20Sebfog   5   1       8176            0      9.6mb          4.9mb
      green  open   .kibana_1                    p2Wy5WrJT8yo5QV3nsfWfQ   1   1          0            0       522b           261b
      
      + oc exec -c elasticsearch elasticsearch-cdm-wd1u5qo4-1-5c6b7b8f65-5gfrj -- es_util '--query=_aliases?pretty'
      {
        ".kibana_-377444158_kubeadmin" : {
          "aliases" : { }
        },
        ".kibana_1" : {
          "aliases" : {
            ".kibana" : { }
          }
        },
        ".kibana_111578566_user1" : {
          "aliases" : { }
        },
        ".security" : {
          "aliases" : { }
        },
        "app-000010" : {
          "aliases" : {
            ".all" : { },
            "app" : { },
            "app-write" : {
              "is_write_index" : false
            },
            "logs.app" : { }
          }
        },
        "app-000011" : {
          "aliases" : {
            ".all" : { },
            "app" : { },
            "app-write" : {
              "is_write_index" : true
            },
            "logs.app" : { }
          }
        },
        "audit-000009" : {
          "aliases" : {
            ".all" : { },
            "audit" : { },
            "audit-write" : {
              "is_write_index" : false
            },
            "logs.audit" : { }
          }
        },
        "audit-000010" : {
          "aliases" : {
            ".all" : { },
            "audit" : { },
            "audit-write" : {
              "is_write_index" : true
            },
            "logs.audit" : { }
          }
        },
        "infra-000010" : {
          "aliases" : {
            ".all" : { },
            "infra" : { },
            "infra-write" : {
              "is_write_index" : false
            },
            "logs.infra" : { }
          }
        },
        "infra-000011" : {
          "aliases" : {
            ".all" : { },
            "infra" : { },
            "infra-write" : {
              "is_write_index" : true
            },
            "logs.infra" : { }
          }
        },
        "qa-openshift-label" : {
          "aliases" : { }
        }
      }
      
      

      Expected results:
      The structured index name is qa-openshift-label--000001 and alias as below, so the cronjob can managed the index.

        "qa-openshift-label--000001" : {
          "aliases" : {
            ".all" : { },
            "qa-openshift-label-" : { },
            "qa-openshift-label--write" : {
              "is_write_index" : true
            },
            "logs.app" : { }
          }
        }
      
      

      Attachments

        Activity

          People

            Unassigned Unassigned
            rhn-support-anli Anping Li
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: