Uploaded image for project: 'Observability Documentation'
  1. Observability Documentation
  2. OBSDOCS-803

kubernetes.container_name as reference for structuredTypeName impacts to the Elasticsearch performance

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Undefined Undefined
    • None
    • Logging 5.8, Logging 5.6, Logging 5.7
    • Logging
    • False
    • Hide

      Wrong project

      Show
      Wrong project
    • False
    • Moderate

      URL

      https://docs.openshift.com/container-platform/4.14/logging/log_collection_forwarding/cluster-logging-enabling-json-logging.html#cluster-logging-configuration-of-json-log-data-for-default-elasticsearch_cluster-logging-enabling-json-logging

      SECTION

      Configuring JSON log data for Elasticsearch

      DESCRIPTION

      It's said:

      You can use the following structure types in the ClusterLogForwarder CR to construct index names for the Elasticsearch log store:
      ...
      kubernetes.container_name uses the container name to construct the index name.
      

      The kubernetes.container_name is one of the worst keys to be used as structuredTypeKey as ** JSON is enable for all the applications indices, it will create a population of new indices created in Elasticsearch that will kill the performance of Elasticsearch.

      Then, or it's removed or even better, it should be indicated if choosen, then, only using for a subset of pods to avoid performance issues associated with having too many indices, consider keeping the number of possible schemas low by standardizing to common schemas as said later in the documentation in a general note.{}{}

              bdooley@redhat.com Brian Dooley
              rhn-support-ocasalsa Oscar Casal Sanchez
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Created:
                Updated: