Uploaded image for project: 'Observability and Data Analysis Program'
  1. Observability and Data Analysis Program
  2. OBSDA-55

Feature-Request: Possibility to set max_size for Elasticsearch rollover

XMLWordPrintable

    • False
    • False
    • Undefined

      What is the problem that your customer is facing?

      In environments with large OpenShift Container Platform 4 - Cluster and retention Policy set to 30 days for example, the rollover is set to maxAge 1d. Meaning if massive amount of logs are sent, the respective index and it's associated shards are growing quickly. But with rollover set to 1d it will grow beyond the recommended size for shards, which is why it would be helpful to either set maxAge manually (and not have it calculated) or even better, be able to set max_size to limit the max size and thus trigger the rollover in the right moment to remain in terms of sizing always in the recommended values.

      What is the business impact, if any, if this request will not be made

      Certain indexes will grow beyond 50 GB in space which has a negative impact on performance as well as when maintenance tasks are being performed (such as updates, etc.). These tasks will simply require more resources and more time which has an impact on availability and stability. With the possibly to control the size of objects, customers can make sure they remain in recommended boundaries and thus limit the performance impact related to the size (of the Cluster) to some extend

      What are your expectations for this feature

      It should be possible to control the rollover of index more detailed and not have to rely on calculated values. Hence being able to either set maxAge for rollover manually or specifying max_size  would be helpful.

      Have you done this before and/or outside of support and if yes, how?

      No - But we have seen at customers that index are growing up to 100GB in size and more with respective shared also in that are with regards to size.

            jamparke@redhat.com Jamie Parker
            jamparke@redhat.com Jamie Parker
            Votes:
            1 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: