Uploaded image for project: 'Observability Documentation'
  1. Observability Documentation
  2. OBSDOCS-170

Improving the Index Management mechanism within EO for better UX and code maintenance

XMLWordPrintable

    • 5
    • False
    • False
    • Undefined

      Documentation Considerations

      Since the introduction of OpenShift Logging 4.6 ES Index Management we supported a only one policy for customizing log retention namely based on logs max age. This is well documented here: https://docs.openshift.com/container-platform/4.10/logging/config/cluster-logging-log-store.html#cluster-logging-elasticsearch-retention_cluster-logging-store

      With LOG-1545 being now complete with OpenShift Logging 5.5, we will support another two retention policies:

      1. Pruning logs per namespace: https://github.com/openshift/cluster-logging-operator/blob/master/apis/logging/v1/clusterlogging_types.go#L173-177
      2. Pruning logs per Disk Threshold (in percentage): https://github.com/openshift/cluster-logging-operator/blob/master/apis/logging/v1/clusterlogging_types.go#L181

      Example custom resource for Pruning by namespace

      apiVersion: "logging.openshift.io/v1"
      kind: ClusterLogging
      metadata:
        name: instance
        namespace: openshift-logging
      spec:
        collection:
          type: fluentd
        logStore:
          type: elasticsearch
          elasticsearch:
            nodeCount: 3
            resources:
              requests:
                memory: 2Gi
            redundancyPolicy: SingleRedundancy
            storage:
              storageClassName: gp2
              size: 200G
          retentionPolicy:
            application:
                pruneNamespacesInterval: 24h
                namespaceSpec:
                - namespace: my-workload-ns
                  minAge: 5h
      

      You can use for namespace also wildcard like: `my-workloads-ns*`

      Example custom resource for Pruning by namespace

      apiVersion: "logging.openshift.io/v1"
      kind: ClusterLogging
      metadata:
        name: instance
        namespace: openshift-logging
      spec:
        collection:
          type: fluentd
        logStore:
          type: elasticsearch
          elasticsearch:
            nodeCount: 3
            resources:
              requests:
                memory: 2Gi
            redundancyPolicy: SingleRedundancy
            storage:
              storageClassName: gp2
              size: 200G
          retentionPolicy:
            application:
               diskThresholdPercent: 75
      

              landerso@redhat.com Libby Anderson
              rkratky@redhat.com Robert Krátký (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

                Created:
                Updated:
                Resolved: