Uploaded image for project: 'OpenShift Logging'
  1. OpenShift Logging
  2. LOG-2479

Support seamless schema migration on S3 from V11 to V12

    XMLWordPrintable

Details

    • Task
    • Resolution: Done
    • Undefined
    • Logging 5.5.0
    • None
    • Log Storage, Loki
    • None
    • False
    • None
    • False
    • NEW
    • OBSDA-7 - Adopting Loki as an alternative to Elasticsearch to support more lightweight, easier to manage/operate storage scenarios
    • VERIFIED
    • Logging (LogExp) - Sprint 217, Logging (LogExp) - Sprint 218, Logging (LogExp) - Sprint 219, Logging (LogExp) - Sprint 220, Log Storage - Sprint 221, Log Storage - Sprint 222, Log Storage - Sprint 223

    Description

      The Loki-Operator should ensure seamless upgrades of the lokistack schema configuration when using S3 as object storage from V11 to V12.

      Acceptance Criteria

      • The Loki-Operator ensures an one-time persistent migration from V11 to V12.
      • New Lokistack installation start always with V11 per default.

      Developer Notes

      • Gradual index schema migrations can be done by using date-range-based schema configuration:
        schema_config: 
          configs: 
            - from: "2020-10-01"
              index: 
                period: 24h
                prefix: index_
              object_store: s3
              schema: v11
              store: boltdb-shipper schema_config: 
            - from: "2022-05-01"
              index: 
                period: 24h
                prefix: index_
              object_store: s3
              schema: v12
              store: boltdb-shipper
         

        e.g. https://grafana.com/docs/loki/latest/storage/#schema-configs

      • Schema Migration require to define a from date in the future. From an operator perspective we cannot hardcode anything here, because we don't know when users will upgrade to cluster. This mean we may need an active eval-loop from the loki operator side. Alternatively we need to open the API for users to opt-in via the ObjectStorageSpec.

      Attachments

        Issue Links

          Activity

            People

              gvanloo Gerard Vanloo (Inactive)
              ptsiraki@redhat.com Periklis Tsirakidis
              Kabir Bharti Kabir Bharti
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: