Uploaded image for project: 'OpenShift Logging'
  1. OpenShift Logging
  2. LOG-4550

Loki - Design and implement all bucket clients to use thanos.io/objstore

XMLWordPrintable

    • 8
    • False
    • None
    • False
    • NEW
    • OBSDA-527 - Enable Grafana support for cloud providers in Loki
    • NEW
    • Release Note Not Required
    • Upstream Only
    • Log Storage - Sprint 244, Log Storage - Sprint 245, Log Storage - Sprint 246

      Propose an upstream Loki migration path to use Thanos Package objstore to unify workload identity implementation among Loki, Mimir and Tempo.

      Goals

      1. Propose a design and migration strategy to gradually update Loki object storage clients from custom implementations to thanos-io/objstore.
      2. Vet and provide configuration migration risk and mitigations for users to use the new package.

      Developer Notes

      1. The thanos.io/objstore package is currently included in the code base using the bucket/client.go wrapper and only use by the ruler component (here and here)
      2. The bucket client related code that needs to replaced per provider is:
        1. Azure: blob_storage_client.go
        2. AWS: s3_storage_client.go
        3. GCP: gcs_object_client.go
        4. OpenStack: swift_object_client.go
        5. Local: fs_object_client.go
      3. We need a per-provider list of flags/config-settings/env-vars that will change when using the new implementation classified in breaking and non-breaking changes.
      4. Additional features should be considers, e.g. SSE in AWS S3, Hedging, Prefixed Buckets.
      5. A checklist of federation identity should be listed and ensured when switching to the new implementation.
      6. Consider to list any changes in exported metrics and their namespaces.
      7. Consider existing implementation in Grafana Mimir: https://github.com/grafana/mimir/tree/main/pkg/storage/bucket

              jmarcal@redhat.com Joao Marcal
              ptsiraki@redhat.com Periklis Tsirakidis
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Created:
                Updated:
                Resolved: