Uploaded image for project: 'OpenShift Monitoring'
  1. OpenShift Monitoring
  2. MON-1771

Recommendations for gathering aggregate Prom data for archival

XMLWordPrintable

    • Icon: Task Task
    • Resolution: Duplicate
    • Icon: Normal Normal
    • None
    • None
    • None
    • False
    • False
    • NEW
    • NEW
    • Undefined

      Currently, the gather-extra step snapshots a single prometheus-k8s-* pod's data. Depending on which Prom was up and scraping when, that can leave holes in the gathered assets, such as:

      alert KubePodNotReady fired for 60 seconds with labels: {namespace="openshift-oauth-apiserver", pod="apiserver-567f9996b4-t5r86", severity="warning"}
      

      in this job which is completely absent from the gathered Prometheus dump. Is there a recommended procedure for gathering a Thanos-aggregated dump with data from both Prom instances?

      Might be related to API-1146?

              Unassigned Unassigned
              trking W. Trevor King
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

                Created:
                Updated:
                Resolved: