-
Bug
-
Resolution: Done
-
Undefined
-
None
procedure
List the size of all TSDB blocks, sorted from oldest to newest, by running the following command:
oc debug <prometheus_k8s_pod_name> -n openshift-monitoring -c prometheus --image=$(oc get po -n openshift-monitoring <prometheus_k8s_pod_name> -o jsonpath='{.spec.containers[?(@.name=="prometheus")].image}') -- sh -c 'cd /prometheus/;du -hs $(ls -dt */ | grep -Eo "[0-9|A-Z]{26}")'
"ls -dt" should be "ls -dtr"
proved in local cluster, add `r` would sort from oldest to newest
$ oc -n openshift-monitoring rsh -c prometheus prometheus-k8s-0 sh-5.1$ pwd /prometheus sh-5.1$ ls -l total 20 drwxr-sr-x. 3 nobody nobody 68 Feb 20 04:57 01JMGVF9NZXR4W4YCYP6MGA5QH drwxr-sr-x. 3 nobody nobody 68 Feb 20 05:12 01JMGWA3VZVK088XX5JJ7S0JE6 drwxr-sr-x. 3 nobody nobody 68 Feb 20 07:12 01JMH35V3ZPRRWPPZWR275542V drwxr-sr-x. 2 nobody nobody 48 Feb 20 08:01 chunks_head -rw-r--r--. 1 nobody nobody 0 Feb 20 01:45 lock -rw-r--r--. 1 nobody nobody 20001 Feb 20 08:57 queries.active drwxr-sr-x. 3 nobody nobody 145 Feb 20 08:49 wal sh-5.1$ ls -dt */ | grep -Eo "[0-9|A-Z]{26}" 01JMH35V3ZPRRWPPZWR275542V 01JMGWA3VZVK088XX5JJ7S0JE6 01JMGVF9NZXR4W4YCYP6MGA5QH sh-5.1$ ls -dtr */ | grep -Eo "[0-9|A-Z]{26}" 01JMGVF9NZXR4W4YCYP6MGA5QH 01JMGWA3VZVK088XX5JJ7S0JE6 01JMH35V3ZPRRWPPZWR275542V
NOTE: the issue is since 4.12
- links to
(4 links to)
