Details
-
Feature
-
Resolution: Done
-
Major
-
None
-
OpenShift 4.9
-
None
-
None
-
False
-
False
-
0
-
0%
-
0
Description
We use the User Workload Monitoring feature to collect metrics from applications running in our namespaces. We are trying to create a custom Grafana deployments that show metrics within the same namespace. Instead of using the cluster-scoped "9091" Thanos port in the openshift-monitoring namespace that requires a service account with cluster scoped cluster-monitoring role, we'd like to use the "9092" port that limits the returned metrics to one namespace only.
The Grafana dashboards with list variables regularly use "label_values" to lookup values for variables (e.g. namespace, instance...). This uses the `/api/v1/series` Prometheus endpoint, which is currently not supported by the namespace scoped Thanos endpoint (port 9092), Http 404 is returned.
An example code to test:
$ oc port-forward -n openshift-monitoring service/thanos-querier 9091 9091 $ oc port-forward -n openshift-monitoring service/thanos-querier 9092 9092 $ BEARER_CLUSTER="$(oc extract secret/monitor-cluster-token-htp4g --to=- --keys=token)" $ BEARER_TOKEN="$(oc extract secret/monitor-namespace-token-k6nbd --to=- --keys=token)" $ curl -vk -H "Authorization: Bearer $BEARER_CLUSTER" 'https://localhost:9091/api/v1/query?query=upx' -> OK $ curl -vk -H "Authorization: Bearer $BEARER_CLUSTER" 'https://localhost:9091/api/v1/series?match%5B%5D=jvm_memory_used_bytes' -> OK $ curl -vk -H "Authorization: Bearer $BEARER_TOKEN" 'https://localhost:9092/api/v1/query?query=upx&namespace=mynamespace' -> OK $ curl -vk -H "Authorization: Bearer $BEARER_TOKEN" 'https://localhost:9092/api/v1/series?match%5B%5D=jvm_memory_used_bytes&namespace=mynamespace' -> HTTP 404
