-
Bug
-
Resolution: Unresolved
-
Undefined
-
None
-
4.12.z
-
None
-
Moderate
-
No
-
False
-
Description of problem:
When having for example 1300 pods in the same namespace, the dashboard in OpenShift Console > Observe > Dashboards > Dashboard: Kubernetes / Compute Resources / Namespace (Pods) and Namespace: <namespace containing 1300 pods> fails to be displayed with the error "The resulting dataset is too large to graph The same result is obtained when trying to list the cpu for all the pods in the same namespace.
Version-Release number of selected component (if applicable):
OCP 4.12
How reproducible:
Always
Steps to Reproduce:
1. Create a namespace containing 1300 pods: ``` $ oc new-project metrics-test $ for number in `seq 1 1300`; do kubectl create deployment hello-node-$number --image=registry.k8s.io/e2e-test-images/agnhost:2.43 -- /agnhost serve-hostname; sleep 1; done``` 2. Verify that 1300 pods are running ``` $ oc get pods -n metrics-test |grep -c Running 1300 ``` Wait some minutes and go to the `OpenShift Console > Observe > Dashboards > Dashboard: Kubernetes / Compute Resources / Namespace (Pods) and Namespace: metrics-test`
Actual results:
Fails to display the metrics for the pods in the namespace with the error "The resulting dataset is too large to graph
Expected results:
The graph `OpenShift Console > Observe > Dashboards > Dashboard: Kubernetes / Compute Resources / Namespace (Pods) and Namespace: metrics-test displays the metrics for the pods.
Additional info:
If the same query is run against the prometheus pods, the result is returned: ``` $ oc exec -c prometheus prometheus-k8s-0 -- curl -s --data-urlencode 'query=pod:container_cpu_usage:sum{pod=~"hello-node-.*",namespace="metrics-test"}' --data-urlencode "&start=2024-02-09T00:00:00.781Z" --data-urlencode "end=2024-02-10T00:00:00.781Z" 'http://localhost:9090/api/v1/query?' ```
Similar bugs in the past for when returning so many metrics, for example for storage: