Description of problem:
Client side throttling observed when running the metrics controller.
Steps to Reproduce:
1. Install an AWS cluster in mint mode 2. Enable debug log by editing cloudcredential/cluster 3. Wait for the metrics loop to run for a few times 4. Check CCO logs
Actual results:
// 7s consumed by metrics loop which is caused by client-side throttling time="2024-01-20T19:43:56Z" level=info msg="calculating metrics for all CredentialsRequests" controller=metrics I0120 19:43:56.251278 1 request.go:629] Waited for 176.161298ms due to client-side throttling, not priority and fairness, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-cloud-network-config-controller/secrets/cloud-credentials I0120 19:43:56.451311 1 request.go:629] Waited for 197.182213ms due to client-side throttling, not priority and fairness, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-cloud-network-config-controller/secrets/cloud-credentials I0120 19:43:56.651313 1 request.go:629] Waited for 197.171082ms due to client-side throttling, not priority and fairness, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-cloud-network-config-controller/secrets/cloud-credentials I0120 19:43:56.850631 1 request.go:629] Waited for 195.251487ms due to client-side throttling, not priority and fairness, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-cloud-network-config-controller/secrets/cloud-credentials ... time="2024-01-20T19:44:03Z" level=info msg="reconcile complete" controller=metrics elapsed=7.231061324s
Expected results:
No client-side throttling when running the metrics controller.
- blocks
-
OCPBUGS-27901 Client side throttling when running the metrics controller
- Closed
- is cloned by
-
OCPBUGS-27901 Client side throttling when running the metrics controller
- Closed
- links to
-
RHEA-2024:0041 OpenShift Container Platform 4.16.z bug fix update