System dashboard:
1)
stats requests per second + heatmap
Sending request to the backend listener internal api endpoint “internal/services/stats” does not produce a response in the grafana stats graph. Instead, the requests seems to be counted as services requests.
curl -H "Authorization: Basic_ {Base64Encoded Username and Password} _" -k -v -X GET "https://backend-3scale.thr-met.apps.example-cluster.com/internal/services/4/stats"
2)
usage limits requests per second + heatmap
Sending request to the backend listener internal api endpoint “internal/services/
{service_id}/plans/{plan_id}/usagelimits” does not produce a response in the grafana Usage Limits graph. Instead, the requests seems to be counted as services requests.curl -H "Authorization: Basic {Base64Encoded Username and Password}" -k -v -X GET "https://backend-3scale.thr-met.apps.example-cluster.com/internal/services/4/plans/36/usagelimits"
3)
utilization requests per second + heatmap
Sending request to the backend listener internal api endpoint “internal/services/{service_id}
/applications/
{app_id}/utilization” does not produce a response in the grafana Utilization graph. It is counted as application requests.
curl -H "Authorization: Basic {Base64Encoded Username and Password}" -k -v -X GET "https://backend-3scale.thr-met.apps.example-cluster.redhat.com/internal/services/4/applications/12/utilization"
System dashboard
4)
system 5xx request per sec
rails_requests_total
{status=~'5[0-9]*'}Bug
Tried to upload logo into developer portal without running minio - causes 500 error in UI, also seen in system-logs
Prometheus didn’t register any traffic
**
5)
notification requests per second
rails_request_duration_seconds_bucket
{controller='notifications'}Created and deleted couple of services
it seems that there is wrong pod selected in query - it might be zync instead of system-app
**
6)
tenants requests per second
rails_request_duration_seconds_bucket
{controller='tenants'}it seems that there is wrong pod selected in query - it might be zync instead of system-app
**
7)
queue sizes
Sidekiq_queue_latency
Created and deleted couple of services
Saw traffic ‘Jobs processed by queue’ and ‘Jobs pushed by queue’ but Prometheus didn’t register any traffic in queue sizes hence i assumed that it’s not working properly
**
8)
queue latencies
sidekiq_jobs_waiting_count
Created and deleted couple of services
Saw traffic ‘Jobs processed by queue’ and ‘Jobs pushed by queue’ but Prometheus didn’t register any traffic in queue latencies hence i assumed that it’s not working properly
- is related to
-
THREESCALE-6403 System Grafana Dashboards heatmap negative values
-
- Closed
-
- relates to
-
THREESCALE-5722 Make monitoring resources available through the 3scale Operator (Phase 2)
-
- Closed
-
-
THREESCALE-4743 System grafana dashboard(s) and Prometheus rule definition (system)
-
- Closed
-
-
-
- Closed
-
-
-
- Closed
-
