-
Bug
-
Resolution: Done
-
Major
-
SaaS, 2.14.0 GA, 2.14.0-mas (0.11.8)
-
False
-
-
False
-
Not Started
-
Not Started
-
Not Started
-
Not Started
-
Not Started
-
Not Started
-
-
There is a bug in the setup of the Prometheus metrics for System, that affects specifically the Rails-related ones (not Sidekiq), and exposed at /yabeda-metrics.
The problem is that each Unicorn worker collects and reports its own metrics, so what's aggregated in Prometheus is not correct (as counter metrics jump back and forth, the resulting numbers are wrong).
We're using yabeda-prometheus-mmap based on prometheus-client-mmap which supports multi-process environments, but for that we need to set prometheus_multiproc_dir environment variable to ensure all the processes (e.g. all unicorn workers) report to aggregated metrics counters.
- mentioned on
(5 mentioned on)