Introducing monitoring #1960
Introducing monitoring #1960
Conversation
BentiGorlich
added
backend
BentiGorlich
requested review from
blued-gear and
melroy89
and removed request for
blued-gear
BentiGorlich
force-pushed
the
new/monitoring
branch
from
c5908b0 to
1564aa8
Compare
BentiGorlich
force-pushed
the
new/monitoring
branch
2 times, most recently
from
b7c6b3a to
499ca01
Compare
- add monitoring capabilities for troubleshooting purposes. This cannot be enabled or disabled from the UI, you have to go into the `.env` files for this - Add monitoring capabilities for the biggest factors: curl requests (AP), twig rendering (frontend) and of course database querying - 2 sources can start an execution context: an incoming request and a started message - added an admin UI for inspecting requests and their response times, with an overview grouped by route name
BentiGorlich
force-pushed
the
new/monitoring
branch
from
499ca01 to
8b4a4d4
Compare
- Add filtering form and dto to the monitoring overview - Fix `Could not convert PHP type 'array' to 'json', as an 'Malformed UTF-8 characters, possibly incorrectly encoded' error was triggered by the serialization` and add a test for it - move chart data generation to the controller
Because the twig render is like flame graph there could be nested templates of the same type which the previous code did not support, move to a "stack" like first in last out model
Reason: we want to group performance by route name and message class. Before this we grouped performance by route name and transport, which does not make that much sense
This will end the execution context **after** the response has been sent to keep the monitoring overhead to a minimum from a users perspective, as quite a few entities will be created on this event
|
Just a warning: In about 2 days the gathered stats take up about 16GB on my relatively small server that mainly gets a lot of AP requests, but not that much user requests
|
|
Maybe the stats then should get their own DB connection configuration. This way admins can store the data somewhere where the fill up is not much of a problem or even choose a DB which allows compression. |
|
This is really exciting ! But yeah, you definitely want this is a separate DB.. Or maybe even in a timeseries optimized DB. Especially due to the sizes. |
|
I think I can bring down the size a lot if I separate the query string out to another table and only save a reference to that in the "instances" of the query. Also I want to make some things optional. At the moment it does record the parameters of each query as well, which might be overkill or just unnecessary, so I want it to be optional |
|
And it is quite a lot of data, so you might not want to monitor it for that long anyway. Otherwise you just don't get through it :D |
We need monitoring of the monitoring, that is monitoring. 😅 |
.envfiles for thisThe main goal of this PR is to add monitoring capabilities, so we can see what our messengers are doing in the background. Additionally I think the graph in the overview (even though it is from a dev instance) shows pretty well that we have to tackle our twig rendering. My guess is that that is a big problem on larger instances, like fedia.io.
Some screenshots: