Performance monitoring

[Vlad K.]

I'm interested in visualizing some performance metrics of my Pyramid app.

1. Average, min and max response time per route group (eg. interface
views vs API calls)
2. Optimal threads per process ratio
3. Backlog size

I'm using uwsgi. I've found this:

http://projects.unbit.it/uwsgi/wiki/ManagementFlag

from which it seems I can monitor the backlog and then some.

For #1, I thought to write a simple tween that would write realtime
stats to memcached, to be read every 5 minutes by munin for some nice
graphs. I can also graph the uwsgi backlog (also there appears to be
some munin plugins for this).

I'm not quite sure what to do for #2. I guess the easiest thing is start
with some numbers and observe graphs for #1 and #3, change the thread
per process ratio periodically and observe whether there's improvement
or degradation in graphs.


Advices? Suggestions?


Thanks,

--

.oO V Oo.

[Roberto De Ioris]

This is a far better approach:

http://projects.unbit.it/uwsgi/wiki/StatsServer

having simple json to manage is an invaluable feature.

You may want to look at the uwsgitop project for an example usage.

--
Roberto De Ioris
http://unbit.it
JID: rob...@jabber.unbit.it

[Vlad K.]

On 05/25/2012 02:19 PM, Roberto De Ioris wrote:
>
> This is a far better approach:
>
> http://projects.unbit.it/uwsgi/wiki/StatsServer
>
> having simple json to manage is an invaluable feature.
>
> You may want to look at the uwsgitop project for an example usage.
>
> --
> Roberto De Ioris
> http://unbit.it
> JID: rob...@jabber.unbit.it
>

Great, looks like an excellent additional set of stats I can monitor.
Thanks for that.



--

.oO V Oo.

[bard stéphane]

you could be interested by this small pyramid tween

pyramid_perfstat

https://bitbucket.org/tuck/pyramid_perfstat

also available in pypi

http://pypi.python.org/pypi/pyramid_perfstat

easy to use and maybe more pyramid aware. It logs
average in sqllite db.
view, route, url, sql min, max average

it could be useful to monitor projects

Hope to be helpfull

Le 25/05/2012 14:41, Vlad K. a �crit :

[Ergo]

Hi,


You can use errormator wsgi client, that allows you to use free https://errormator.com service.

It will show you slow requests/slow sqlalchemy queries within your app,
you would need to tweak the client to show you requests below 1s (that's min default for this in the client).

Because it's more geared towards error monitoring and finding code bottlenecks, but maybe it's a good start.

[Vlad K.]

Thanks for the suggestion, but the external service is not something I
can use here.

--

.oO V Oo.

[Vlad K.]

On 05/27/2012 10:30 PM, bard stéphane wrote:
> you could be interested by this small pyramid tween
>
> pyramid_perfstat
>
> https://bitbucket.org/tuck/pyramid_perfstat
>
> also available in pypi
>
> http://pypi.python.org/pypi/pyramid_perfstat
>
> easy to use and maybe more pyramid aware. It logs
> average in sqllite db.
> view, route, url, sql min, max average
>
> it could be useful to monitor projects
>
> Hope to be helpfull
>

Thanks, this seems to be what I need. I originally meant to use
memcached for performance reasons. I need this for constant monitoring
not just to troubleshoot, so I'm afraid sqlite might become a
bottleneck, but we'll see.

--

.oO V Oo.

[bard stéphane]

you're right, It's just a starter kit to help young developers
getting started with pyramid, and for the
moment I only target alchemy scaffold
(get inspired from dozer project and so one ...).

my goal :
- a project with tiny finger print (low package dependency)
- scaffold agnostic, it should work with url dispatch or traversal
o disable sql reports for traversal
- be easy to configure and high level information
- high performance, reduce the latency induced.
(I've hardcoded sql to improve that, but it's a mess => brown bag)
- offer some kind of api to work with different backend ...

I hope to get some times in few week to push another release
Feel free to fork the project and propose a work around, I'll be
happy to go further.

Le 28/05/2012 13:23, Vlad K. a �crit :