Call for use cases of metrics in django core

[Jeremy Dunck]

If you use/monitor/graph metrics (the idea, not Coda's library, but that would be good, too), I'd like to hear from you.  

What sort of metrics, under what implementation, would be useful to you operationally?

[Eric Holscher]

A couple obvious places:

Latency to backend systems. So, any time that I call out to my cache backend or database, keep track of round trip latency of those. 

Full system latency. So, From the time a request enters the URL routing until it gets sent back to a client.

Request counts. Keep track of the number of times a specific view or urlconf entry has been hit. This might be a bit intensive, and not worth having on.

Average latency for every view. So you can keep track of what views are the slowest.

Query counts on a per-view and overall system view.

Cache hit rates as viewed by the cache backend.

Error counts on a per-view basis. Also probably for any backend system interface. 

These are the first things that came to me. I'm sure some of them are a bit heavy weight for what we want to do. There are likely also other places where you would want to provide stats.

As for implementation, we've been using mmstats[0] at Urban Airship successfully. It does require C bits, so it might not be great for Django, especially in regards to pypy integration. However, it is pretty darn low overhead, and might be worth at least stealing some ideas from.

0: http://mmstats.readthedocs.org/en/latest/

Cheers,

Eric

[Andy McKay]

We use django-statsd in conjunction with statsd and rely upon
inserting monkey patches into the stack. Having some hooks in Django
that can be used to monitor that stuff would be really useful.

Currently we've got patches for redis, sql queries, cache queries,
template parsing and rendering along with developer generated metrics
in the code.

[Alex Gaynor]

On Wed, Oct 31, 2012 at 9:52 PM, Eric Holscher <eric.h...@gmail.com> wrote:

A couple obvious places:

Latency to backend systems. So, any time that I call out to my cache backend or database, keep track of round trip latency of those. 

Full system latency. So, From the time a request enters the URL routing until it gets sent back to a client.

Request counts. Keep track of the number of times a specific view or urlconf entry has been hit. This might be a bit intensive, and not worth having on.

Average latency for every view. So you can keep track of what views are the slowest.

Query counts on a per-view and overall system view.

Cache hit rates as viewed by the cache backend.

Error counts on a per-view basis. Also probably for any backend system interface. 

These are the first things that came to me. I'm sure some of them are a bit heavy weight for what we want to do. There are likely also other places where you would want to provide stats.

As for implementation, we've been using mmstats[0] at Urban Airship successfully. It does require C bits, so it might not be great for Django, especially in regards to pypy integration. However, it is pretty darn low overhead, and might be worth at least stealing some ideas from.

That's not, as far as I know, correct. It doesn't use any C extensions and runs fine on pypy (at least, last time I checked), and probably on Jython as well.

Alex

 

0: http://mmstats.readthedocs.org/en/latest/

Cheers,

Eric

On Wednesday, October 31, 2012 3:41:05 PM UTC-7, jdunck wrote:

If you use/monitor/graph metrics (the idea, not Coda's library, but that would be good, too), I'd like to hear from you.  

What sort of metrics, under what implementation, would be useful to you operationally?

--
You received this message because you are subscribed to the Google Groups "Django developers" group.

To view this discussion on the web visit https://groups.google.com/d/msg/django-developers/-/6fJavfHAVNgJ.

To post to this group, send email to django-d...@googlegroups.com.
To unsubscribe from this group, send email to django-develop...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.

--
"I disapprove of what you say, but I will defend to the death your right to say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
"The people's good is the highest law." -- Cicero