Detecting threads with a current request

[Kent]

Is there an external way to detect whether a thread is currently handling a request or how many threads are currently handling requests?

Thanks,

Kent

[Graham Dumpleton]

On 3 Dec 2015, at 2:03 AM, Kent <jkent...@gmail.com> wrote:

Is there an external way to detect whether a thread is currently handling a request or how many threads are currently handling requests?

In the first instance have a look at:

https://code.google.com/p/modwsgi/wiki/DebuggingTechniques#Extracting_Python_Stack_Traces

If don’t need so much detail and just after counts to determine capacity utilisation for the process/server, then there are ways of getting such metrics out of mod_wsgi itself.

What I can’t remember right now is whether this specific metric is available in latest released version or still sitting on a branch with a bunch of other new metrics changes.

What is the specific problem you are trying to solve?

Graham

[Kent Bower]

Just trying to determine capacity utilisation for the process/server.
Thank you!

--
You received this message because you are subscribed to a topic in the Google Groups "modwsgi" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/modwsgi/efdtI39ccRI/unsubscribe.
To unsubscribe from this group and all its topics, send an email to modwsgi+u...@googlegroups.com.
To post to this group, send email to mod...@googlegroups.com.
Visit this group at http://groups.google.com/group/modwsgi.
For more options, visit https://groups.google.com/d/optout.

[Graham Dumpleton]

On 3 Dec 2015, at 11:48 PM, Kent Bower <ke...@bowermail.net> wrote:

Just trying to determine capacity utilisation for the process/server.
Thank you!

On Wed, Dec 2, 2015 at 4:21 PM, Graham Dumpleton <graham.d...@gmail.com> wrote:

On 3 Dec 2015, at 2:03 AM, Kent <jkent...@gmail.com> wrote:

Is there an external way to detect whether a thread is currently handling a request or how many threads are currently handling requests?

In the first instance have a look at:

https://code.google.com/p/modwsgi/wiki/DebuggingTechniques#Extracting_Python_Stack_Traces

If don’t need so much detail and just after counts to determine capacity utilisation for the process/server, then there are ways of getting such metrics out of mod_wsgi itself.

What I can’t remember right now is whether this specific metric is available in latest released version or still sitting on a branch with a bunch of other new metrics changes.

What is the specific problem you are trying to solve?

What version of mod_wsgi are you using?

After checking, the more recent versions (not the much older versions on most Linux distros) do have the metric counter I mention.

The process metrics are available using:

    import mod_wsgi

    current_metrics = mod_wsgi.process_metrics()

They include something like:

    mod_wsgi.process_metrics: {'cpu_system_time': 0.009999999776482582, 'request_busy_time': 0.00033, 'current_time': 1449186182.184397, 'memory_max_rss': 9646080L, 'memory_rss': 9646080L, 'pid': 5485, 'restart_time': 1449186179.973325, 'request_count': 0L, 'cpu_user_time': 0.03999999910593033, 'running_time': 2L}

If you poll this at 1 second interval from a background thread, then capacity utilisation is calculated using:

    (current_metrics[‘request_busy_time’] - last_metrics[‘request_busy_time’]) / (current_metrics[‘current_time’] - last_metrics[‘current_time’])

Personally I have been feeding this as well as a bunch of other stuff from unreleased version in a mod_wsgi branch into InfluxDB and charting it using Grafana. I should probably finally look at merging the branch so that people can play with it. Just not 100% sure I may want to tweak it a bit yet.

Graham

[Kent Bower]

Using mod_wsgi-4.4.6.

Thanks for that information, that could be useful.  I suppose there is no external way to ask Apache how many workers are currently handling mod_wsgi python requests or figure out the total mod_wsgi thread count busy with requests?

Also, only partly off-topic: if running in daemon mode like this:

WSGIDaemonProcess rarch processes=30 threads=8 inactivity-timeout=1800 display-name=%{GROUP} python-eggs=/home/app/tg2env/lib/python-egg-cache

Is there any substantial advantage of worker MPM over prefork?  It seems to my understanding that daemon mode overcomes the limitations of prefork.

Thanks again!

--

[Graham Dumpleton]

Are you effectively after live performance monitoring?

What I have been doing is pushing all the metrics into InfluxDB and charting it with Grafana.

Right now I have been treating it as a play thing for myself and so haven’t really said anything about the capability nor merged some of my latest work for it back into main code. 

If others want to play with it also, I can though finally merge my changes, outline how you can hook it up to InfluxDB and share some Grafana dashboards for a few things.

Graham

You received this message because you are subscribed to the Google Groups "modwsgi" group.
To unsubscribe from this group and stop receiving emails from it, send an email to modwsgi+u...@googlegroups.com.

[Kent Bower]

Precisely, live performance monitoring, but it is also at this point more of just curiosity and a play thing, so I don't want you to take that extra effort currently. 

Thanks much!