WSGIDaemonProcess: processes vs threads
Maria
been looking through the mod_wsgi docs and haven't been able to find
an answer to one question I have. What would make me want to choose
to use processes vs threads vs a combination of the two? For example,
if I want a total of 10 threads, I could do any of the following:
threads=10 [leaving processes out so it defaults to 1]
processes=10 threads=1
processes=5 threads=2
processes=2 threads=5
What are the tradeoffs?
What I would expect is that generally, you'd want a large number of
threads and a small number of processes, to reduce resource usage on
the server. Except that it may be good to have 2 processes, rather
than one, if max-requests is set. That way there is one process that
is probably still running when the other is being restarted after
hitting it's request limit. Is this logic correct? Any factors that
I am failing to take into account?
Thanks!
Graham Dumpleton
On 29 July 2010 06:56, Maria <mariagu...@gmail.com> wrote:
> I'm using mod_wsgi in daemon mode with a django application. I've
> been looking through the mod_wsgi docs and haven't been able to find
> an answer to one question I have. What would make me want to choose
> to use processes vs threads vs a combination of the two? For example,
> if I want a total of 10 threads, I could do any of the following:
>
> threads=10 [leaving processes out so it defaults to 1]
> processes=10 threads=1
> processes=5 threads=2
> processes=2 threads=5
>
> What are the tradeoffs?
>
> What I would expect is that generally, you'd want a large number of
> threads and a small number of processes, to reduce resource usage on
> the server.
Unfortunately there is no simple answer as it really depends to a
degree on your application and the hardware available.
One of the mistakes that people make is configuring a lot of threads
overall. In the worst case this is because of a mistaken belief that
to handle 100 requests per second they need 100 threads. What they
fail to take into consideration is how long an individual requests
takes.
What you should always be striving for in your application is
optimising it so that requests take as little time as possible. Thus,
if you manage to get requests down to taking only 50ms, then
technically a single thread could handle 20 requests per second,
although for various other reasons it will be less than that.
So, if you went and configured 100 threads, then that is technically
up to 2000 requests per second and not just 100. Even with the default
configuration for daemon mode of a single process and 15 threads, then
that is still 300 requests per second.
Is your web site really going to handle that many requests? Even for
the default configuration of 15 threads, that is 25 million requests
per day. Frankly, if you are getting that many requests you are going
to have much bigger issues and aren't going to be running on a single
machine any way.
Why would requests never actually achieve that theoretical maximum?
Well there are various reasons for that but I'll only mention a few
below.
First is that request response times aren't uniform. It would be a
rare or very specific application that would always return in same
amount of time. Instead applications can perform many tasks and
different requests will take longer than others. Thus, the amount of
long running requests one can expect can drastically effect the
overall throughput that can be achieved. If only running with a small
number of processes/thread, just a few long running requests can
really impact throughput in a bad way.
As well as server side response time, you also have to contend with
slow HTTP clients. These can be an issue because Apache is using a
thread per request, and if client request headers dribble in slowly,
that means the request cant do anything else, not even call the
application, while it waits for request details to arrive. Even once
the application is called, if the request content also dribbles in
slowly, still slowing things down. So the client itself can impact
overall request time even if you have optimised the hell out of your
application.
This is where using a front end nginx server to Apache can be a big
win. Not only can nginx be used to handle static file requests, it by
default will buffer request headers as well as request content up to a
default of about 1MB. This means that the request will not get passed
off to Apache until point where the application can actually do
something with it immediately. Thus nginx will isolate Apache from
slow HTTP clients.
This isolation also works for the response in two ways. First off is
that Apache can offload the response quickly, with buffering implicit
in socket buffers and nginx allowing Apache to finish up quicker and
not have to deal directly with dribbling out a response.
Further, nginx proxy doesn't use keep alives which means can be
disabled in Apache and Apache can therefore close the socket
connection immediately and not have to worry about whether another
request will come down the same connection. The nginx front end itself
will worry about keep alive with client, with subsequent request
perhaps being for static files or another dynamic request that gets
proxied to back end.
So, there are lots of things to consider. Ultimately what you want to
strive for is keeping number of processes/threads down to avoid memory
consumption by having many threads, but you still need to have
sufficient capacity to handle the occasional long running request.
Most efficient use of Apache may however mean using other servers to
offloads specific tasks.
In respect of the balance between processes and threads what also has
to start worrying about the Python GIL, whether you have multiple
CPU/cores available and whether request handlers are compute intensive
or i/o bound waiting on backend databases. This really gets hard to
quantify in a general way so not even going to try at this point.
> Except that it may be good to have 2 processes, rather
> than one, if max-requests is set. That way there is one process that
> is probably still running when the other is being restarted after
> hitting it's request limit. Is this logic correct? Any factors that
> I am failing to take into account?
I would question why you are using maximum-requests. In a production
system you would want to steer clear of having application processes
restarting at all. If you have issues with memory/resource leakage in
an application you should really be trying to solve those problems.
The only situation where maximum-requests may be appropriate in my
mind, is where you have some request handlers which have large
transient memory requirements but which aren't called that often. You
don't really want the overall memory usage of an application be
dictated by these few request handlers.
In that case though would suggest use of multiple daemon process
groups with different processes/threads settings and with the
application distributed across the different process groups. Those
URLs which had large transient memory requirements you would delegate
to separate daemon process group which single or minimal threads per
process to avoid chance of same request handler being executed
concurrently and cause double whammy on memory usage. These processes
can then be reclaimed quickly using maximum-requests or
inactivity-timeout without effecting the other parts of applications
where request times and memory usage more constant and more worthwhile
to keep process persistent.
That all said, if using maximum-requests use of multiple processes
would usually be a good idea to avoid chance that startup delays of
process would cause delay in handling new request. You also would want
to look at preloading of application rather than lazy loading on first
request.
Anyway, I know this isn't a quick simple answer, but hopefully gives
you some ideas about the things that have to be considered.
Graham
Maria
using nginx in front of apache as you suggest, and it's serving our
static content. I hadn't realized the benefits it would provide with
slow clients though.
Sounds like it's worth considering separate daemon processes for the
admin requests that are less frequent, but messing with data a lot and
the regular requests that are just loading things. And it's easy
enough for us to try turning off maximum-requests for a little while
and keep an eye on how it affects memory utilization. If things stay
steady, I agree that leaving it off is a good idea.
Thanks for the help.