Optimal daemon configuration?
Szymon
I have some problems with mod_wsgi performance, so I will ask here for
help. I hope it's ok? :P
Ok. I have Django application, big one. My setup is based on Apache
and mod_wsgi of course. There is WWW server and DB one. In rush hours
average load is higher than 30[1]. In rush hours I mean 80-90 req/s.
It is maximum what I can get from Core2Duo / 4 GB of RAM server or can
I make some configuration tweaks?
What daemon processes (number) / mpm prefork configuration will be the
best?
[1] - DB server is under 3.0 then.
Graham Dumpleton
What is the configuration you are using now? So, provide information such as:
1. Are you using separate media server for static files?
2. Are you using a front end proxy to Apache/mod_wsgi like nginx?
3. Are you using prefork or worker MPM?
4. Are you running other web application on same Apache using mod_php
or other systems?
5. Is your Python web application running in embedded mode or daemon mode?
6. Have you read:
http://blog.dscpl.com.au/2009/03/load-spikes-and-excessive-memory-usage.html
It is hard to give any advice without knowing. For all we know, you
could just have configured your current system wrongly and if tweaked
it could work fine.
Graham
Szymon
> 1. Are you using separate media server for static files?
Nope. Static content is served from same server, but using nginx.
> 2. Are you using a front end proxy to Apache/mod_wsgi like nginx?
No.
> 3. Are you using prefork or worker MPM?
Prefork.
> 4. Are you running other web application on same Apache using mod_php
> or other systems?
Yes, there is one PHP app, but it's not busy at all. But if it's make
difference I can resign from that app at all.
> 5. Is your Python web application running in embedded mode or daemon mode?
Daemon mode.
> 6. Have you read:
>
> http://blog.dscpl.com.au/2009/03/load-spikes-and-excessive-memory-usa...
No. :) But I'll do. :)
> It is hard to give any advice without knowing. For all we know, you
> could just have configured your current system wrongly and if tweaked
> it could work fine.
Here is my configuration:
WSGIDaemonProcess app user=app group=app processes=30 maximum-
requests=10000
WSGIProcessGroup app
And my prefork config:
StartServers 5
MinSpareServers 10
MaxSpareServers 25
MaxClients 250
MaxRequestsPerChild 0
Keep-Alive is turned off. I'm on FreeBSD if that's make difference.
Szymon
I've removed PHP, turned Apache to worker, changed wsgi daemon to
processes=4 threads=75... and omg. It's HUGE difference. :O Really.
From 60-70 requests currently being processed Apache goes down to
10-15. Thanks for that blog note! Link should be in some visible place
on mod_wsgi home page! :)
Van Gale
You can keep PHP with Apache worker... just switch from mod_php to mod_fcgid. Worked for me, although admittedly my PHP sites are low traffic (8k reqs/day).
Also, even though I was very happy with Apache+mod_wsgi performance I had to put nginx at the front-end to use as SSL reverse proxy. Turns out this also made a big difference just serving the static media directly from nginx. This surprised me since both servers are using sendfile kernel call. So now, when I look at Munin, my Apache charts are all flat lines :)
--
You received this message because you are subscribed to the Google Groups "modwsgi" group.
To post to this group, send email to mod...@googlegroups.com.
To unsubscribe from this group, send email to modwsgi+u...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/modwsgi?hl=en.
Graham Dumpleton
> Hi,
>
> I've removed PHP, turned Apache to worker, changed wsgi daemon to
> processes=4 threads=75... and omg. It's HUGE difference. :O Really.
> From 60-70 requests currently being processed Apache goes down to
> 10-15. Thanks for that blog note! Link should be in some visible place
> on mod_wsgi home page! :)
You probably don't even need 75 threads per process and may even find
default of 15 per process is more than sufficient.
Whether that will be true though depends on how many long running
requests you expect to get. If all requests are handled in under
100ms, then even with only 4x15 threads, that is theoretical 600
requests/sec you could handle, although in practice overheads mean
that will come down. I have seen people who were still getting very
good performance on a high volume site out of running 3 single
threaded processes. Granted they were good at what they did and had
tuned the hell out of their application and database and used caching
well to remove all the bottlenecks from it so request times were very
very short.
Do you have any idea what you average request times are and what
proportion of requests are longer and what maximum times are.
As I alluded to by my questions and as someone else pointed out, you
can also get better utilisation out of Apache/mod_wsgi by using a
nginx front end proxy. Can be same one serving static files.
Why this works is that nginx acts to isolate Apache/mod_wsgi from slow
HTTP clients as nginx for request body under a certain size will
buffer up request headers and request content and only proxy it on
when it has it all. This means that Apache/mod_wsgi will only get the
request when it has all of it and is able to process it immediately
and dump back response. The buffering implicit in socket connections
and nginx for response also means Apache/mod_wsgi can release the
connection quicker as well and nginx again will deal with a slow
client consuming the response.
Using nginx as front end proxy like this will allow you to get more
out of Apache/mod_wsgi with less processes/threads and thus less
memory overall. Likely a lot of your problems were due to memory
needing to be paged and different processes worken up, due to having
many large fat processes.
Graham