mod_wsgi on VPS with multiple virtualhosts
Michael Ellis
I have a VPS running Apache 2.2.3/Python 2.4.3 and would like to
deploy some Django sites on it (under different domain names). The
server already hosts multiple corporate websites (virtual hosts)
utilizing PHP/MySQL. These are fairly low-volume sites (no forums,
etc.).
I've read the documentation and countless blog/usergroup posts, but
I'm still left wondering what would be the optimal configuration. I'm
guessing that daemon mode would suit me best, but I'm entirely
clueless as how httpd.conf, .htaccess, etc. should be configured.
Would anyone be so kind as to suggest the best option and explain it
to me — step-by-step — like I'm a 6-year-old?
Thanks,
ME
gert
http://code.google.com/p/modwsgi/wiki/ConfigurationGuidelines
I should look something like this if you want daemon mode
LoadModule wsgi_module modules/mod_wsgi.so
WSGIScriptAliasMatch "^/([^/]+)/servlet" "/srv/trunk/wsgi/$1.py"
WSGIDaemonProcess wsgid processes=1 threads=1 user=www-data group=www-
data display-name=(wsgid:wsgid) "python-path=/srv/trunk/wsgi"
WSGIProcessGroup wsgid
Nimrod A. Abing
VPS (Virtuozzo, OpenVZ, Xen, VMWare Server) you're using and the
number of guests you will be running on the host. The first problem
you will probably run into will be file handle exhaustion so you
should tweak your host OS to allow for a large amount of file handles
to be available for guests.
As for configuration, the link on gert's post is a good start.
--
Best Regards,
Nimrod A. Abing
W http://arsenic.ph/
W http://preownedcar.com/
W http://preownedbike.com/
W http://abing.gotdns.com/
Graham Dumpleton
Hmmm, although the linked documentation is a good start and always
recommend people start out with basic examples in that linked
documentation, the example configuration in this email is however not
really appropriate and may just confuse you, especially since for
Django hosting it is mostly irrelevant. I'll therefore start over and
cover a few different issues which are relevant.
First off, because you still need to run PHP applications, you
probably need to stick with using Apache prefork MPM. Normally for a
VPS I would suggest using multithreaded Apache worker MPM since it
cuts down on the number of Apache child processes and thus the overall
amount of memory that Apache users. Because various PHP third party
extensions and applications are not multithread safe, one can't really
use Apache worker MPM unless you are confident all your PHP stuff is
also multithread safe.
Note that whether you use prefork or worker MPM is actually irrelevant
to your Python web applications, because in a VPS system, where one
usually has low memory constraints, I'd recommend use of mod_wsgi
daemon mode. This is because overall it gives you more predictability
as to memory usage, albeit with some caveats. This is because if
embedded mode is used, there are copies of your Python web
applications in every Apache child process. If these are fat, that is
a lot of memory being used. Further, because Apache can scale up to
meet demand by creating more Apache child processes, the amount of
memory used can really spike up and this is usually enough to hit
memory limits on a constrained VPS system.
So second thing is thus to ensure you use mod_wsgi daemon mode for
Python we applications. If only hosting one Django instance per
VirtualHost, then use one set of daemon processes for each
VirtualHost. The simplest configuration for this would be:
<VirtualHost *:80>
ServerName www.site1.com
WSGIDaemonProcess site1 threads=15 display-name=%{GROUP}
WSGIProcessGroup site1
WSGIApplicationGroup %{GLOBAL}
...
</VirtualHost>
The three WSGI directives would be duplicated for each VirtualHost,
with 'site1' changed to unique identifier for each VirtualHost.
Suggest using a short name so that default display name, ie., as shown
for process when using 'ps', will be completely displayed and not
truncated.
With this configuration there would be one daemon process for each
VirtualHost. That process would run with 15 threads to handle
requests.
All WSGI applications would be forced to run in the context of the
global interpreter, that is the first Python interpreter. This is to
save a bit of memory, but also to avoid problems with third party
Python modules that may not work in secondary interpreters. Forcing
use of global interpreter means that can only run one Django instance
in that daemon process group. So, if needing to run multiple Django
instances in same virtual host, you either shouldn't override
WSGIApplcationGroup, or use multiple daemon process groups in same
VirtualHost contexts and delegate different Django instances to
different daemon process groups.
Now, because multiple threads are used, you do need to ensure that
your custom Django application is itself multithread safe however. If
your Django applications aren't multithread safe, then you would
instead have to use something like:
<VirtualHost *:80>
ServerName www.site1.com
WSGIDaemonProcess site1 processes=5 threads=1 display-name=%{GROUP}
WSGIProcessGroup site1
WSGIApplicationGroup %{GLOBAL}
...
</VirtualHost>
That is, run with multiple processes in the daemon process group, with
each being single threaded.
If you have to do this, you have to be very conscious of how much
memory your VPS has available. This is because rather than one process
containing one copy of your potentially fat Django application, you
now have multiple copies. As such, if you are using a very memory
constrained VPS, using a multiprocess (ie. prefork) model for daemon
processes may not be practical as you will simply run out of memory.
Thus, much preferable to ensure your Django applications are multithread safe.
Do be aware though that one can still run into memory problems if
using single multithreaded process. This is because multiple requests
can be handled concurrently within the same process. Thus process may
end up using more memory than single thread process because of
concurrent requests needing to allocate their own transient memory at
the same time. Thus memory usage of single process can spike.
Generally though the memory level still tops out something less than
combined memory usage of many single threaded processes.
If you are concerned about spiking in memory usage, then you can set
daemon processes to be recycled after set number of requests and also
when they are idle for set time. Thus:
<VirtualHost *:80>
ServerName www.site1.com
WSGIDaemonProcess site1 threads=15 display-name=%{GROUP}
maximum-requests=1000 inactivity-timeout=30
WSGIProcessGroup site1
WSGIApplicationGroup %{GLOBAL}
...
</VirtualHost>
This will result in daemon process being recycled every 1000 requests,
or when idle for 30 seconds. The result is that memory usage will drop
back to base level periodically. Also good idea to do this if you have
some sort of memory accumulation problem with your application.
Third thing is that none of the above addresses how to map URLs to
your Django application itself. For this see:
http://code.google.com/p/modwsgi/wiki/IntegrationWithDjango
Fourth and perhaps final thing in mod_wsgi sphere, is that depending
on VPS system being used, you may have problems if memory limits take
into consideration theoretical virtual memory usage rather than actual
physical memory allocated. If you use one of these types of VPS
systems and are using Linux, you may incur problem with fact that
Linux by default creates 8MB stacks for each thread in a process
rather than something more sensible. To overcome this you may have to
override what the per thread stack size is. Thus:
<VirtualHost *:80>
ServerName www.site1.com
WSGIDaemonProcess site1 threads=15 display-name=%{GROUP}
maximum-requests=1000 inactivity-timeout=30 stack-size= 524288
WSGIProcessGroup site1
WSGIApplicationGroup %{GLOBAL}
...
</VirtualHost>
For more detail on the above, you should now look properly through the
mod_wsgi documentation. How exactly you do end up configuring mod_wsgi
will depend a lot on how much memory your VPS has available, how many
Django instances you want to run and what the memory requirements of
each are. These are things, especially memory available and expected
to be used, that you haven't said anything about.
Also note that the above isn't the end of the matter. It doesn't touch
on other areas such as using Python virtual environments where each
Django instance should be truly separate and have their own Python
packages/modules. Also don't touch on running daemon processes as
different users etc. Thus, make sure you read the documentation
properly and ask more questions if necessary.
Hope this helps. One of these days I will document all this and more
as a proper tutorial. :-)
Graham
Michael Ellis
read through the docs again, try some of this out, and post back to
the thread.
Here's some more info on my VPS — if at all relevant:
CentOS 5.x
Virtuozzo
CPanel/WHM
576MB RAM (500MB typically available) — upgradeable
Thanks,
ME