mod_python, multiple django-sites, memory usage
Gábor Farkas
how does it work exactly, when i have multiple django sites running in
one apache server using mod_python?
for example, imagine that i have 10 django sites running in one apache
server.
does that mean, that in every apache process, i have 10 python
interpreters loaded?
or in other words, if i have 10 django sites in one apache, does it use
10times more memory than when running only one django site?
thanks,
gabor
Graham Dumpleton
> hi,
>
> how does it work exactly, when i have multiple django sites running in
> one apache server using mod_python?
>
> for example, imagine that i have 10 django sites running in one apache
> server.
>
> does that mean, that in every apache process, i have 10 python
> interpreters loaded?
Yes.
> or in other words, if i have 10 django sites in one apache, does it use
> 10times more memory than when running only one django site?
More or less, yes.
If you tested that your code on top of Django and any third party
modules you use is multithread safe then you can limit the overal
memory used across all processes by using the 'worker' MPM for Apache
rather than the 'prefork' MPM. This helps because with 'worker' you
run with a lot less Apache child processes than with 'prefork'. The
number of Apache child processes can still spike up with 'worker' MPM
when there is the demand, but as demand drops off, Apache will kill
off excess processes. Even so, that each instance occupies memory in
each process still means a lot of memory overall.
If you need more control than that then you would need to explore
other options such as daemon mode of mod_wsgi or mod_fastcgi, which
both provide the ability to run each Django instance in a defined
number of distinct daemon processes. In the extreme case, you could
run Django on top of a Python WSGI server and run it in just a single
process behind Apache by proxying requests to it.
Memory use of running multiple Django instances under Apache using
mod_python is only one of the problems that can arise in this
instance. Other problems are the ability for the Django instances to
interfere with each other due to C extension modules which haven't
been written properly to be used from multiple sub interpreters at the
same time, or even attempts to use different versions of a C extension
module from the different Django instances. Using mod_wsgi or
mod_fastcgi may therefore be a better solution given they allow
separation of the Django instances.
So, the question is, what are you trying to achieve or want? Was there
a specific reason for the question? Knowing what you are really trying
to do, might be able to suggest others things you can read to learn
how Apache manages processes and how Python sub interpreters are used
in this scenario.
Graham
David Reynolds
On 24 Jul 2007, at 10:35 am, Graham Dumpleton wrote:
> Memory use of running multiple Django instances under Apache using
> mod_python is only one of the problems that can arise in this
> instance. Other problems are the ability for the Django instances to
> interfere with each other due to C extension modules which haven't
> been written properly to be used from multiple sub interpreters at the
> same time, or even attempts to use different versions of a C extension
> module from the different Django instances. Using mod_wsgi or
> mod_fastcgi may therefore be a better solution given they allow
> separation of the Django instances.
I'm glad you say this, as this has been happening to me and I wasn't
sure what was going on. We have about 38 sites running on mod_python/
apache and on odd occasions we get the wrong site appearing, although
a swift restart of apache seems to cure it. Presumably the more
sites we get, the worse this problem will get and it may be wise for
us to look into using mod_fastcgi in future.
Does anyone have any experience of using fastcgi with 30 odd sites?
Thanks,
David
--
David Reynolds
da...@reynoldsfamily.org.uk
Graham Dumpleton
wrote:
> On 24 Jul 2007, at 10:35 am, Graham Dumpleton wrote:
>
> > Memory use of running multiple Django instances under Apache using
> > mod_python is only one of the problems that can arise in this
> > instance. Other problems are the ability for the Django instances to
> > interfere with each other due to C extension modules which haven't
> > been written properly to be used from multiple sub interpreters at the
> > same time, or even attempts to use different versions of a C extension
> > module from the different Django instances. Using mod_wsgi or
> > mod_fastcgi may therefore be a better solution given they allow
> > separation of the Django instances.
>
> I'm glad you say this, as this has been happening to me and I wasn't
> sure what was going on. We have about 38 sites running on mod_python/
> apache and on odd occasions we get the wrong site appearing, although
> a swift restart of apache seems to cure it.
Seeing the completely wrong site is not something I would expect based
on the problems I have seen so far. Generally it is much more subtle
things causing strange Python exceptions. This doesn't mean it can't
happen, as have seen one instance of the wrong site appearing before.
All I could put it down to at the time was Apache not always selecting
virtual host definitions correctly because of a ServerAlias in one
VirtualHost overlapping with actual ServerName of another VirtualHost.
> Presumably the more
> sites we get, the worse this problem will get and it may be wise for
> us to look into using mod_fastcgi in future.
>
> Does anyone have any experience of using fastcgi with 30 odd sites?
Care to give mod_wsgi a go instead? :-)
Grahamd
Gábor Farkas
thanks for the information.
for various reasons, some of our django applications are running using
apache+fastcgi, and some using apache+mod_python. we were preferring
mod_python (because it's the "recommended" setup for django),
and only used the fastcgi-solution where we could not use mod_python
(again, various stupid reasons :)
but now we will probably have to add more django applications to the
system, so i was thinking if it should be done using mod_python or
fastcgi (i am sorry but i did not consider mod_wsgi, because i had the
feeling that it's still "too new". btw. would you consider it
production-ready? (i understand that it's a stupid question :))
so from what i've read here, it seems fastcgi/mod_wsgi-daemon-mode would
be the safest route (i've read the discussion about psycopg2+decimal on
django-devel).
btw. what do you think about having multiple apache-servers with
mod_python, each hosting only one django-application.
would that be an overkill?
p.s: the django-applications i am talking about are intranet-apps, so we
do not think there will be much load on them...
thanks,
gabor
Graham Dumpleton
> Graham Dumpleton wrote:
> > So, the question is, what are you trying to achieve or want? Was there
> > a specific reason for the question? Knowing what you are really trying
> > to do, might be able to suggest others things you can read to learn
> > how Apache manages processes and how Python sub interpreters are used
> > in this scenario.
>
> thanks for the information.
>
> for various reasons, some of our django applications are running using
> apache+fastcgi, and some using apache+mod_python. we were preferring
> mod_python (because it's the "recommended" setup for django),
> and only used the fastcgi-solution where we could not use mod_python
> (again, various stupid reasons :)
>
> but now we will probably have to add more django applications to the
> system, so i was thinking if it should be done using mod_python or
> fastcgi (i am sorry but i did not consider mod_wsgi, because i had the
> feeling that it's still "too new". btw. would you consider it
> production-ready? (i understand that it's a stupid question :))
There will hopefully be a 1.0 release candidate for mod_wsgi made
available in the next few days. Only reason it will not be a 1.0
release is that there is still a bit of extra documentation that I
want to finish up first, especially in the area of how web hosting
companies may make best use of it so we might finally get some good
cheap Python hosting available. So, releasing it as 1.0c1 rather than
1.0 is partly to try and defer large scale adoption a bit longer so I
can still find time to write the documentation instead of perhaps
answering questions from a lot of new users. :-)
> so from what i've read here, it seems fastcgi/mod_wsgi-daemon-mode would
> be the safest route (i've read the discussion about psycopg2+decimal on
> django-devel).
>
> btw. what do you think about having multiple apache-servers with
> mod_python, each hosting only one django-application.
>
> would that be an overkill?
>
> p.s: the django-applications i am talking about are intranet-apps, so we
> do not think there will be much load on them...
Nothing wrong with using multiple Apache instances. When one goes that
track the issue is more along the lines of the extra effort involved
in setting them up and maintaining them and any proxy in front. If the
load is indeed light and you can validate your applications are
multithread safe, then running each server with 'worker' MPM and 2
child processes would probably be a reasonable solution which still
keeps memory usage down thus allowing you to still run many servers on
the one box.
Graham
Damodhar
I,am new one to django,
I am very intrst to learn Django, so i install python latest verson,
and try to install daango latest verson but it want;s's to
configure .. mod_python module , can u help me
am using WINDOWS XP SYSTEM now am working in PHP MYSQL USING XAMPLITE
(APACHE +PHP + MYSQL Package)
i read the documentation for mod_python intallation but i cant able to
get ckear idea
wher do iwant to configure path apache\bin i found APXS file ...I
wwant to configure(write inside the apxs file ) or apache\conf\
httpd.config file... where do i want to write a configuration code ,
can u pls give me the details and code and were exactly write it
please give me the step by step.. to run django .
thanks
what
Atilla
mod_wsgi first. It's quite stable already, the setup is not any more
difficult than mod_fcgi and it's a lot more easy to configure the
finer details. Additionally, some simplistic tests show quite good
performance, although that's yet to be targeted with a more suitable
benchmark.