Figuring out prefork v. worker

[Doug Van Horn]

I run several applications on a Slicehost VPS and I recently switched
them to run the worker MPM and mod_wsgi.

After switching, I ran into an issue with the worker MPM leaking
timezone information across django applications (actually the timezone
leaks under both mod_python and mod_wsgi/embedded mode).

That leak prompted me to put mod_wsgi into daemon mode
(WSGIDaemonProcess) to prevent the timezone leak, but with that mode I
ended up with an apache process per daemon, which seemed to defeat the
whole purpose of moving to worker (low process count).

So now I'm back to, apparently, running apache under the prefork MPM,
as I'll get the protection between django apps without having to run a
process per site.

I was just wondering if anyone else had been through this cycle? My
testing showed I didn't have any issues with mod_wsgi/prefork leaking
info across apps, but maybe someone else has?

I'm pretty much thinking I shouldn't have bothered with all this fancy
worker and wsgi stuff and just stuck with the prefork/mod_python
configuration. After all, I'm running business applications, not
social networking sites.

[Graham Dumpleton]

On Oct 17, 7:14 am, Doug Van Horn <dougvanh...@gmail.com> wrote:
> I run several applications on a Slicehost VPS and I recently switched
> them to run the worker MPM andmod_wsgi.
>
> After switching, I ran into an issue with the worker MPM leaking
> timezone information across django applications (actually the timezone

> leaks under both mod_python andmod_wsgi/embedded mode).
>
> That leak prompted me to putmod_wsgiinto daemon mode

> (WSGIDaemonProcess) to prevent the timezone leak, but with that mode I
> ended up with an apache process per daemon, which seemed to defeat the
> whole purpose of moving to worker (low process count).

But a small number of Apache worker MPM child processes and a single
or small number of mod_wsgi daemon processes is much better than the
huge number of Apache prefork MPM child processes that would be
required to handle same load.

What is the mod_wsgi configuration you are using and how many
processes/threads are you using for Apache child processes and
mod_wsgi daemon processes?

> So now I'm back to, apparently, running apache under the prefork MPM,
> as I'll get the protection between django apps without having to run a
> process per site.

Depending on how timezone stuff is being set, prefork MPM will not
protect you.

This is because a timezone change normally has to be done at C
environment variable level, thus outside of the context of Python
interpreters. If the timezone is set only at application first
initialisation, whichever application initialises last will override
timezone settings of all others.

If timezone is set on every request, then it may work, but still
wouldn't if any background threads were created to do stuff which were
dependent on timezone setting.

Although Django adapter for mod_python updates os.environ on every
request from Apache per request environment (something which isn't
particularly safe to do), no such thing is done with WSGI adapter for
Django. So, if timezone being set via Apache configuration and updated
on each request, might work with mod_python on prefork MPM, but not on
worker MPM, and also not with mod_wsgi.

> I was just wondering if anyone else had been through this cycle?  My

> testing showed I didn't have any issues withmod_wsgi/prefork leaking

> info across apps, but maybe someone else has?
>
> I'm pretty much thinking I shouldn't have bothered with all this fancy
> worker and wsgi stuff and just stuck with the prefork/mod_python
> configuration.  After all, I'm running business applications, not
> social networking sites.

I don't see how business applications vs social networking sites makes
any difference. Care to explain?

Graham

[jo...@fairviewcomputing.com]

On Oct 16, 2008, at 4:14 PM, Doug Van Horn wrote:

> I was just wondering if anyone else had been through this cycle? My
> testing showed I didn't have any issues with mod_wsgi/prefork leaking
> info across apps, but maybe someone else has?
>
> I'm pretty much thinking I shouldn't have bothered with all this fancy
> worker and wsgi stuff and just stuck with the prefork/mod_python
> configuration. After all, I'm running business applications, not
> social networking sites.

Yeah, actually I had the same problem with timezones in embedded mode,
and some other weird problems with psycopg2, too. Daemon mode fixed
all of them, and my experience bears out what Graham said: this setup
handles more load than prefork with fewer processes and less memory,
and the footprint's more predictable. There are other benefits, like
the ability to run different sites as different users, but that was my
main concern.

[Doug Van Horn]

On Oct 16, 5:31 pm, Graham Dumpleton <Graham.Dumple...@gmail.com>
wrote:

> On Oct 17, 7:14 am, Doug Van Horn <dougvanh...@gmail.com> wrote:

> What is the mod_wsgi configuration you are using and how many
> processes/threads are you using for Apache child processes and
> mod_wsgi daemon processes?

For what it's worth, here's what I ended up settling on:

MPM Worker (mod_worker)

WSGIDaemonProcess foo maximum-requests=1000 stack-size=524288
threads=10
WSGIProcessGroup foo

> I don't see how business applications vs social networking sites makes
> any difference. Care to explain?

I was trying to convey the fact that my applications aren't oriented
around thousands or more users as a typical social site would likely
be, or at least hope to be.

doug.