How to configure Apache MPM directives when using mod_wsgi in daemon mode ?

[Damien de Lemeny]

Note : this is a crosspost from ServerFault

Could you please help me understand some aspects about the relation between Apache MPM configuration directives and mod_wsgi's daemon mode ?

How to best configure apache mpm when serving only WSGI applications with WSGIDaemonProcess and WSGIRestrictEmbedded On ? (these mod_wgsi directives are already — and correctly AFAICT — configured)

Now what should I do with StartServers, Min/MaxSpareThreads and the other Apache level MPM configuration directives ?

As far as I understood, those are usually addressed to properly configure processes and threads in prefork/worker embedded mode, but what is their influence in daemon mode ?

[Graham Dumpleton]

Until I get a chance to response specifically about it, have you already watched:

http://lanyrd.com/2012/pycon/spcdg/

http://lanyrd.com/2013/pycon/scdyzk/

Graham

--
You received this message because you are subscribed to the Google Groups "modwsgi" group.
To unsubscribe from this group and stop receiving emails from it, send an email to modwsgi+u...@googlegroups.com.
To post to this group, send email to mod...@googlegroups.com.
Visit this group at http://groups.google.com/group/modwsgi.
For more options, visit https://groups.google.com/d/optout.

[Damien de Lemeny]

I did, I actually made some improvements on our previously bloated production setup based on your 2013 talk and a bit of guesswork, but I don't recall those giving a clear answer to my question.
You usually oppose apache MPM process/thread configurations in embedded modes to WSGIDaemonProcess process/thread configuration in daemon mode (as far as I understood, correct me if I'm wrong).
I am just not clearly understanding what should happen to the former directives when using the latter.
I don't really have a bottleneck to fix here, just trying to get a better understanding of how things work.

Thanks again for answering !

[Graham Dumpleton]

Just a warning in case you are wondering why you haven't got a response yet. I am on the tail end of a US trip and soon to fly home. This is why I haven't responded. I will get to this after I get back home and recover somewhat.

Graham

[Damien de Lemeny]

Okay, no hurry, no worries, I was assuming you either had a few days off or more important things to do. It's not like I have subscribed for 24/7 business service, I'll just be patient :)

[Graham Dumpleton]

Lets see if we can reboot this conversation.

What version of Apache are you using?

What version of mod_wsgi are you using?

What Apache MPM are you currently using and what are the Apache MPM directives for that MPM set to? For example:

# prefork MPM

# StartServers: number of server processes to start

# MinSpareServers: minimum number of server processes which are kept spare

# MaxSpareServers: maximum number of server processes which are kept spare

# MaxRequestWorkers: maximum number of server processes allowed to start

# MaxConnectionsPerChild: maximum number of connections a server process serves

#                         before terminating

<IfModule mpm_prefork_module>

    StartServers             5

    MinSpareServers          5

    MaxSpareServers         10

    MaxRequestWorkers      250

    MaxConnectionsPerChild   0

</IfModule>

# worker MPM

# StartServers: initial number of server processes to start

# MinSpareThreads: minimum number of worker threads which are kept spare

# MaxSpareThreads: maximum number of worker threads which are kept spare

# ThreadsPerChild: constant number of worker threads in each server process

# MaxRequestWorkers: maximum number of worker threads

# MaxConnectionsPerChild: maximum number of connections a server process serves

#                         before terminating

<IfModule mpm_worker_module>

    StartServers             3

    MinSpareThreads         75

    MaxSpareThreads        250

    ThreadsPerChild         25

    MaxRequestWorkers      400

    MaxConnectionsPerChild   0

</IfModule>

# event MPM

# StartServers: initial number of server processes to start

# MinSpareThreads: minimum number of worker threads which are kept spare

# MaxSpareThreads: maximum number of worker threads which are kept spare

# ThreadsPerChild: constant number of worker threads in each server process

# MaxRequestWorkers: maximum number of worker threads

# MaxConnectionsPerChild: maximum number of connections a server process serves

#                         before terminating

<IfModule mpm_event_module>

    StartServers             3

    MinSpareThreads         75

    MaxSpareThreads        250

    ThreadsPerChild         25

    MaxRequestWorkers      400

    MaxConnectionsPerChild   0

</IfModule>

Also what do you have set for the following Apache directives:

#

# Timeout: The number of seconds before receives and sends time out.

#

Timeout 60

#

# KeepAlive: Whether or not to allow persistent connections (more than

# one request per connection). Set to "Off" to deactivate.

#

KeepAlive On

#

# MaxKeepAliveRequests: The maximum number of requests to allow

# during a persistent connection. Set to 0 to allow an unlimited amount.

# We recommend you leave this number high, for maximum performance.

#

MaxKeepAliveRequests 100

#

# KeepAliveTimeout: Number of seconds to wait for the next request from the

# same client on the same connection.

#

KeepAliveTimeout 5

and finally, what are the actual mod_wsgi directives you are using and in particular the options to WSGIDaemonProcess directive.

Graham

[Damien de Lemeny]

> What version of Apache are you using?

Server version: Apache/2.2.22 (Debian)

> What version of mod_wsgi are you using?

3.3-4+deb7u1

> What Apache MPM are you currently using and what are the Apache MPM directives for that MPM set to?

Server MPM:     Worker

<IfModule mpm_worker_module>
    StartServers          2
    MinSpareThreads      25
    MaxSpareThreads      75
    ThreadLimit          64
    ThreadsPerChild      25
    MaxClients          150
    MaxRequestsPerChild   0

</IfModule>

> Also what do you have set for the following Apache directives:

Timeout 300
KeepAlive Off # Nginx acts as reverse proxy
MaxKeepAliveRequests 100 # distribution's default
KeepAliveTimeout 5 # distribution's default

> and finally, what are the actual mod_wsgi directives you are using and in particular the options to WSGIDaemonProcess directive.

WSGIRestrictEmbedded On
# In vhost :
WSGIScriptAlias / /projects/myapp/wsgi.py
WSGIDaemonProcess myapp processes=5 threads=5 umask=0002 python-path=/projects/myapp/virtualenv/lib/python2.7/site-packages
WSGIProcessGroup myapp

Sorry for the answering delay, I was on holiday.

[Graham Dumpleton]

I am on holiday this time, so expect slow and delayed responses from me for at least another week and a half.

Couple of quick questions.

Can you confirm you are in fact using nginx as a front end proxy to Apache/mod_wsgi as comment in configuration suggests and if so indicate what version of nginx and whether it is new enough to support HTTP/1.1 for proxying.

If you are using nginx as proxy, does all traffic go via nginx or are there circumstances that Apache/mod_wsgi is still serving requests direct for some reason.

Graham

[Damien de Lemeny]

Yes, nginx acts as a front-end proxy to Apache/mod_sgi. Version is 1.2.1

All _user_ traffic is served via nginx, apache/mod_wsgi is supposed not to handle a request directly from clients except for two cases :
 * exceptional request explicitely directed at the listened port (8000) emitted by a developer
 * automated local monitoring checks (GET /) which happen once per minute

Have a wonderful holiday ! :)

Damien