Proper cherrypy config to run multiple sites on same server

[Benj]

Hello,

I'm going to use cherrypy on a multicore dedicated server, that will host 15 different sites.

What's the proper way to configure cherrypy to take advantage of the available power ?

1 Cherrypy process for all the sites (with thread pool taking advantage of multi core)

or

1 Cherrypy process for each site ?

or

1 Cherrypy process per core ?

I want to keep things simple, just like in the old days where you had LAMP on shared server serving 100 sites on a single server.

Please advise, thanks.

[David Allouche]

> On 2 Nov 2016, at 13:03, Benj <webk...@gmail.com> wrote:
>
> Hello,
>
> I'm going to use cherrypy on a multicore dedicated server, that will host 15 different sites.
> What's the proper way to configure cherrypy to take advantage of the available power ?
>
> 1 Cherrypy process for all the sites (with thread pool taking advantage of multi core)
> or

Python thread pools cannot use multiple cores effectively because of the Python GIL

> 1 Cherrypy process for each site ?
> or
> 1 Cherrypy process per core ?

1 server process per core is right.

[Benj]

Thanks. What if I replace the cherrypy server with phusionpassenger ?

has this been tried before ?

[Johan Hartzenberg]

> 1 Cherrypy process for all the sites (with thread pool taking advantage of multi core)
> or

Python thread pools cannot use multiple cores effectively because of the Python GIL

I have heard this alluded to before.  I would like to understand this limitation better - where can I read about it and about what can and cannot be done about it?

 

> 1 Cherrypy process for each site ?
> or
> 1 Cherrypy process per core ?

1 server process per core is right.

Are you saying one process per site and one process per core?  That would assume that every "site" is equally busy and that no site ever needs more than one core.  I would start by saying one process per SITE and then tune the number of threads based on the load each site gets, but this may run into these Python-WSGI serialisation and/or Python GIL thread-pool issues... In that case you need to start looking at multiple processes per SITE for the sites with a heavier demand.

[Andrew Maizels]

The easy solution is to write and test your application using CherryPy's built-in web server, and then deploy using an application server like Green Unicorn or uWSGI.

You just need a few lines of wrapper code to tell the server how to run your app, and the server will take care of maintaining the necessary processes and threads.

--
You received this message because you are subscribed to the Google Groups "cherrypy-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cherrypy-users+unsubscribe@googlegroups.com.
To post to this group, send email to cherrypy-users@googlegroups.com.
Visit this group at https://groups.google.com/group/cherrypy-users.
For more options, visit https://groups.google.com/d/optout.

--

Ahh... We are all heroes; you and Boo and I.
Hamsters and rangers everywhere, rejoice!
Read Peopleware! http://www.amazon.com/Peopleware-Productive-Projects-Teams-Second/dp/0932633439

[Thibault Nélis]

On Sat, 2016-11-05 at 13:35 -0700, Johan Hartzenberg wrote:
> > > 1 Cherrypy process for all the sites (with thread pool taking
> > advantage of multi core) 
> > > or 
> >
> > Python thread pools cannot use multiple cores effectively because
> > of the Python GIL 
> >
> I have heard this alluded to before.  I would like to understand this
> limitation better - where can I read about it and about what can and
> cannot be done about it?

This presentation by David Beazley at the Chicago Python User Group is
quite comprehensive and entertaining (June 2009):

https://www.youtube.com/watch?v=ph374fJqFPE

Threads cannot execute Python code simultaneously, so they're mostly
used to simplify I/O bound concurrent tasks (like handling HTTP
requests).  C extensions can release the GIL and take advantage of
multiple cores however, which is where the vagueness of some statements
regarding the GIL comes from (i.e. some modules can take advantage of
multiple cores, some can't).

If your system is CPU-bound, do use one process per core.  Using an
application server to do that is a good idea.

-t