How do I use something like multiprocessing.cpu_count() * 2 + 1 in an INI file?

[Zak]

I'm trying to setup Gunicorn to use with my Pyramid app. I want to set the number of workers as "workers = multiprocessing.cpu_count() * 2 + 1" but obviously "multiprocessing" is a python module and won't run in development.ini/production.ini. If I use a separate config file for Gunicorn, then the "pserve development.ini" command won't work because then Gunicorn has no way to locate the config file. Is there a way I can either use cpu_count() in the INI file or put a path to the config file for Gunicorn in the INI files?

[Whit Morriss]

You need to create your own starter script to do that calculation.  Something like this (but that starts gunicorn w/. the # of workers you want)::

https://github.com/whitmo/CheesePrism/blob/master/runapp.py

-w

On Sep 26, 2012, at 1:04 PM, Zak <zakd...@gmail.com>

 wrote:

I'm trying to setup Gunicorn to use with my Pyramid app. I want to set the number of workers as "workers = multiprocessing.cpu_count() * 2 + 1" but obviously "multiprocessing" is a python module and won't run in development.ini/production.ini. If I use a separate config file for Gunicorn, then the "pserve development.ini" command won't work because then Gunicorn has no way to locate the config file. Is there a way I can either use cpu_count() in the INI file or put a path to the config file for Gunicorn in the INI files?

--
You received this message because you are subscribed to the Google Groups "pylons-discuss" group.
To view this discussion on the web visit https://groups.google.com/d/msg/pylons-discuss/-/sh430wysAXYJ.
To post to this group, send email to pylons-...@googlegroups.com.
To unsubscribe from this group, send email to pylons-discus...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en.

[Zak]

So if you can run custom scripts beforehand with a runapp.py file, then what exactly does the "pserve development.ini" command do? Does it just start running everything with default settings?

[Whit Morriss]

On Sep 26, 2012, at 1:45 PM, Zak <zakd...@gmail.com>
wrote:

> So if you can run custom scripts beforehand with a runapp.py file, then what exactly does the "pserve development.ini" command do? Does it just start running everything with default settings?

yeah more or less. It uses the classic paste defaults (app:main, server:main, etc) and gives you some interpolation options. At it's core, it's doing roughly the same thing… calling loadapp and then handing the result to a server you've defined in your ini.

The compromise (perhaps virtuous), is that you can do any computation in the ini. pserve does give you interpolation so you could accomplish the same thing by something like this::

in your ini:
```
workers=%(workers)s
```

on the cmdline:

$ pserve dev.ini workers=`python -c 'import multiprocessing as m; print m.cpu_count() * 2 + 1`

But at that point (or close to it), might as well write a runner…

in general pserve offers some conveniences like the reload flag and allows for a generalized way for running an app from **solely** an ini file.

The philosophy behind paste and paste script (ala paster from which pserve is derived) is to give operations people a unified interface for running wsgi applications whereas as custom runners may be unpredictably heterogenous as the people who wrote them.

opinions stated here are my own… Chris and others might have more correct or nuance opinions.

-w

[Whit Morriss]

>
> The compromise (perhaps virtuous), is that you can do any computation in the ini. pserve does give you interpolation so you could accomplish the same thing by something like this::

s/can/can't/

-w

[Zak]

Cool, I think I understand much better now, thanks. I wish there was more examples of a Gunicorn-powered runnapp.py...they seem to be in short supply.