Account Options

  1. Sign in
The old Google Groups will be going away soon, but your browser is incompatible with the new version.
Google Groups Home
« Groups Home
How do I use something like multiprocessing.cpu_count() * 2 + 1 in an INI file?
There are currently too many topics in this group that display first. To make this topic appear first, remove this option from another topic.
There was an error processing your request. Please try again.
flag
  6 messages - Collapse all  -  Translate all to Translated (View all originals)
The group you are posting to is a Usenet group. Messages posted to this group will make your email address visible to anyone on the Internet.
Your reply message has not been sent.
Your post was successful
 
From:
To:
Cc:
Followup To:
Add Cc | Add Followup-to | Edit Subject
Subject:
Validation:
For verification purposes please type the characters you see in the picture below or the numbers you hear by clicking the accessibility icon. Listen and type the numbers you hear
 
Zak  
View profile  
 More options Sep 26 2012, 2:04 pm
From: Zak <zakdan...@gmail.com>
Date: Wed, 26 Sep 2012 11:04:36 -0700 (PDT)
Local: Wed, Sep 26 2012 2:04 pm
Subject: How do I use something like multiprocessing.cpu_count() * 2 + 1 in an INI file?

I'm trying to setup Gunicorn to use with my Pyramid app. I want to set the
number of workers as "workers = multiprocessing.cpu_count() * 2 + 1" but
obviously "multiprocessing" is a python module and won't run in
development.ini/production.ini. If I use a separate config file for
Gunicorn, then the "pserve development.ini" command won't work because then
Gunicorn has no way to locate the config file. Is there a way I can either
use cpu_count() in the INI file or put a path to the config file for
Gunicorn in the INI files?


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Whit Morriss  
View profile  
 More options Sep 26 2012, 2:08 pm
From: Whit Morriss <w...@surveymonkey.com>
Date: Wed, 26 Sep 2012 18:08:32 +0000
Local: Wed, Sep 26 2012 2:08 pm
Subject: Re: How do I use something like multiprocessing.cpu_count() * 2 + 1 in an INI file?

You need to create your own starter script to do that calculation.  Something like this (but that starts gunicorn w/. the # of workers you want)::

https://github.com/whitmo/CheesePrism/blob/master/runapp.py

-w

On Sep 26, 2012, at 1:04 PM, Zak <zakdan...@gmail.com<mailto:zakdan...@gmail.com>>

 wrote:

I'm trying to setup Gunicorn to use with my Pyramid app. I want to set the number of workers as "workers = multiprocessing.cpu_count() * 2 + 1" but obviously "multiprocessing" is a python module and won't run in development.ini/production.ini. If I use a separate config file for Gunicorn, then the "pserve development.ini" command won't work because then Gunicorn has no way to locate the config file. Is there a way I can either use cpu_count() in the INI file or put a path to the config file for Gunicorn in the INI files?

--
You received this message because you are subscribed to the Google Groups "pylons-discuss" group.
To view this discussion on the web visit https://groups.google.com/d/msg/pylons-discuss/-/sh430wysAXYJ.
To post to this group, send email to pylons-discuss@googlegroups.com<mailto:pylons-discuss@googlegroups.com>.
To unsubscribe from this group, send email to pylons-discuss+unsubscribe@googlegroups.com<mailto:pylons-discuss+unsubscri be@googlegroups.com>.
For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en.


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Zak  
View profile  
 More options Sep 26 2012, 2:45 pm
From: Zak <zakdan...@gmail.com>
Date: Wed, 26 Sep 2012 11:45:09 -0700 (PDT)
Local: Wed, Sep 26 2012 2:45 pm
Subject: Re: How do I use something like multiprocessing.cpu_count() * 2 + 1 in an INI file?

So if you can run custom scripts beforehand with a runapp.py file, then
what exactly does the "pserve development.ini" command do? Does it just
start running everything with default settings?


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Whit Morriss  
View profile  
 More options Sep 26 2012, 3:07 pm
From: Whit Morriss <w...@surveymonkey.com>
Date: Wed, 26 Sep 2012 19:07:04 +0000
Local: Wed, Sep 26 2012 3:07 pm
Subject: Re: How do I use something like multiprocessing.cpu_count() * 2 + 1 in an INI file?

On Sep 26, 2012, at 1:45 PM, Zak <zakdan...@gmail.com>

 wrote:
> So if you can run custom scripts beforehand with a runapp.py file, then what exactly does the "pserve development.ini" command do? Does it just start running everything with default settings?

yeah more or less.  It uses the classic paste defaults (app:main, server:main, etc) and gives you some interpolation options.  At it's core, it's doing roughly the same thing… calling loadapp and then handing the result to a server you've defined in your ini.

The compromise (perhaps virtuous), is that you can do any computation in the ini.  pserve does give you interpolation so you could accomplish the same thing by something like this::

in your ini:
```
workers=%(workers)s
```

on the cmdline:

$ pserve dev.ini workers=`python -c 'import multiprocessing as m; print m.cpu_count() * 2 + 1`

But at that point (or close to it), might as well write a runner…

in general pserve offers some conveniences like the reload flag and allows for a generalized way for running an app from **solely** an ini file.  

The philosophy behind paste and paste script (ala paster from which pserve is derived) is to give operations people a unified interface for running wsgi applications whereas as custom runners may be unpredictably heterogenous as the people who wrote them.

opinions stated here are my own… Chris and others might have more correct or nuance opinions.

-w


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Whit Morriss  
View profile  
 More options Sep 26 2012, 3:08 pm
From: Whit Morriss <w...@surveymonkey.com>
Date: Wed, 26 Sep 2012 19:08:09 +0000
Local: Wed, Sep 26 2012 3:08 pm
Subject: Re: How do I use something like multiprocessing.cpu_count() * 2 + 1 in an INI file?

> The compromise (perhaps virtuous), is that you can do any computation in the ini.  pserve does give you interpolation so you could accomplish the same thing by something like this::

s/can/can't/

-w


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Zak  
View profile  
 More options Sep 26 2012, 7:28 pm
From: Zak <zakdan...@gmail.com>
Date: Wed, 26 Sep 2012 16:28:34 -0700 (PDT)
Local: Wed, Sep 26 2012 7:28 pm
Subject: Re: How do I use something like multiprocessing.cpu_count() * 2 + 1 in an INI file?

Cool, I think I understand much better now, thanks. I wish there was more
examples of a Gunicorn-powered runnapp.py...they seem to be in short supply.


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
End of messages
« Back to Discussions « Newer topic     Older topic »