how many Starman workers do you run?

[Kevin Goess]

We just did some benchmarking with Starman on an eight-core box, against an application page with light database and memcached access and some Template Toolkit templates, a light page but not a no-op by any means, and it looks like the best configuration for that load is 20 workers.  At that rate they can handle 190 requests per second.  At that point apache-status shows all the workers busy all the time.  Raising the number of workers never raises the number of requests past 190, but does raise the load on the box substantially as the extra workers have to wait around for the kernel to give them their timeslice.

So we have that figure of 20 workers which seems low to me.  If we have a bunch of clients who decide to be slow about accepting their request packets, they could easily use up all 20 slots.

Is that analysis about right?  Anybody else on the list have a suggestion or a different configuration that would help address those concerns?



--
Kevin M. Goess
Software Engineer
Berkeley Electronic Press
kgo...@bepress.com

510-665-1200 x179
www.bepress.com

bepress: sustainable scholarly publishing  

[Tatsuhiko Miyagawa]

Hi, 

On Jan 25, 2013, at 5:41 PM, Kevin Goess <kgo...@bepress.com> wrote:

We just did some benchmarking with Starman on an eight-core box, against an application page with light database and memcached access and some Template Toolkit templates, a light page but not a no-op by any means, and it looks like the best configuration for that load is 20 workers.  At that rate they can handle 190 requests per second.  At that point apache-status shows all the workers busy all the time.  Raising the number of workers never raises the number of requests past 190, but does raise the load on the box substantially as the extra workers have to wait around for the kernel to give them their timeslice.

You saw the worker status with apache-status...? Does it mean you have Apache frontend and use mod_proxy or some sort to proxy through starman workers?

So we have that figure of 20 workers which seems low to me.  If we have a bunch of clients who decide to be slow about accepting their request packets, they could easily use up all 20 slots.

See my question above, but if you run apache (or other frontend servers such as nginx or lighttpd) in front, slow clients should not be an issue because they are handled in frontend before reaching Starman workers, as long as you have higher maximum connections. 

Once your client connection number gets beyond the number of workers, they will wait for the workers to complete processing, but that sounds like fairly fast based on your numbers.

If your request really gets beyond what your workers can handle, you should just make another cluster of workers, usually just a new box. One option to make that work nicely would be to use nginx/apache's mod_proxy_balancer with lower backlog option with Starman, so that frontend will give up soon when all the workers are busy and then try another cluster.

I believe that the backlog option behavior is documented in man starman. You can also handle that kind of load balancing with more on the frontend side as well.

Is that analysis about right?  Anybody else on the list have a suggestion or a different configuration that would help address those concerns?

--
Kevin M. Goess
Software Engineer
Berkeley Electronic Press
kgo...@bepress.com

510-665-1200 x179
www.bepress.com

bepress: sustainable scholarly publishing  

--
 
 
 

[Kevin Goess]

On Fri, Jan 25, 2013 at 5:52 PM, Tatsuhiko Miyagawa <miya...@gmail.com> wrote:

if you run apache (or other frontend servers such as nginx or lighttpd) in front, slow clients should not be an issue because they are handled in frontend before reaching Starman workers, as long as you have higher maximum connections. 

Once your client connection number gets beyond the number of workers, they will wait for the workers to complete processing, but that sounds like fairly fast based on your numbers.

Ah, thanks, Miyagawa, that's really helpful (and "hi" btw).  We were wondering whether we should put Varnishes (or maybe nginx) in front of our Starman nodes or just connect the Starmans directly to our external load balancer.   Your explanation clarified/confirmed that the former is the way to go, so that the Varnishes can absorb the slow connections

    client-----[maybe slow]----->varnish pool-----[always fast]---->starman

Oh, and I should have said, we're using Plack::Middleware::ServerStatus::Lite for the server-status page.  And yes, we're currently running Starman behind mod_proxy, but we're looking at shifting stuff around.

--