large scale deployment

[Damien Hunter]

I'm looking to deploy a digital sign-up/appointment sheet application
at a university. Can anyone give suggestions on the best setups to
handle sporadic use by 8,000 individuals?

Two major factors here are reliability and cost: meaning reliability
needs to be high, and cost needs to be low. However, the data is being
served on the same LAN as the clients.

[Damien Hunter]

To be more specific, let's ballpark that at any given second there are
40 users sending server requests. At rare peak times there may be
100+.

[Jason Roelofs]

Note: There's no such thing as a "best" setup. For your simple case, the following will do.

From my experience, you'll be fine with a single beefy server, a good HTTP server front-end (I highly recommend nginx) proxying off to mongrels (start with 10 or so, play around with the number to see what kind of performance you can sustain).

For my server setup this way, I easily pull 300 req / sec.

Jason

[Ericson Smith]

I'd suggest you do the following:

1. Get two servers, one for web and another for DB
2. Web server should have 4 gigs of ram at least and at least 4 cores (8 would be better)
3. DB well scaled and connected directly to web server
4. Go with Litespeed web server. Why? Because here you have only one tier to content with instead of ngix + mongrel + your app, or apache + mongrel + your app. Besides its super easy to setup.

We have that setup at http://www.funadvice.com and we get a few hundred thousand requests per day with typical server load of 0.3 even at peak hours of 3pm EST. One of the biggest things with Litespeed is that you no longer have to worry about keeping your mongrels up, scaling them, or having them not consume too much memory along with your app. Litespeed controls all that crap for you my gracefully restarting and scaling depending on traffic.

There's a free version, or an enterprise version to pick your poison.

- Ericson Smith
CTO
Funadvice.com

[Alexey Verkhovsky]

On Jan 4, 2008 7:51 AM, Ericson Smith <escon...@gmail.com> wrote:
> 4. Go with Litespeed web server.

Litespeed, as far as I have seen, works pretty well, but... Their free
version has this little downside:
"Multi-Processor Scalability: No"

If that means limiting the server to a single request handler thread
(what else could it be?), then you'd have to buy the non-free version
for production use. Correct me if I'm wrong.

--
Alexey Verkhovsky
CruiseControl.rb [http://cruisecontrolrb.thoughtworks.com]
RubyWorks [http://rubyworks.thoughtworks.com]

[Philip Hallstrom]

>> 4. Go with Litespeed web server.
>
> Litespeed, as far as I have seen, works pretty well, but... Their free
> version has this little downside:
> "Multi-Processor Scalability: No"
>
> If that means limiting the server to a single request handler thread
> (what else could it be?), then you'd have to buy the non-free version
> for production use. Correct me if I'm wrong.

Why? All depends on the amount of traffic. My memory is the original
poster wasn't going to have that many users (despite the subject) but I
could be wrong.

Sure, you'd get more from the production version, but the free should do
pretty well for quite a bit of traffic.

[Ericson Smith]

We use the free on another pretty large site with over a million uniques per month. Works pretty well for us.

- Ericson Smith
CTO
http://www.funadvice.com

[Alexey Verkhovsky]

On Jan 4, 2008 1:44 PM, Philip Hallstrom <ra...@philip.pjkh.com> wrote:
> > If that means limiting the server to a single request handler thread
> > (what else could it be?), then you'd have to buy the non-free version
> > for production use.

> Why? All depends on the amount of traffic.

Of course. However, on a quad core commodity hardware of today, this,
very roughly speaking, limits your throughput to one fourth of what
your server is otherwise capable of. Even less, if you have a separate
DB box. Ruby is a slow language, therefore Rails apps are almost
invariably CPU-bound. Assuming I understand the meaning of
"Multi-Processor Scalability: No" correctly.

> We use the free on another pretty large site with over a million uniques per month

Question is: how many dynamic page hits per second at peak load does
that translate to? Single core, with Rails == 10, maximum.

By the way, if you are going to handle one request at a time, anyway,
single Mongrel on port 80 works just fine, too.
Litespeed (paid edition) is good for handling multiple low traffic
sites (where you are bottlenecked by RAM, not CPU). It's not bad for
hosting a single application, either, as long as you are OK with
paying money for commercial close-sourced middleware, but a Mongrel
cluster is free and not all that hard to set up and run, either.

[Ericson Smith]

Really, it comes down to this... Does your site have a revenue model?

If it does, the $1299 (or $699 per year, or $65 per month) that Litespeed charges for the 4 CPU version, is chump change. If you run a database on the same server, then you will only need the 2 CPU version for $799 (since you need the other 2 CPU's for OS and database activity anyway). If you don't have a revenue model, then go with one of the free stacks instead.

I don't work for litespeed or anything, we just use their software (both free and paid versions). And really, we do have a revenue model (and are not funded), and after 25 years in the programming deployment webmaster biz, i really dont have time to waste with a stack that bombs out, so discovering litespeed a few months ago was a big boon for us.

I just feel sad when I hear people crying over deployment issues with rails when there is at least one decent solution out there available, and they just stick their heads in their sand with Apache + Mongrel or nginx + Mongrel, or Apache + fcgi or lighty + mongrel (all sheperded by monit). All those are pretty good solutions, and we've tried them all, none have come close to making it possible to pretty much relax about the web serving part of the deployment as litespeed has.

Zed brought us out of the fcgi ghetto with Mongrel. Amen to that. Its time that someone wrote a good mod_ruby for apache or something. That to me is where we need to head.

- Ericson

[Jason Roelofs]

How does litespeed compare to Swiftiply, if there's anyone who has used either / both? I am quite enjoying the nginx + Mongrel setup, but we are not yet to the point of heavy traffic and needing to scale to multiple servers and the like.

Jason

[Ericson Smith]

It seems that you still have to manually start Swiftiply processes as your traffic scales. You set the mins and maxes and litespeed does it for you. This is what I meant by not having to worry about the web serving deployment aspect of your application.

Swiftiply does still seem like the answer to some of the large issues with mongrel though. Based on my cursory reading of the site, it still seems that you will have the problem of stopping/starting/restarting processes though. Lets face it, our Rails apps will from time to time have memory leaks and crash their containers. You may be lucky to write the perfect app that does neither, but in my experience, this is a pretty common thing.

Having something that manages that aspect is a no brainer.

- Ericson Smith
CTO
http://www.funadvice.com

[Thibaut Barrère]

> 4. Go with Litespeed web server. Why? Because here you have only one tier to
> content with instead of ngix + mongrel + your app, or apache + mongrel +
> your app. Besides its super easy to setup.

+1 on this one. I don't work for Litespeed neither, but I really
recommend having a look at LiteSpeed (either standard or enterprise
edition), pretty much for the same reasons Ericson Smith describes.

I've seen LS overlooked by some because it was not open-source, but
really their software is great, simple to use and reliable.

hope this helps!

Thibaut Barrère / LoGeek
--
http://blog.logeek.fr - about writing software
http://evolvingworker.com - tools for a better day

[tmor...@gmail.com]

On Jan 4, 2:59 pm, "Ericson Smith" <esconsu...@gmail.com> wrote:

> It seems that you still have to manually start Swiftiply processes as your
> traffic scales. You set the mins and maxes and litespeed does it for you.
> This is what I meant by not having to worry about the web serving deployment
> aspect of your application.

The problem with auto start scaling is that it destabilizes the
environment. I don't mean that you'll crash more, but that your
performance will change over time, and sometimes in ways that you
might not predict.

For high volume sites, you want to bring up the system initially into
an optimal state, and then keep it there, IMHO.

Auto starting processes is good for low end stuff, but not really
applicable to high volume sites

> Swiftiply does still seem like the answer to some of the large issues with
> mongrel though. Based on my cursory reading of the site, it still seems that
> you will have the problem of stopping/starting/restarting processes though.
> Lets face it, our Rails apps will from time to time have memory leaks and
> crash their containers. You may be lucky to write the perfect app that does
> neither, but in my experience, this is a pretty common thing.
>
> Having something that manages that aspect is a no brainer.

You can use monit or god to monitor those processes. One advantage of
this method is that you only need to make sure your process monitor
starts automatically at startup, and then it will handle starting the
application servers too. This is, of course, exactly the same behavior
as LightSpeed, so I'm not claiming advantages here.

We'll soon be releasing an open source plug-in for nginx called "nginx
fair proxy" that using a much improved proxy algorithm that limits the
number of connections nginx will open against mongrel. We've seen far
better performance, and much less mongrel bloat on high volume sites
during our testing.

nginx + mongrel ++ (with or without evented mongrel)

Also, take a look at merb with datamapper for applications that need
really high performance.

[Raymond O'Connor]

Regarding litespeed and the multiprocessor scalability: I do notice it
spawns several ruby processes when you start the server. As far as I can
tell, there's nothing in the free version that keeps these ruby
processes from being run on multiple cores... So I think you do get
some advantage with multiple cores even with the free version. Correct
me if I'm wrong though...

But just to echo what everyone else has been saying, our company has had
great success with the litespeed server. We've been using it for about
6 months now with several rails apps and have had no problems. And
whenever I have a question, their support people on their forum usually
respond within a few hours with an answer.

-Ray

--
Posted via http://www.ruby-forum.com/.

[Ericson Smith]

That is correct. The ruby processes actually will run on multiple cores. Its only the dispatcher that looks and traffic and dishes out to the ruby lsapi processes that actually runs on one core.

For tmor...@engineyard.com:
Of course unlimited scaling will destabilize your environment. That's why you do some basic investigations and set your min and max values for spawned lsapi children. Its the same issue if the startup 20 mongrels on a dual core machine. The point is if I know that my theorerical maximums are 10 mongrels (lsapi clients). Then once I set that, I no longer have to worry about it. Litespeed will put them up or tear them down -- up to the maximums I set, or down to the minimums -- based on traffic, memory and load. So destabilizing your environment is not an issue. After all Apache does the same thing governed my max child clients.

And the gravy on the steak? You don't need monit for all of that.

- Ericson Smith
CTO
http://www.funadvice.com

[ChrisR]

Does multiprocessor mean it will only use one physical processor or
just one core? If it's just one processor then a quad core would be
fine.

Chris

On Jan 5, 11:41 am, Raymond O'Connor <ruby-forum-incom...@andreas-

[Ericson Smith]

There is one parent process that starts up child processes. The parent process is somewhat akin to a proxy (apache etc) dispatching requests to multiple mongrels. The parent process for the free version runs on 1 CPU. However each child process will run on its own processor.

I believe litespeed says the free version can handle up to 150 requests per second, which means you can handle nearly 13 million hits per day with decent hardware. Pretty good for the free version I think.

- Ericson Smith
CTO
http://www.funadvice.com

[Philip Hallstrom]

> I believe litespeed says the free version can handle up to 150 requests per

150 *simultaneous* *connections*. Very different from 150 r/s.

But yeah, it's still plenty for a lot of sites out there.

[tmor...@gmail.com]

On Jan 5, 6:02 am, "Ericson Smith" <esconsu...@gmail.com> wrote:

> Of course unlimited scaling will destabilize your environment. That's why
> you do some basic investigations and set your min and max values for spawned
> lsapi children. Its the same issue if the startup 20 mongrels on a dual core
> machine.

No, it's decidedly not the same. If you startup 20 mongrels on a dual
core machine, then the performance will be the same whether those 20
are in use or not. It's not an issue of CPU, to a large extent, but of
memory...

> The point is if I know that my theorerical maximums are 10 mongrels
> (lsapi clients). Then once I set that, I no longer have to worry about it.

But why wouldn't you want all 10 running if 10 is the maximum?

> Litespeed will put them up or tear them down -- up to the maximums I set, or
> down to the minimums -- based on traffic, memory and load.

Why would you pay the penalty of processes startup and teardown?

> So destabilizing your environment is not an issue.

Sure it is. What happens if you have high traffic during a cron job?
You'll have max mongrels running, using all available RAM with enough
left over to allow the machine to run efficiently. The cron job starts
and *bam* you're in swap.

Now, I know what you're going to say: You've set the maximum too high.
Yes, I agree with that technically, but in practice, a LOT of people
forget about daily, weekly, and monthly cron jobs, etc.

Also, people tend to log into production machine and run code for
various purposes, sometimes as part of routine. With fluctuating
number of mongrel processes, this can work for a long time, then
suddenly take the machine into swap for no apparent reason.

Technically, you're 100% right, all of this can be avoided with your
method. Operationally, I'm quite convinced it's not a good idea. :-)

> After all Apache does the same thing governed my max child clients.

Yes, and in my experience large websites don't use this feature for
the reasons I've mentioned above. :-)

> And the gravy on the steak? You don't need monit for all of that.

But you do need monit to monitor lightspeed itself, so you've gained
nothing. :-)

[Ericson Smith]

Lets start with the last thing u said. Litespeed has basically the same kind of stability of apache. Do you really need monit to monitor apache? Maybe if you're seriously having some kind of major app that will break the underlying web server, then yes, but for 99% of us, no.

I think what you're forgetting I can also set litespeed to adjust based on memory AND process limits per user.

I can set 2 min and 10 max and forget about it. Litespeed adds more if necessary, then based on capacity, gracefully ramps down if I dont need them, thus leaving the system for other things. A great example is if I'm running CRON's in the middle of the night, this is when my traffic is low. Why would I want 10 instances running? I can use the resources used from not having 8 useless instances running doing nothing.

My point in all of this is not to knock the great open source software out there which I have used and will continue to do so, but just to point out that Litespeed is an extremely credible contender for Rails. It really makes it super simple to get going with top-notch performance as a great side bonus. For guys like me who want to spend time coding and dealing with the strategic issues on my site (our core competence), Litespeed is a godsend.

- Ericson Smith
CTO
http://www.funadvice.com

On Jan 5, 2008 3:38 PM, tmor...@engineyard.com <tmor...@gmail.com> wrote:

[Alexey Verkhovsky]

On Jan 5, 2008 1:38 PM, tmor...@engineyard.com <tmor...@gmail.com> wrote:
> We'll soon be releasing an open source plug-in for nginx called "nginx
> fair proxy" that using a much improved proxy algorithm that limits the
> number of connections nginx will open against mongrel.

That would be awesome.

[Philip Hallstrom]

>> The point is if I know that my theorerical maximums are 10 mongrels
>> (lsapi clients). Then once I set that, I no longer have to worry about it.
>
> But why wouldn't you want all 10 running if 10 is the maximum?

Not as useful for consistently large traffice sites, but to take it
another direction, I love litespeed for the fact that I can run several
Rails sites on my little 1gb box that also does postfix/postgresql/mysql
and all sorts of other things and not have to have 6-7 mongrel processes
running constantly. The sites are so small that most of the time there
aren't any lsapi processes running.

True, this isn't large scale, but it's a nice feature :)

I also suspect that we all may have different ideas of large scale :)

>> And the gravy on the steak? You don't need monit for all of that.
>
> But you do need monit to monitor lightspeed itself, so you've gained
> nothing. :-)

What monitors monit? :)