Will RoR Scale?

[-kinetic]

I'm trying to decide which path to go down for developing a new web
app. Scalability in the long run is a huge requirement, but without
sacrificing the development time.

The rails community is huge, there's support everywhere, documentation
is fantastic, but I keep hearing things from others about how if we
use rails we're not going to be able to scale.

Some have said Rails is great for prototyping, but going to production
it's going to fail. The largest sites out there never use Rails. I
believe Twitter tried and failed. I'm not sure what they ended up
doing though.

I really want to develop using RoR, but I'm hesitant because of the
scale factor.

Can anyone give me some information concerning this?

Thanks!

[Frederick Cheung]

On 8 Feb 2008, at 21:22, -kinetic wrote:

>
> I'm trying to decide which path to go down for developing a new web
> app. Scalability in the long run is a huge requirement, but without
> sacrificing the development time.
>
> The rails community is huge, there's support everywhere, documentation
> is fantastic, but I keep hearing things from others about how if we
> use rails we're not going to be able to scale.
>
> Some have said Rails is great for prototyping, but going to production
> it's going to fail. The largest sites out there never use Rails. I
> believe Twitter tried and failed. I'm not sure what they ended up
> doing though.

Twitter is still using rails.

Fred

[lunaclaire]

also large sites like Revolution Health are using rails

I think that if you're smart about your use of the DB, if you use
caching, etc you can get pretty far.

...but I also wonder about this

[-kinetic]

Do you think the troubles Twitter were realizing was just a well
advertise demise while all the others go through just as much scaling
problems?

In short, would scaling be less of a problem with a PHP framework like
Symfony?

I think the reason why not many of the larger sites are using RoR is
because it's so new. And twitter was the newest one to go big, and
because of the quick development, maybe they staggered into the
scaling problems early?

I've read a ton of articles this afternoon, and they all say how much
more processing power Ruby uses, and that scaling is seemingly more
complicated.

[Lionel Bouton]

-kinetic wrote:
>
> I've read a ton of articles this afternoon, and they all say how much
> more processing power Ruby uses, and that scaling is seemingly more
> complicated.
>

Usually people writing these articles never had any real application to
scale themselves.

Sure, some applications might have better performance on a single box
with Java in a micro-benchmark because they are CPU bound and Java is
much faster than Ruby (now) for CPU intensive processing but this has
nothing to do with scaling as most people mean it which usually is : "go
beyond the point where your application can fit on a single server and
get more performance by adding pizza boxes".

So saying that an application framework scales more than another is pure
bulls**t. Every modern web framework uses a "share-nothing" approach to
solve scalability problems so about anyone can scale by adding
application servers today.

When anybody comes to the point where the only shared component becomes
a bottleneck (usually the relational database but might be filesystems,
LDAP servers, network, whatever) scaling is a matter of architecture and
solving the bottleneck is specific to the application not a problem the
framework can come with a solution for.

The problem is that the "share-nothing" term is misleading: you always
share something and this is what comes back to bite you when your
application is under stress. You'll need one(/several) good
specialist(s) to find the bottlenecks and re-architecture your
application when the problem arises.

For example, I recently had to spend some time optimizing some parts of
one Ruby process that was getting slower. After I finished, on the same
hardware the application was 10 times faster. Turns out that some
database queries weren't optimal, the database lacked some indexes and
PostgreSQL 8.0 was much slower than 8.2. Ruby didn't have anything to do
with it. I'm pretty confident that even if I spent the rest of my life
recoding this application from Ruby to raw assembler I wouldn't get more
than a 10% speedup and a ton of bugs.

In the software industry believing that software in a shiny box will
solve your problems is amateurism: people solve your high-level problems
using tools, the tools are only meant to low-level ones like minimizing
the time spent coding simple tasks and providing easy access to lots of
libraries from which to extend your application as quickly as possible
(in fact most of them probably create more problems than they solve :-)
). So if you have problems to solve, I suggest you look for bright
people who know how to solve them, not bright software, as bright
software is only a marketing myth...

Whow, looking at what I just typed it seems I was tired of coding and
needed to vent a bit :-)

Lionel

[DyingToLearn]

The fact that Ruby uses more processing power is actually irrelevant
to the scalability of a RoR app. Scalability is "how effectively the
system architecture can grow in proportion to increasing demand." (The
Rails Way by Obie Fernandez). The important thing is your algorithms,
not the language they are written in.

If your algorithms are well written, then you can hopefully just
double the number of servers when you double your users. This isn't
always possible, but that has nothing to do with Ruby and everything
to do with the complexity of the business problem at hand.

So if you write it in PHP, you might need to buy a new $1000 server
for every 100 new users. If you write it in Ruby, then you might need
to spend $1200 (or even $2000, but not likely) for the server that
supports 100 users. So if you have 1,000,000 users, in PHP you'd need
100 servers X $1000 equals $100,000. In RoR you'd need 100 servers X
$1200 equals $120,000. You spend more on hardware, but far less on
software development and maintenance.

I think part of why people still wonder if Rails can scale is that
there are many parts of ActiveRecord that put simplicity before
performance. There are 1000s of projects using rails that don't need
high performance so I think this is a great decision. It is also a
rails convention (or maybe best practice) to not pre-maturely
optimize. That means it is ok to have unscalable algorithms when you
don't have tons of users--start by keeping your code simple. Ruby (and
the good practices that the rails community encourages) makes
refactoring pretty easy.

Cheers,

[MrLipid]

Interesting article on Twitter:

http://assetbar.wordpress.com/2008/02/08/twitter-proxy-any-interest/

[AndyV]

Agreed. This issue of scalability is much more an issue of
application design than anything else. If you make good design
choices, taking into account things like caching, background
processing, etc up front, you'll be able to scale. But those design
decisions will exist in every framework you choose.

The main advantage that you'll get in RoR iss in the code base. Well-
written Ruby code is more expressive than most other languages and
consequently tends to be easier to maintain. Similarly, the power of
the language itself tends to reduce the number of lines of code that
are required. I recently read of an application (forget the name...
maybe YellowPages.com?) that reduced its code base from 200K+ LOC to
around 120K. If you're like me, less is better... less to break, less
to remember, less to archive.... Also, the goal of Matz in designing
Ruby was to make programming enjoyable, even fun. Ask any hardcore
RoR developer and i think you'll find general agreement that he's
achieved his goal. Writing in Ruby is fun.

One other note, the latest version of Ruby (1.9.x) had performance as
its primary target. Rumors are that there has been significant gains
across the board. Part of the Rails core is working towards Ruby 1.9
compatability so we should see that fairly soon (maybe at
RailsConf?). Soon you may not be shouting back to Scotty for more
power...

[-kinetic]

Wow, great information everyone.

My app needs to scale to hundreds of thousands of users per day to a
million or so. The numbers are ideal, but as all engineers say, you
must prepare for the worst, and you must prepare for the best.

There's a huge need to develop the application with scaling in mind
every step of the way. Ruby on Rails seems to skim over that
initially, and just have you develop quickly and efficiently. Would
you recommend using RoR for something like this? The trade off might
not be worth it?

Thanks again!

[Carsten Gehling]

I could write a lot, to answer this question, but Chris @TheKeyboard
has phrased it very simple and beutiful:

http://www.littlehart.net/atthekeyboard/2007/01/02/just-build-it-damnit/

- Carsten

[Piyush Ranjan]

In my view all this talk of scalability is premature. People plan for a million users  from start and that is a premature optimization. It's much easier to do the site in Rails and when bottlenecks become a pain scale like this
1. Optimize using fragment/page cahing, server static files from apache, profile and optimize the code(check for N+1 bugs and stuff)
2. Optimize the SQLs. Use 'explain' on your mysql queries and see how much you can optimize.
3. Use more mongrels, write mongrel handlers for slow requests(or requests which depend on 3rd party servers like fetching rss feeds et al)
4. Use memcache servers to cache the queries for data that queried a lot.
5. Add more application servers
6. Write cpu intensive functions in C and add a ruby wrapper for these
7. Cluster you database servers

I run a few sites which serve upto 100-150k requests per day much of the above(1 app server, 2 mongrels, 1 db server and step 1).
Scaling upto 1 million with all the above steps should not be a big problem)

Piyush

[Rimantas Liubertas]

> My app needs to scale to hundreds of thousands of users per day to a
> million or so.

It need right now? Or you think it will need? There is a big difference here.
Some wisdom from the 37 signals:
http://gettingreal.37signals.com/ch04_Scale_Later.php

Regards,
Rimantas
--
http://rimantas.com/

[Philip Hallstrom]

> My app needs to scale to hundreds of thousands of users per day to a
> million or so. The numbers are ideal, but as all engineers say, you
> must prepare for the worst, and you must prepare for the best.

That's not really a useful question. For example... my site has 10
million users (it doesn't but let's pretend :) but they only visit once a
month. Or, perhaps I have 10,000 and they visit my site 5-6 times a
minute each.

You need to look at how many requests you are going to get over a day, or
an hour, what your peaks are, how much *traffic* you have to be able to
sustain during those times.

That all said... I agree with everyone else... just build it :)

[Phlip]

> I've read a ton of articles this afternoon, and they all say how much
> more processing power Ruby uses, and that scaling is seemingly more
> complicated.

RoR is a LAMP system. The P stands for an interPreted language, and the M
stands for MySQL.

Write developer-friendly code with the P, and put the M on another server.
If the P gets slow, put more servers under it. This system allows you to
optimize developer time, while keeping the potentially slowest part of the
system - MySQL - on a dedicated server.

--
Phlip

[Phillip Koebbe]

On Feb 10, 2008, at 5:22 PM, Phlip wrote:

>
>> I've read a ton of articles this afternoon, and they all say how much
>> more processing power Ruby uses, and that scaling is seemingly more
>> complicated.
>
> RoR is a LAMP system. The P stands for an interPreted language, and
> the M
> stands for MySQL.
>

Um, LAMP actually stands for something more along the lines of Linux,
Apache, MySQL, and Perl/PHP. RoR can run on Windows, Linux, or Mac
and can use any of quite a few databases. Some (maybe many) choose
MySQL. I don't. I use PostgreSQL.

Peace,
Phillip

[Nathan Esquenazi]

You guys interested in scaling with RoR might be interested in

http://blog.caboo.se/articles/2007/7/29/scale-rails-from-one-box-to-three-four-and-five

Basically covers how to scale and the steps to take.
--
Posted via http://www.ruby-forum.com/.

[Gregory Seidman]

That post is good advice, in general, but it starts from a premise that is
never explicitly mentioned -- that the site you are building is new.
While it's refreshing when clients ask for something brand-spanking-new,
the work is more likely to be "do exactly what my site already does, plus
this set of additional features."

In that case, the traffic is already coming to the site. If you're lucky,
the traffic is light enough that you can still ignore scaling issues while
you're developing the new Rails app to replace the existing whatever app.
On the other hand, the client may be coming to you to reimplement their
site because it is getting slow and they want you to create the new one so
you can be responsible for scaling it.

This may not be the most common Rails project, but such work is definitely
out there. It means that, yes, you need to think about scaling from the
start. You don't have the luxury of a gradual user uptake to see where the
bottlenecks are, because the moment your new site goes live you have a full
user base (see below for a way around that). You need to address problems
before they happen, because downtime and slowness are not just growing
pains but regressions that the client will be justifiably screaming about.

I'd be interested in some thoughts toward handling the situation starting
from that premise, where "just build it, damnit (sic)" doesn't apply.

As an example, one can play tricks with a frontend server to put some
segment of the user base on the new system while leaving the majority on
the old to simulate a gradual user uptake. Of course, that may not be
feasible depending on the particular site's functionality. Consider
Twitter, for example, where the users' interaction with one another is
central to the site.

> - Carsten
--Greg

[Jeremy Kemper]

On 2/10/08, Gregory Seidman <gssli...@anthropohedron.net> wrote:
> On Sun, Feb 10, 2008 at 02:06:18PM +0100, Carsten Gehling wrote:
> > I could write a lot, to answer this question, but Chris @TheKeyboard
> > has phrased it very simple and beutiful:
> >
> > http://www.littlehart.net/atthekeyboard/2007/01/02/just-build-it-damnit/
>
> That post is good advice, in general, but it starts from a premise that is
> never explicitly mentioned -- that the site you are building is new.
> While it's refreshing when clients ask for something brand-spanking-new,
> the work is more likely to be "do exactly what my site already does, plus
> this set of additional features."

No difference: just build it, damnit; scale to meet traffic demands; launch.

You're going to have more trouble with integration and compatibility, anyway.

Iteratively,
jeremy

[subimage interactive]

On Feb 10, 2008 5:37 AM, Piyush Ranjan <piyu...@gmail.com> wrote:

In my view all this talk of scalability is premature. People plan for a million users  from start and that is a premature optimization. It's much easier to do the site in Rails and when bottlenecks become a pain scale like this

Bingo. People who worry too much about scaling off the bat never get their projects launched.

Make something worthwhile first, scale it when necessary.

 

--------------------
seth at subimage interactive
-----
http://sublog.subimage.com
-----
Cashboard - Estimates, invoices, and time tracking software - for free!
http://www.getcashboard.com
-----
Substruct - Open source RoR e-commerce software.
http://code.google.com/p/substruct/

[Lionel Bouton]

Gregory Seidman wrote:
> [...]

> In that case, the traffic is already coming to the site. If you're lucky,
> the traffic is light enough that you can still ignore scaling issues while
> you're developing the new Rails app to replace the existing whatever app.
> On the other hand, the client may be coming to you to reimplement their
> site because it is getting slow and they want you to create the new one so
> you can be responsible for scaling it.
>
> This may not be the most common Rails project, but such work is definitely
> out there. It means that, yes, you need to think about scaling from the
> start. You don't have the luxury of a gradual user uptake to see where the
> bottlenecks are, because the moment your new site goes live you have a full
> user base (see below for a way around that). You need to address problems
> before they happen, because downtime and slowness are not just growing
> pains but regressions that the client will be justifiably screaming about.
>
> I'd be interested in some thoughts toward handling the situation starting
> from that premise, where "just build it, damnit (sic)" doesn't apply.
>

If it is getting slow you can check out the whole architecture to find
out where the bottlenecks are and rebuild it accordingly. As I already
said, when you have performance problems you need people knowledgeable
enough to find out why, the framework itself is only a tool that might
need some tuning (or not).

Lionel

[Marc Byrd]

Advice from smart and well-intentioned people notwithstanding, I firmly believe you should think about scale early and often.  Google thought about scale early - their very name is scale. 

My deep concern is that those who work on improving the Ruby on Rails framework may have this "worry about scale later" mentality.  I'm concerned that many of the programming paradigms in Rails lead to unscalable practices like "transparent" joins that novice programmers won't know to avoid (e.g. image.user.comments.categories - in views no less).  It would be very helpful if RoR actually solved some of the tough scale issues by, for example:

• Including support for Amazon's Simple DB service
• Intelligently unwrapping joins in AR layer, or allowing/encouraging developers to do that
• Providing a mysql proxy (better than mysql-proxy) which is scriptable in Ruby, that provides for db load-balancing, failover, multi-master, master-per-model, index-modulo multi-master, etc.  acts_as_readonlyable is a great start, but you can only scale so far without multi-master. 
  
• Support for lazy insertion, insert ignore, insert later, insert many, batch insertion, etc.  Perhaps "acts_as_batched" ? 
  
• Providing better examples and support for logging using syslog over udp (there actually may be plenty of support for this already, I just haven't had time to dig into it yet).  Educate/encourage through examples how to write certain log-type info to logs instead of writing to db. 
  

m

[Philip Hallstrom]

> - Providing a mysql proxy (better than mysql-proxy) which is

You may find this useful...

http://blog.rapleaf.com/dev/?p=5

[Marc Byrd]

I guess the main difference between mysql-proxy and things like this and acts_as_readonlyable is that mysql-proxy acts at the mysql layer.  You can connect to mysql-proxy from the command line just as if it were a mysql server.  It allows you to say things like "if it's an INSERT or UPDATE and the table is users, then use this db" - thus it hides those details from the RoR code, which may be kept clean, simple, and flexible if/when you need to add a slave, move a master, etc. 

My only real problem with mysql-proxy is lua - what's up with that?  ;^)

Just my $0.02,


m

[Lionel Bouton]

Marc Byrd wrote:
> Advice from smart and well-intentioned people notwithstanding, I
> firmly believe you should think about scale early and often. Google
> thought about scale early - their very name is scale.
>
> My deep concern is that those who work on improving the Ruby on Rails
> framework may have this "worry about scale later" mentality. I'm
> concerned that many of the programming paradigms in Rails lead to
> unscalable practices like "transparent" joins that novice programmers
> won't know to avoid (e.g. image.user.comments.categories - in views no
> less).

I'm quite sure Google architects knew these pitfalls and worked out by
themselves how to avoid them... As they simply *coded* their own
framework (and probably several times along their path).

If you really believe you can throw a "scaling" framework into the hands
of "novice programmers" and expect them to code something that scales
anything like the Google underlying architecture, just wait a minute,
I'll fetch a good seat and popcorn to watch the show. I'm sure I'll find
it either quite instructive or good laughing material :-)

Good luck,

Lionel

[Lon]

My experience taught me to think about it often, but code for it only
when needed.

Just build it sounds cool to some and stupid to others, but it does
works.

Until you have real people smashing around in your carefully crafted
application, you will never know where and more importantly when to
address scaling.

You can assume it is in location X, Y, or Z, but who knew users of the
application would be clicking on A, B, C like hamsters on a treadmill.

When I started with Rails, it was only a few weeks after David and
37Signals released it to the world. At the time my application ran
under Apache, FastCGI and MySQL. That was the entire technology chain
using 1 web server and 1 database server.

Today, the very same application runs on 5 servers, using MySQL,
Nginx, Mongrel, Amazon S3, a dedicated email platform and numerous
back processing scripts on dedicated servers.

Some of my scaling challenges:

- Concurrent users blocking others with long running actions, which
are now offloaded to the background.
- Storing large objects in the database, are instead sent to Amazon S3
or the filesystem and only store metadata in the database
- Unreliable stability of FastCGI under heavy loads, are now running
on proxy mongrels
- Slow loading of images and javascripts, are now served statically
outside application using a couple lines of Nginx configuration
- Long render times for large pages were retooled to avoid slow
methods like url_for and more intelligently paginated
- Sluggish performance when using MySQL Fulltext searches, now use an
external search tool like Sphinx

After several years of real world use by customers, I now have enough
experience to see some issues down the road.

Rails can and will scale if you think of it as only one part of the
solution.

If you think every tool needed is in the framework itself, your
application will not scale.

--
Lon