Developing a WebFarm

[Satish Kota]

Hi All,

 

I have a requirement to develop a web farm for my client. Currently we have just one website www.wallhogs.com which has an customer focus part, an upload image part and an administration part.

Now we are supposed to strip and make them as 3 different websites but still connect to same website.

 

Also to add complexity to the problem our next phase is to build additional 3 more websites similar in nature but differnt business strategies.

 

Now all the 4 customer focus websites will have to be controlled by same admin website and all use the upload section. Finally all data are stored in a single database.

We expect large traffic based on our experience last christmas. So just running on few mongrels do not do the work.

 

Any ideas on how we should approach this problem? May be a webfarm with any load balancing options might do good.

 

All help is well regarded with cash and kindness.

Thanks and Regards

Satish N Kota

[Abhishek Parolkar]

On Wed, May 21, 2008 at 6:50 PM, Satish Kota <satis...@googlemail.com> wrote:

Now we are supposed to strip and make them as 3 different websites but still connect to same website.

Do you mean connect to same data base?, consider mysql clustering

We expect large traffic based on our experience last christmas. So just running on few mongrels do not do the work.

 

Any ideas on how we should approach this problem? May be a webfarm with any load balancing options might do good.

 

All help is well regarded with cash and kindness.

1.) Move all static content to server where you could do mod_mem_cache and mod_deflate for apache (I am not talking Memcached here.) Let upload also happen on this server.
2.) Use apache with MPM-Worker and increase max-allowed connections to your taste.
3.) Use Memcached with application logic to store session and app data which seems heavy.
4.)  Find out what exactly are the bottle-necks (ab , sar , dtrace)
5.)  Tune your existing  infrastructure  and bench mark the  performance , then if you have more money look at hardware load balancer and  HA clusters
 

[Subbu]

You should consider developing all these as independent applications
and connect to each other using ActiveResource. For the admin part,
you can authenticate the incoming requests using HTTP Basic
Authentication. Check out more about it at http://railscasts.com/episodes/82.

I don't have practical experience with this kind of a solution. But
you can give it a try.

On May 21, 6:20 am, "Satish Kota" <satishnk...@googlemail.com> wrote:
> Hi All,
>
> I have a requirement to develop a web farm for my client. Currently we have

> just one websitewww.wallhogs.comwhich has an customer focus part, an