Clustering linux / load balancing
Daniel gibbs
Hi, I was wondering if it was possible (I assume it is) to cluster
sever linux boxes to provide load balancing?
If so what distribution is recommended?
Are there any how-to's about as I've looked on linux.org and the
nearest I can find are beowulf systems?
Cheers
dan
David Coulson
> Hi, I was wondering if it was possible (I assume it is) to cluster
> sever linux boxes to provide load balancing?
Depends what you're load balancing.
> If so what distribution is recommended?
Doesn't matter - Unless you've got a ready made cluster from someone,
you'll have to do the hard work yourself.
> Are there any how-to's about as I've looked on linux.org and the
> nearest I can find are beowulf systems?
Depends what you're doing. www.linux-ha.org is handy for network services,
and mosix.org for process migration.
--
David Coulson http://davidcoulson.net/
d...@vidcoulson.com http://journal.davidcoulson.net/
Jonathan McDowell
http://www.linuxvirtualserver.org/
is probably worth a look.
J.
--
101 things you can't have too much of : 29 - T-shirts.
Ask me about server co-location - in...@blackcatnetworks.co.uk
Chris Newport
What are you trying to achieve ?.
For simple load balancing you just set up both IP addresses to the
same hostname in DNS.
To get fancy you probably need to spend some money at
http://www.alteonwebsystems.com/
I have not seen a high reliability cluster with proper failover for
Linux, it should not be to difficult but I am a bit short of round
tuits at the moment.
It's on my 2do list if nobody else has done it.
Tim Haynes
[snip]
> > If so what distribution is recommended? Are there any how-to's about as
> > I've looked on linux.org and the nearest I can find are beowulf
> > systems?
>
> What are you trying to achieve ?. For simple load balancing you just set
> up both IP addresses to the same hostname in DNS.
That's an option for distributing the load, but gives no resillience
against one member dropping off the planet.
Another option along the same lines is iptables/netfilter with DNAT: you
can specify multiple different destinations and it'll go round them in turn
as well, if you want.
> I have not seen a high reliability cluster with proper failover for
> Linux, it should not be to difficult but I am a bit short of round tuits
> at the moment.
Compared to this time last year, Mosix has come on leaps & bounds; it does
now lose & re-gain cluster members cleanly at the network level which it
didn't a while ago. (I've not checked what happens to processes that were
running when a node goes down, though.)
> It's on my 2do list if nobody else has done it.
The Failsafe Linux project at <http://oss.sgi.com/> looks very promising.
It's got support for both failover and load-balancing, and talks about
networks services / objects that you want to supply (e.g. an IP# that must
be up, an NFS fileserver that must be up, a web server, a database) with a
view to saying "that thing on yonder box is dead, quick, ifconfig the alias
up over there instead". It's just that the dependencies nearly killed me,
so I didn't have enough time to do it before the Expo.
Another idea: you can go great guns with PVM if your application/s
support/s it. Repeat: your application has to be PVM-aware to make use of
it.
As for what distro... Whatever makes you happiest. I dispute the claim that
you need to be a rocket scientist to get a cluster up & running, but it
sure helps if you have either identical hardware on all the nodes, or a
means to clone the same configs for everything apart from hostname across
as many boxes as you have.
Just one little example amongst many: for the expo thing, I wanted a 4-node
cluster of some sort, drawing pretty piccies. Started by installing Stormix
on one node, and I dist-upgraded my way up to debian/testing. Built a
kernel with Mosix support, on the grounds that I wanted that. Did a few
apt-get installs to make sure some decent packages were available - sudo,
zsh, pvm, some compilers, that kind of thing. Built a version of povray
that used PVM. `apt-get install mosix' and that's the userspace bits sorted
out. Burnt a CD with a complete image of the HD on it. Installed identical
copies on the other nodes, and edited /etc/hostname so they'd come up
differently. A little bit of configuration later, and I've got PVM-PoV-ray
running on a 4-node cluster getting a 3.5x speedup factor over the same
image on just one of the nodes, and using mosix' _mon_ tool to watch the
relative loads on all 4 boxes. Cluster, party! ;)
But then, some people think I'm a Debian-head. RH has its kickstart ability
(including, as of v7.1, the ability to write this custom bootdisk as part
of the install), Mandrake has had the same "now write a kickstart disk"
facility for yonks, Debian has the FAI project/package if you're
interested. Or, if you've got a few weeks to spare, you could download
Rock, rebuild it all from source and add a few useful packages & kernel
mods, then re-ship it as a cluster distro using the supplied scripts to
build a bootable CD. In short, there is no `what is the best distro?'
question, there is only `what makes you happiest?'.
~Tim
--
The sun is melting over the hills, |pig...@stirfried.vegetable.org.uk
All our roads are waiting / To be revealed |http://spodzone.org.uk/
Daniel Gibbs
Daniel gibbs wrote:
{SNiP}
> Basicly were looking into clustering webservers (aka web-farms).
> Because were looking at about 50 million hits upwards per month we
> will need some form of clustering more than just DNS Round-Robin.
{SNiP}
my bad just relised its morelike 100 mil not 50
Jax
check out Dr Dobbs technet cast, theres an mp3 to download that tells you
about it
JAx
"Tim Haynes" <use...@stirfried.vegetable.org.uk> wrote in message
news:86u20ab...@potato.vegetable.org.uk...
Daniel Gibbs
Jax wrote:
>
> Google uses like 10,000 linux PC's with some sort of load balancing -
> check out Dr Dobbs technet cast, theres an mp3 to download that tells you
> about it
Hi, have you got a url as I cant find it.
cheers
dan
Jax
http://www.technetcast.com/tnc_catalog.html?item_id=70
Although i can't find the google one, im sure it there - keep looking
ok i found it - look for #18
http://www.technetcast.com/tnc_rankings.html
Let me know what u think
Jax
"Daniel Gibbs" <na...@dread.eu.org> wrote in message
news:3B56092D...@dread.eu.org...
MJ Ray
> Hi, I was wondering if it was possible (I assume it is) to cluster
> sever linux boxes to provide load balancing?
If you're doing computation, GNU Queue might be worth a look.
--
MJR
Member of the Anglian Linux User Group, UK
Meets every month or so and OPN #alug on Mondays at 2000 UK
http://www.anglian.lug.org.uk/
Pierre Vaillancourt
We also are looking at this type of cluster. We are network integrators and
believe we have a great solution.
We are proposing a ZUMA Networks switch for clustered computing to a
government client next month.
The Zuma Z16 switch has PC processors built into the network fabric with
Linux running on it. (up to 32 CPUs on 256g backplane)
Check out the site www.zumanetworks.com or ccontact me for information.
Pierre Vaillancourt
Director of sales
ACNI Technologies
sa...@acnitech.com
tel: 450-681-3131 ext 103
free: 877-660-ACNI
"Daniel gibbs" <spa...@plus.net> wrote in message
news:3b55bc3f...@News.CIS.DFN.DE...