Clustering in node.js using mesos
Farshad P
I'm
working on a project with Node.js that involves a server. Now due to
large number of jobs, I need to perform clustering to divide the jobs
between different servers (different physical machines). Note that my
jobs has nothing to do do with internet, so I cannot use stateless
connection (or redis to keep the states) and a load balancer in front of
the servers to distribute the incoming connection...
I already read about the "cluster" module, but, from what i understood,
it seems to scale only on multiprocessors on the same machine.
My question: is there any suitable distributed module available in
Node.js for my work? I have heard that mesos can abstract multiple
physical machines into a single server? is it correct? If yes, it is
possible to use the node.js cluster module on top of the mesos, since
now we have only one virtual server?
Farshad P
Nathan
Christian Taltas
--
Job board: http://jobs.nodejs.org/
New group rules: https://gist.github.com/othiym23/9886289#file-moderation-policy-md
Old group rules: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
---
You received this message because you are subscribed to the Google Groups "nodejs" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nodejs+un...@googlegroups.com.
To post to this group, send email to nod...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/nodejs/27f7a8a5-29d8-48a4-b2f6-fa49f2ca677d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
KPMG Technologies Services - Directeur Général
Ryan Schmidt
If you need to start processes on more than one physical server, then you need something else. nginx is a popular web server that can be used in front of a bunch of node processes, which can be on any number of physical servers.
If your processes are not serving a web site, or are not serving any kind of network socket at all, then nginx and cluster are not what you need.
I am not familiar with mesos. The last time I was looking into ways to distribute work among multiple processes on multiple computers (which was a few years ago), I used Kue:
http://learnboost.github.io/kue/
The idea was that a master process would create units of work, which would get written into a redis database, and then a pool of workers, which could be on as many different physical machines as you like, would each pick off units of work and do them, and return their results to the database, which would give them to the master, which could then do what it liked with the results as they came in.
