[nodejs] [Newbie] Multicore leverage?

[Daniel Ingalls]

I think Node.JS is very cool. But in crowing about it at work, a
colleague said...
-----
I had done informal performance tests with swiftiply, which uses
EventMachine, against Yaws. Swiftiply did amazingly well for a non-
concurrent test load but the single threaded model of the Reactor
pattern only maxed out to near 100% of 800% total on an 8-core
machine ;-) With a concurrent test load the performance degraded
quickly.

Yaws – the Erlang http server - scaled to over 30,000 req/s with a
highly concurrent load – consuming well over 600% CPU of the SMP
machine ;-)
-----
I think Node.JS is an important pivot in moving the world into the web
and into the cloud, but if I'm going to be serious about it I have to
have some sense that it will use what's there on the multicores.

Was this an old version? Will Node now come close to, eg, 800% on an
octopus? I know Erlang is made for this, but I would think a V8 Node
ought to come close.

Thanks in advance for any explanation or pointers to relevant
benchmarking.

- Dan

--
You received this message because you are subscribed to the Google Groups "nodejs" group.
To post to this group, send email to nod...@googlegroups.com.
To unsubscribe from this group, send email to nodejs+un...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/nodejs?hl=en.

[Neville Burnell]

I was thinking about this very point this morning.

The "plan" is to support the W3C Webworker API for performing discrete
"jobs" [as I understand it] kicked off by the main event loop.

The "workaround" is to start many nodejs instances on a server, and to
use a load balancer to dispatch requests to each instance.

I'd be keen to see nodejs support the "workaround" natively, ie, a
simple round robin load balancer passing requests to N independent
event loops [webworkers?].

This might give something of the Erlang level of concurrency you
referred to "out of the box", ie, without an external dispatcher, I
think.

Thoughts?

[inimino]

Hi Daniel,

On 2010-04-22 23:53, Daniel Ingalls wrote:
> I think Node.JS is an important pivot in moving the world into the web
> and into the cloud, but if I'm going to be serious about it I have to
> have some sense that it will use what's there on the multicores.

An HTTP server is more likely bound by network and I/O bandwidth than
by CPU.

If you are hitting actual CPU limits on real-world HTTP traffic, you
could use one node process per CPU and a separate load-balancer.

Just make sure you're not solving a problem you don't have because
of an artificial benchmark...

--
http://inimino.org/~inimino/blog/

[Neville Burnell]

On Apr 23, 4:36 pm, inimino <inim...@inimino.org> wrote:
>
> An HTTP server is more likely bound by network and I/O bandwidth than
> by CPU.
>

Is that really true with modern servers?

My Dell R710 is fairly vanilla with an 8 disk raid which sustains 1GB/
s read/write and has 4xGb Nics and 2xQuad Core 2ghz CPUs

Would 1 nodejs instance find the bottlenecks in that I wonder?

[billywhizz]

On Apr 23, 7:52 am, Neville Burnell <neville.burn...@gmail.com> wrote:
> On Apr 23, 4:36 pm, inimino <inim...@inimino.org> wrote:
>
>
>
> > An HTTP server is more likely bound by network and I/O bandwidth than
> > by CPU.
>
> Is that really true with modern servers?
>

I think it's a bit of a catchall statement. it all depends on what the
web server is doing. if it's just serving static pages from disk/
cache, then this holds true, but if it's processing a lot of business
logic on each request then the CPU could well become the bottleneck.

> My Dell R710 is fairly vanilla with an 8 disk raid which sustains 1GB/
> s read/write and has 4xGb Nics and 2xQuad Core 2ghz CPUs
>
> Would 1 nodejs instance find the bottlenecks in that I wonder?
>
> --
> You received this message because you are subscribed to the Google Groups "nodejs" group.
> To post to this group, send email to nod...@googlegroups.com.
> To unsubscribe from this group, send email to nodejs+un...@googlegroups.com.
> For more options, visit this group athttp://groups.google.com/group/nodejs?hl=en.

erlang/yaws has a very different architecture to node.js. as far as i
know erlang uses "lightweight processes" which are basically very
lightweight threads which run within a virtual machine, so a single
erlang "process" should utilise all available cpu's on the system out
of the box. node.js only has one thread of execution, so you will have
to spawn multiple processes and somehow distribute the workload across
them. it's the "distributing the workload" part that isn't there in
node.js yet, but it is still possible using a load balancer like nginx
or a custom solution using child processes/unix domain sockets/tcp....

[r...@tinyclouds.org]

On Thu, Apr 22, 2010 at 11:52 PM, Neville Burnell
<neville...@gmail.com> wrote:
> On Apr 23, 4:36 pm, inimino <inim...@inimino.org> wrote:
>>
>> An HTTP server is more likely bound by network and I/O bandwidth than
>> by CPU.
>>
>
> Is that really true with modern servers?
>
> My Dell R710 is fairly vanilla with an 8 disk raid which sustains 1GB/
> s read/write and has 4xGb Nics and 2xQuad Core 2ghz CPUs
>
> Would 1 nodejs instance find the bottlenecks in that I wonder?

It's rather your network connection that will be saturated with the
HTML it serves.

[Matthew Ranney]

On Fri, Apr 23, 2010 at 10:56 AM, <r...@tinyclouds.org> wrote:

My Dell R710 is fairly vanilla with an 8 disk raid which sustains 1GB/
> s read/write and has 4xGb Nics and 2xQuad Core 2ghz CPUs
>
> Would 1 nodejs instance find the bottlenecks in that I wonder?

It's rather your network connection that will be saturated with the
HTML it serves.

In other words, do you really have 4Gbits/sec of connectivity to your users?  

[Isaac Schlueter]

While I'm all for not solving problems that don't really exist, I
don't know that it's so easy to make these kinds of assertions in a
general way.

Imagine a server like this:

http.createServer(function (_, res) {
doSomethingExpensive()
res.writeHead(200)
res.end("ok")
})

If "doSomethingExpensive" is expensive enough, it's quite possible to
max out the CPU long before I'm blocked by the amount of network data.

Even if it's asynchronous:

http.createServer(function (_, res) {
doSomethingExpensive(function () {
res.writeHead(200)
res.end("ok")
})
})

if it takes the entire CPU for even a short while to do this, then it
doesn't take very many requests until you're unable to respond before
the client times out. All the while, 3 CPUs are sitting idle, which
is not ideal.

The easy answer is: start as many child procs as you have CPUs, and
offload the work to them in some intelligent fashion. Maybe it's
enough of an edge case that this isn't really worth making any more
user-friendly than that. Certainly, most "normal" web servers don't
have to worry about this kind of thing, as the network will generally
always be the bottleneck.

--i

[r...@tinyclouds.org]

Hi Dan,

Node is inherently single threaded and will not scale over multiple
cores. However multiple processes will be scheduled across different
cores. The solution, one as old as time, is to use multiple processes.
I'm in the processes of adding facilities to make starting and sharing
between node 'workers' more easy. In the future starting an 8 process
web server will probably be something like:

s = http.createServer(handler);
for (i = 0; i < 8; i++) {
var w = createWorker(script);
w.sendFD(s.fd);

[billywhizz]

in this case you can use a worker process to do the heavy processing
on another core and continue serving requests in your server or you
can break up the heavy processing into chunks (like map/reduce) and
process it at regular intervals so you can continue to handle requests
in between processing chunks.

utilising multiple cores is not an issue. *nix has been able to do
this for a long time using an essentially process based architecture
rather than a threaded one... there are trade offs and compromises
involved in both ways of doing things. being able to do it both ways
means you can make better decisions about those trade-offs and
compromises...

getting 800% out of your octopus shouldn't be a problem with
node.js... ;)

i think the plan is to implement web workers using unix domain sockets
and with a message passing interface, similar to how erlang handles
concurrency... am sure ry will correct me if i am wrong on this...

happy noding!

> For more options, visit this group athttp://groups.google.com/group/nodejs?hl=en.

[Isaac Schlueter]

On Fri, Apr 23, 2010 at 12:02, <r...@tinyclouds.org> wrote:
> I'm in the processes of ...

_ry is multi-core ;P

--i

[Dan Ingalls]

Thanks to all who responded (and feel free to add more if you wish).

Fortunately my first application has no need for an ink-spouting
octopus, but I wanted at least to confirm what appear to be the
limitations and ways around them.

My first experiment will be to hook Node up to the Lively Kernel.
The immediate goal will be a really simple brazil-style server on a
database whose sole purpose is to synchronize events from multiple
collaborating LK users.

Best to all, and thanks for doing what I thought I might have to

- Dan

[Edwin Khodabakchian]

For m.feedly.com, we have a similar use case in that some of our
javascript ends up processing JSON scripts and massaging content and
compressing things, operations which are CPU intensive. In order to
make that we take advantage of the full power of our servers, we have
created the following thing (which we will be open sourcing soon).

We have an "admin" node process which is started with as input the
number of "feedly" node processes it should launch and monitor. When
the admin starts, it spawns multiple "feedly" node processes, padding
as input the port the feedly node should listen to 9701 for the first
one,...9710 for the 10th one.

On the server we have a varnish server which load balances the traffic
from m.feedly.com to 127.0.0.1:9701 --- 127.0.0.1:9710.

Each "feedly" node process has a 127.0.0.1:970x/heath endpoint where
it reports stats about its execution. The admin node monitor each
feedly node every 2 minutes and makes sure that the feedly node is
healthy and responding. If not, it kills the child and restart a new
instance of the feedly node listening to the same port.

All feedly nodes point to a redis instance for shared memory/session
management. This allows incoming HTTP requests to be load balances
across any feedly nodes transparently.

In dev/staging, we have varnish and all the node processes running on
the same box.

In production, we can scale out by having varnish on one server and
multiple node cell (where a cell is an admin+10 feedly). This allows
us to do rolling upgrades, etc. without any interruption to the
service.

Overall, we are still learning but we are huge fans of the work the
nodejs/ryan is doing.

We will try to document/open source this blueprint as soon as we have
some time.

Edwin

[billywhizz]

feedly.com is a superb little piece of engineering by the looks of it.
is it yours edwin? brilliant work if so... has made my google-reading
life a whole lot easier at a stroke!

On Apr 23, 9:46 pm, Edwin Khodabakchian

[Stephen Belanger]

I agree. I just discovered feedly, and it's pretty slick. I've been using Google Reader until now, but I think I may switch to feedly because it's just so clean. The only thing I'd change is moving the little control buttons from being at the end of article title to be floating at the side or something, so they are always in the same spot. I don't like having to repeatedly move the mouse all over the screen to do the same action. (mark as read) >.>

[Neville Burnell]

> Node is inherently single threaded and will not scale over multiple
> cores. However multiple processes will be scheduled across different
> cores. The solution, one as old as time, is to use multiple processes.
> I'm in the processes of adding facilities to make starting and sharing
> between node 'workers' more easy. In the future starting an 8 process
> web server will probably be something like:
>
>   s = http.createServer(handler);
>   for (i = 0; i < 8; i++) {
>     var w = createWorker(script);
>     w.sendFD(s.fd);
>   }
>

Hi Ryan,

Any plans to enable creating a new EventLoop worker?

This would facilitate my desire to create a native node load balancer
dispatching requests to multiple event loop processes.

Kind regards

[Edwin Khodabakchian]

Thanks for the kind words. Yes. I am one of the contributor to the
feedly project. @Stephen: yes. A lot of people have asked to keep all
the actions aligned. We are looking into that. -Edwin

[billywhizz]

great. i really like feedly. really shows off the browser as a
platform. with the power of v8 in the browser combined with HTML5
local storage, there's no real need to host the UI part of the
application on the server any more. the server can just be a big API/
cloud that has a well defined protocol for interacting with it and all
the rendering and user interaction takes place in the browser. the fat
client is well and truly back! ;)

would be nice to add some social interactivity to feedly so i could
keep track of what my network of friends are reading or share stuff
with them directly rather than going through facebook/Google/AN Other
social platform. wouldn't it be possible to use twitter for something
like this? (i.e. as a notification service for my social network)

On Apr 24, 5:24 am, Edwin Khodabakchian