Node CPU use analysis

[Cosmere Infahm]

I analysed node-0.6.10/windows2008r2 running my clustered http server script in VTune with a view to understanding why node took 50% more CPU than nginx/apache for a given scenario. The results indicate that GetQueuedCompletionStatusEx takes 40% of CPU. Does this make sense?

I was wondering if the reported stacks are meaningful for a HTTP server script that pipes the contents of a static file as response to all requests coming over 1000's of concurrent connections. Are there really any exceptions and buffer initializations as shown?

Call Stack Time Total Module Function (Full)

Total [Unknown] 100.0%

  v8::V8::GetCompressedStartupDataAlgorithm 0s 92.2% node.exe v8::V8::GetCompressedStartupDataAlgorithm(void)

    RtlInitializeExceptionChain 0s 92.2% ntdll.dll RtlInitializeExceptionChain

      RtlInitializeExceptionChain 0s 92.2% ntdll.dll RtlInitializeExceptionChain

        BaseThreadInitThunk 0s 92.2% kernel32.dll BaseThreadInitThunk

          v8::V8::GetCompressedStartupDataAlgorithm 0s 92.2% node.exe v8::V8::GetCompressedStartupDataAlgorithm(void)

            node::Buffer::Initialize 0s 92.2% node.exe node::Buffer::Initialize(class v8::Handle<class v8::Object>)

              node::FatalException 0s 90.5% node.exe node::FatalException(class v8::TryCatch &)

                uv_unref 0.015s 41.4% node.exe uv_unref

                  GetQueuedCompletionStatusEx 0.292s 41.4% KERNELBASE.dll GetQueuedCompletionStatusEx

                    NtRemoveIoCompletionEx 180.77s 41.4% ntdll.dll NtRemoveIoCompletionEx

                    IsNLSDefinedString 0.017s 0.0% KERNELBASE.dll IsNLSDefinedString

                  uv_timer_get_repeat 0.025s 0.0% node.exe uv_timer_get_repeat

                [Unknown stack frame(s)] 0s 37.1% [Unknown] [Unknown stack frame(s)]

                  node::Buffer::Initialize 0.955s 16.3% node.exe node::Buffer::Initialize(class v8::Handle<class v8::Object>)

                    uv_write 0.181s 10.1% node.exe uv_write

                      uv_tcp_getpeername 0.332s 10.1% node.exe uv_tcp_getpeername

                        WSASend 0.179s 9.9% WS2_32.dll WSASend

                          [mswsock.dll] 0.397s 9.6% mswsock.dll [mswsock.dll]

                            NtDeviceIoControlFile 41.577s 9.5% ntdll.dll NtDeviceIoControlFile

For readability here are some stacks that correspond to the busy data:

ntdll.dll!NtRemoveIoCompletionEx - [Unknown]

KERNELBASE.dll!GetQueuedCompletionStatusEx+0x7790 - [Unknown]

node.exe!uv_unref+0x4ec35 - [Unknown]

node.exe!node::FatalException(class v8::TryCatch &)+0x1afb7 - [Unknown]

node.exe!node::Buffer::Initialize(class v8::Handle<class v8::Object>)+0x2b0ed - [Unknown]

node.exe!v8::V8::GetCompressedStartupDataAlgorithm(void)+0x298759 - [Unknown]

kernel32.dll!BaseThreadInitThunk+0x13399 - [Unknown]

ntdll.dll!RtlInitializeExceptionChain+0x39ef1 - [Unknown]

ntdll.dll!RtlInitializeExceptionChain+0x39ec4 - [Unknown]

node.exe!v8::V8::GetCompressedStartupDataAlgorithm(void)+0x2987af - [Unknown]

ntdll.dll!NtDeviceIoControlFile - [Unknown]

mswsock.dll![mswsock.dll]+0x44f3 - [Unknown]

WS2_32.dll!WSASend+0x447e - [Unknown]

node.exe!uv_tcp_getpeername+0x5d7b6 - [Unknown]

node.exe!uv_write+0x52e0f - [Unknown]

node.exe!node::Buffer::Initialize(class v8::Handle<class v8::Object>)+0x2bcdd - [Unknown]

[Unknown]![Unknown stack frame(s)]+0x0 - [Unknown]

node.exe!node::FatalException(class v8::TryCatch &)+0x1afb7 - [Unknown]

node.exe!node::Buffer::Initialize(class v8::Handle<class v8::Object>)+0x2b0ed - [Unknown]

node.exe!v8::V8::GetCompressedStartupDataAlgorithm(void)+0x298759 - [Unknown]

kernel32.dll!BaseThreadInitThunk+0x13399 - [Unknown]

ntdll.dll!RtlInitializeExceptionChain+0x39ef1 - [Unknown]

ntdll.dll!RtlInitializeExceptionChain+0x39ec4 - [Unknown]

node.exe!v8::V8::GetCompressedStartupDataAlgorithm(void)+0x2987af - [Unknown]

--

[Matt]

Why are people expecting Node to compare well with nginx or Apache for serving static files? Node is faster than PHP or Ruby, but comparing it to something specifically designed for high performance static file serving is always going to be a losing proposition.

Use Node for your dynamic content. Use nginx or apache for your static content. It's a very simple equation.

Matt.

[Cosmere Infahm]

Why are people expecting Node to compare well with nginx or Apache for serving static files? Node is faster than PHP or Ruby, but comparing it to something specifically designed for high performance static file serving is always going to be a losing proposition.

Use Node for your dynamic content. Use nginx or apache for your static content. It's a very simple equation.

Atleast on Windows, neither nginx nor Apache is a clear winner for serving static files. Nginx native Windows version just does not scale and it is limited to 1024 concurrent connections. The cygwin version works well upto ~2000 concurrent connections. The official 32-bit Apache supports ~3000 concurrent connections but it needs a thread for each connection.

In comparison Node supports 3x/4x the Apache limit (if there are enough workers). The only downside I saw was the somewhat higher CPU usage.

Looking at the Node I did not see a particular reason for the higher CPU especially since AIO/IOCP is supported. This is why I tried VTune. But looking at the VTune numbers I was unable to conclude if there was sth bad or if sth could be changed to reduce CPU.

--

[Matt]

It's written in Javascript. This isn't rocket science :)

[Tim Caswell]

It may be written in JavaScript and thus has the overhead of a
language VM. This especially hurts in terms of memory overhead per
process. But node wins over apache and nginx by having the right
event model. Good low-level primitives in libuv and the ability to
script-in assumptions. For example, in the wheat engine that powers
howtonode.org, My database is a git repo. I know that in git requests
for the same resource using the same git sha will *never* change. So
I can cache that type of request aggressively in memory. (This
includes the static resources, I also store those in git) Nginx and
Apache are general purpose static file servers. That's can't make
that kind of caching assumptions. Node serving a static file from
memory is faster than nginx serving the same file from disk. Yes
javascript is slower than C, but the smarter algorithm wins out over a
slower language.

> --
> Job Board: http://jobs.nodejs.org/
> Posting guidelines:
> https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
> You received this message because you are subscribed to the Google
> Groups "nodejs" group.
> To post to this group, send email to nod...@googlegroups.com
> To unsubscribe from this group, send email to
> nodejs+un...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/nodejs?hl=en?hl=en

[Bert Belder]

I am not sure - but you should take into consideration that
GetQueuedCompletionStatus(Ex) is the only function in libuv that
blocks. Are you sure that VTune does not take the time into account
that GetQueuedCompletionStatus blocks because there are no events to
report?

- Bert

[Matt]

On Mon, Feb 13, 2012 at 4:28 PM, Tim Caswell <t...@creationix.com> wrote:

But node wins over apache and nginx by having the right
event model.

Maybe over Apache, but you need to read more about nginx.

 

 Node serving a static file from
memory is faster than nginx serving the same file from disk.

Benchmarks please. I very much doubt that.

Matt.

[billywhizz]

from tests i have done using the low level node.js bindings and
sendfile on linux (which requires a small patch to the node.js c++
source), node.js can get pretty close to nginx (within 10-20%) for
serving static files. the larger the files, the smaller the difference
as most of the work is then being done in the kernel. i'm pretty
confident with some further work and improvements in the node.js
internals that the difference between it and nginx will be negligible,
and you will have the advantage with node.js that you can serve very
rich applications from the same process as the static files. one thing
node.js will never beat nginx on will be memory usage however...

i'll publish a benchmark up here when i get a chance - hopefully in
next few days.

[Tim Caswell]

On Mon, Feb 13, 2012 at 5:18 PM, Matt <hel...@gmail.com> wrote:
> On Mon, Feb 13, 2012 at 4:28 PM, Tim Caswell <t...@creationix.com> wrote:
>>
>> But node wins over apache and nginx by having the right
>> event model.
>
>
> Maybe over Apache, but you need to read more about nginx.

Sorry, that period was a typo. I know that nginx is event based too.
my point was that high-level algorithms matter more than the constant
overhead of the vm sometimes.

>
>>
>>  Node serving a static file from
>> memory is faster than nginx serving the same file from disk.
>
>
> Benchmarks please. I very much doubt that.

I've benchmarked it before, but it should be obvious too. Node is
doing less work than nginx in this case. (writing an already buffered
response to the socket is much easier than reading a file from disk
and then sending it to the socket. Even sendfile has to spin up the
disk even if the kernel is doing the rest)

The point is we're not doing the same thing. Nginx is really good at
what it's written for. Node is really good at what it's written for
and that can include serving static resources with business logic
smarts scripted in.

>
> Matt.

[Matt]

On Mon, Feb 13, 2012 at 8:27 PM, Tim Caswell <t...@creationix.com> wrote:

I've benchmarked it before, but it should be obvious too.  Node is
doing less work than nginx in this case.  (writing an already buffered
response to the socket is much easier than reading a file from disk
and then sending it to the socket.  Even sendfile has to spin up the
disk even if the kernel is doing the rest)

I'm sorry, but if you think any OS is spinning the disk to serve a static file you know nothing about how operating systems work. Benchmarks or GTFO in this case.

[Juraj Vitko]

I'd be really curious about that patch and the benchmarks as well, and
I'm sure it will make it to Node's core. The furthest I had time to
get so far is: https://gist.github.com/1350901
I don't have the benchmarks but it was a lot faster than serving the
file in a classic way, though less than 50% faster. But with much less
CPU usage.

[Juraj Vitko]

A meaningful thing to do might be to run Node in the v8 profiler mode,
and then analyze the log file with the v8 tick-analyzer script.

E.g.

cd /yourdir

node --prof --prof_auto yourscript.js

cd /v8-source/out/native

../../tools/<your-os>-tick-processor /yourdir/v8.log

I could not get it to work without the exact CD'ing above, so that's
why I'm spelling that out. The "v8-source" is a separately built v8,
as I wasn't able to get the tick-processor from Node's sources to run
(but I was trying an unstable/dev version of Node (0.7.3), so that
might have been the problem).

HTH
J.

[Liam]

There's an issue filed for a sendfile-based http/socket transfers,
https://github.com/joyent/node/issues/1802

It entails a fairly significant revision to the net.Socket queue
mechanism. I don't know of anyone working on that.

[billywhizz]

Liam - i asked Ben about exposing the fd on the socket handle so linux
folks could use sendfile but it seems the core team don't really want
to do that. there was a mention of providing a sendfile and/or pipe
between two handles internally in the c++ code at some stage, but my
feeling was that this is some way off. my patch has nothing to do with
big changes to the c++ - it is just a patch to expose the fd so i can
pass it to the fs.sendfile system call.

Matt - there's no need to be so rude, especially to one of the best
guys in the community. from what i understand nginx will cache small
files in memory if configured to do so and it will also use sendfile
if configured to do so. when using sendfile, there could be a blocking
operation if the file has not been cached by the kernel, but once it's
in memory then subsequent requests should be as fast as possible and
most of the work will be done in the kernel. the difference between
using sendfile from v8/node.js and directly from c/c++ is very small
from the testing i have done. getting close to nginx performance is
definitely achievable, but probably not using the standard node.js
libs (at least not right now anyway). the only way i have got good
performance is to use the c++ bindings directly along with sendfile.
i'll post up a benchmark asap which should shed some light on all
this.

On Feb 14, 10:45 am, Liam <networkimp...@gmail.com> wrote:
> There's an issue filed for a sendfile-based http/socket transfers,https://github.com/joyent/node/issues/1802

[Dean Landolt]

On Tue, Feb 14, 2012 at 8:49 AM, billywhizz <apjo...@gmail.com> wrote:

Liam - i asked Ben about exposing the fd on the socket handle so linux
folks could use sendfile but it seems the core team don't really want
to do that. there was a mention of providing a sendfile and/or pipe
between two handles internally in the c++ code at some stage, but my
feeling was that this is some way off. my patch has nothing to do with
big changes to the c++ - it is just a patch to expose the fd so i can
pass it to the fs.sendfile system call.

Matt - there's no need to be so rude, especially to one of the best
guys in the community. from what i understand nginx will cache small
files in memory if configured to do so and it will also use sendfile
if configured to do so. when using sendfile, there could be a blocking
operation if the file has not been cached by the kernel, but once it's
in memory then subsequent requests should be as fast as possible and
most of the work will be done in the kernel. the difference between
using sendfile from v8/node.js and directly from c/c++ is very small
from the testing i have done. getting close to nginx performance is
definitely achievable, but probably not using the standard node.js
libs (at least not right now anyway). the only way i have got good
performance is to use the c++ bindings directly along with sendfile.
i'll post up a benchmark asap which should shed some light on all
this.

Well put, Billy.

Also Matt: please reread Tim's original message. Does nginx know anything about a git hash? Of course not. So even if it's cached it has to at least stat the file, right?

[Matt]

On Tue, Feb 14, 2012 at 8:49 AM, billywhizz <apjo...@gmail.com> wrote:

Matt - there's no need to be so rude, especially to one of the best
guys in the community.

I didn't mean "benchmarks or GTFO" in any rude way - it's just that people need to understand that Node will never be the high performance static file server that nginx is. And trying to get there seems pretty futile to me - the web is GREAT at small pluggable components - so use it like that. Stick nginx up front, haproxy in front of that if you need it, and node in the backend doing your dynamic content. This design has HUGE advantages over just using Node for everything, and it's simple to manage and run.

 

getting close to nginx performance is
definitely achievable

Like I said, let's see some benchmarks then. I just really doubt you'll get all that close in any sane flexible manner supporting all the correct headers (eTags, If-Modified-Since, etc). There's value in having both node and nginx in your architecture.

Matt.

[Nuno Job]

I don't really understand your tone here.    Imho it was unnecessary and uncalled for. 

--

[Tim Caswell]

Matt,

I'm not offended by the tone.  I understand the intent and tone are hard to convey on the internet.  That's why I love going to tech conferences to meet people face to face.  I hope to be at nodeconf this summer, maybe we can discuss this there.

But I do disagree with your statement "it's just that people need to understand that Node will never be the high performance static file server that nginx is. And trying to get there seems pretty futile to me".

I can see that you're just trying to do your civic duty and make sure people know that serving static files isn't something node is good for.  I personally think it can be good for that.

If it helps any, my role in the node community is and has always been a mix of helping people learn the basics while at same time pushing node to do things it wasn't meant to do.  Even if node never becomes as good as nginx at what nginx is good at, it's a good experience to try and node will be better off for the attempt.  I've tried a lot of crazy things with node including desktop gui apps, desktop games, webgl on tablets (yes node, not webkit), and I serve all my static sites with node.  It's plenty fast and stable for my needs so I don't bother with throwing nginx in front of it.  I know how to configure nginx and have done some pretty advanced stuff with it, but why use an extra layer I don't need.

-Tim

--

[Matt]

On Tue, Feb 14, 2012 at 3:22 PM, Tim Caswell <t...@creationix.com> wrote:

Matt,

I'm not offended by the tone.  I understand the intent and tone are hard to convey on the internet.  That's why I love going to tech conferences to meet people face to face.  I hope to be at nodeconf this summer, maybe we can discuss this there.

Thanks - probably won't get there - new startup, etc etc. I think some Haraka guys might try and talk there though - we'll see.

Just to clarify for those who thought the tone was off - "benchmarks or GTFO" is a turn of phrase often used in the tech community to actually get some real results into a conversation rather than people talking around the subject. I'm surprised people haven't heard it said before.

 

But I do disagree with your statement "it's just that people need to understand that Node will never be the high performance static file server that nginx is. And trying to get there seems pretty futile to me".

Maybe the latter part was a bit strong (nothing is futile in coding - this shit is fun), but it's true that we'll never hit the performance of nginx, unless nginx is doing something really wrong (and let's assume I'm talking about Linux here - it may be significantly slower on Windows for all I know, but that's not a platform I will ever use).

 

I can see that you're just trying to do your civic duty and make sure people know that serving static files isn't something node is good for.  I personally think it can be good for that.

It can be mediocre for that, yes - your needs are not high traffic, I would guess. Node can probably serve files faster than anyone but a top 1000 web site needs - I mean we're talking thousands of requests/sec here - enough to serve 100 million hits a day from a single box. But when people come with benchmarks saying "Node is slower than nginx!" they need to be told: Of course it is.

I really don't mean to be a snot about this, and my tone reflected more of a bad personal time than anything to do with the list.

Matt.

[billywhizz]

Matt, there are all sorts of optimisations available. if you really
want top performance, then you could write a c++ module that does
static file serving and can be easily plugged into a node.js http
server. it would be able to spend most of it's time in c++ land
serving static files so there is no reason it could not be as fast as
nginx. also, nginx is only optimised once - at compile time. in v8,
the JIT compiler has the opportunity to optimise on the fly as the
load on the server changes. this is a big advantage over something
like nginx and it wouldn't surprise me at all to see a node.js
solution match or out perform nginx at static file serving in the near
future.

i've put a very basic benchmark up here:
https://gist.github.com/1831760

this just serves a buffer straight out of memory and doesn't do any
processing of the request headers, other than putting them into an
object instead of an array, but it's a useful baseline so we can see
how adding in the various aspects of static file serving affect the
performance.

on my server, i get 27k responses per second using ab. this pegs one
cpu and i made sure to run ab against localhost and with the server
and ab pinned to specific cpu's so they can share a cpu cache (this
makes a big difference to the results). memory usage is 9MB idle and
runs up to 16-18MB under load. nginx serving a 0k static file with
exactly the same headers can do 32k rps in the exact same environment,
so it's not a world away and there is lots of room for optimisation.

i'll start adding in the rest of the static file serving functionality
and will post results of tests to the gist.

[Mark Hahn]

> it would be able to spend most of it's time in c++ land serving static files so there is no reason it could not be as fast as nginx.  

That would only be true if you coded your c++ to be event-loop based like nginx is.

[billywhizz]

and like libuv is...

[Matt]

On Tue, Feb 14, 2012 at 7:00 PM, billywhizz <apjo...@gmail.com> wrote:

Matt, there are all sorts of optimisations available. if you really
want top performance, then you could write a c++ module that does
static file serving and can be easily plugged into a node.js http
server.

Yes, but why would you do that, when there's perfectly good open source code (nginx) to do it already?

 

it would be able to spend most of it's time in c++ land
serving static files so there is no reason it could not be as fast as
nginx. also, nginx is only optimised once - at compile time. in v8,
the JIT compiler has the opportunity to optimise on the fly as the
load on the server changes. this is a big advantage over something
like nginx and it wouldn't surprise me at all to see a node.js
solution match or out perform nginx at static file serving in the near
future.

It's *very* rare for a JIT to do better than compiled C, except on very synthetic hard looping problems - HTTP serving really doesn't fit into that.

 

i've put a very basic benchmark up here:
https://gist.github.com/1831760

Thank you. Kind of proves my point. Nginx serves more data, from the filesystem, faster, is checking for changes to the file, isn't doing sendfile(), etc. At least you turned logging off :)

But I really appreciate seeing real numbers.

Matt.

[billywhizz]

heh. proves your point. for now. i'm not dissing nginx btw - i love
nginx. i also like lighttpd and gatling a lot too, but there are many
scenarios where you may not want to serve static files from a separate
server listening on a separate port. node.js is also easily
programmable, none of the other static file servers are. anyway, i'll
keep posting up benchmarks so we have something real to argue about
and have a goal to aim at in terms of optimization.

let me know if there are any further nginx optimisations that can be
done. as far as i can see this is a minimal nginx configuration with
no extra modules but there are probably tweaks that can be made to it.
i'll download an build a later version of nginx when i get a chance.

On Feb 15, 12:42 am, Matt <hel...@gmail.com> wrote:

[Dean Landolt]

On Tue, Feb 14, 2012 at 7:42 PM, Matt <hel...@gmail.com> wrote:

On Tue, Feb 14, 2012 at 7:00 PM, billywhizz <apjo...@gmail.com> wrote:

Matt, there are all sorts of optimisations available. if you really
want top performance, then you could write a c++ module that does
static file serving and can be easily plugged into a node.js http
server.

Yes, but why would you do that, when there's perfectly good open source code (nginx) to do it already?

Really? Did you really just ask this question? Really?

 

it would be able to spend most of it's time in c++ land
serving static files so there is no reason it could not be as fast as
nginx. also, nginx is only optimised once - at compile time. in v8,
the JIT compiler has the opportunity to optimise on the fly as the
load on the server changes. this is a big advantage over something
like nginx and it wouldn't surprise me at all to see a node.js
solution match or out perform nginx at static file serving in the near
future.

It's *very* rare for a JIT to do better than compiled C, except on very synthetic hard looping problems - HTTP serving really doesn't fit into that.

 

i've put a very basic benchmark up here:
https://gist.github.com/1831760

Thank you. Kind of proves my point. Nginx serves more data, from the filesystem, faster, is checking for changes to the file, isn't doing sendfile(), etc. At least you turned logging off :)

But I really appreciate seeing real numbers.

Matt.

--

[billywhizz]

btw - with some tweaks i am able to get 47k rps from nginx on 85% of a
single core (rest is IO wait). i am serving the 0k file from /tmp
which is shared memory. have updated the nginx.conf in the gist. matt
- you are way ahead at the moment!! ;)

[Mark Hahn]

Why test a  0k file?  What could that possibly tell you?

[billywhizz]

it tells me how long it takes to process the http headers. is just a
baseline. i'll add tests for different file sizes.

[Matt]

Also remember for Node you'll have to add in an fs.watch() event on that file. Probably won't have much of an impact at all, but nginx does that to be sure it re-loads the file if it changes.

[Louis Santillan]

Wow. Someone who has heard of Gatling.

-L

> --
> Job Board: http://jobs.nodejs.org/
> Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
> You received this message because you are subscribed to the Google
> Groups "nodejs" group.
> To post to this group, send email to nod...@googlegroups.com
> To unsubscribe from this group, send email to
> nodejs+un...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/nodejs?hl=en?hl=en
>

--
Sent from Gmail Mobile

[Jann Horn]

Not just one. :) FEFE!!! :D

Am 15.02.2012 06:24 schrieb "Louis Santillan" <lpsa...@gmail.com>:

Wow. Someone who has heard of Gatling.

-L

On Tuesday, February 14, 2012, billywhizz <apjo...@gmail.com> wrote:

> heh. proves your point. f...

--
Sent from Gmail Mobile

--
Job Board: http://jobs.nodejs.org/

Posting guidelines: https://github.com/joyent/node/wiki/Mai...

[Chris Scribner]

On the topic of "why use node to service static files"...

Because if you don't need to understand, configure, and maintain an
extra piece of software in your stack, things get simpler.

If node can get 10-50% faster at serving static files, then that's X
number of more deployments that don't need to complicate their
infrastructure more than it needs to be.

Given that node has built-in gzip and SSL, it's taking strides at
being a standalone web server -- even though nginx or other front end
proxies are undoubtedly better suited for those tasks.

I fully support effort being spent at making node better at doing
things web servers commonly do. It will benefit many in the community
and is time well spent.

Chris

On Feb 14, 7:42 pm, Matt <hel...@gmail.com> wrote:

[Matt]

On Wed, Feb 15, 2012 at 12:01 PM, Chris Scribner <scr...@gmail.com> wrote:

On the topic of "why use node to service static files"...

Because if you don't need to understand, configure, and maintain an
extra piece of software in your stack, things get simpler.

On a very basic level yes, but there are more advantages to using a front-end nginx server than there are this one downside of "it's another piece of software".

 

If node can get 10-50% faster at serving static files, then that's X
number of more deployments that don't need to complicate their
infrastructure more than it needs to be.

I can almost guarantee you there are no deployments where this is a limiting factor.

I just tested the simple file server at https://gist.github.com/701407 serving up a ~7kB png file. It gets around 2000 requests per second on my ageing macbook with Core 2 Duo CPUs.

That's enough to serve 172 MILLION requests per day.

Matt.

[Matt]

On Wed, Feb 15, 2012 at 12:35 PM, Matt <hel...@gmail.com> wrote:

If node can get 10-50% faster at serving static files, then that's X
number of more deployments that don't need to complicate their
infrastructure more than it needs to be.

I can almost guarantee you there are no deployments where this is a limiting factor.

BTW I'm not trying to say people should stop trying to make this faster, because it will more than likely result in performance tweaks to libuv and so on anyway, which will be useful for people. All I'm trying to say is it's silly to benchmark it against nginx and complain that node is slower. It always will be.

Matt.

[billywhizz]

i don't hear anyone on this thread complaining that node.js is slower.
we expect it to be slower and it probably always will be, although
"always" is a dangerous word in computer science. my goal with this
research is to get node.js to a point where it's fast enough that
people have an option not to use nginx or ANOther static file server
if they would prefer not to. i think without a huge amount of work we
could get to 60-70% of nginx performance, which will be good enough
for me, for now at least! ;)

[Matt]

On Wed, Feb 15, 2012 at 1:56 PM, billywhizz <apjo...@gmail.com> wrote:

i don't hear anyone on this thread complaining that node.js is slower.

This thread started with: "I analysed node-0.6.10/windows2008r2 running my clustered http server script in VTune with a view to understanding why node took 50% more CPU than nginx/apache for a given scenario."

And it's not the first time I've seen Node serving files benchmarked against nginx.

Matt.

[Chris Scribner]

Matt,

The original post started with an observation and a request for more
details, presumably from those who know what's going on under the
hood. I didn't detect any hint of "expecting Node to compare well with
nginx or Apache."

A conversation about the underlying differences in the servers and
thoughts on how or how much node can be optimized would have been
welcome.

Instead, your response was not only unhelpful, but condescending.
Someone willing to go to the trouble of running the profiler and set
up benchmarks is already aware that they could simply use nginx.

Shouldn't we appreciate and support others' efforts - as it could lead
directly to improvements in node?

Chris





On Feb 15, 2:01 pm, Matt <hel...@gmail.com> wrote:

[Matt]

On Wed, Feb 15, 2012 at 2:23 PM, Chris Scribner <scr...@gmail.com> wrote:

Matt,

The original post started with an observation and a request for more
details, presumably from those who know what's going on under the
hood. I didn't detect any hint of "expecting Node to compare well with
nginx or Apache."

You should follow other channels too - this comes up all the time on stackoverflow and on the IRC channel.

 

A conversation about the underlying differences in the servers and
thoughts on how or how much node can be optimized would have been
welcome.

I value technical contributions, and asking for benchmarks in response to some of the comments made was an attempt to get facts rather than wishy washy comments.

 

Instead, your response was not only unhelpful, but condescending.

I'm sorry you felt that way. Emotions are hard to convey via an email list.

 

Shouldn't we appreciate and support others' efforts - as it could lead
directly to improvements in node?

I did that in several posts. One in particular pointed out the things you will have to support in order to equate to what nginx is doing under the hood. That's a technical contribution, even if it didn't provide code.

Matt.

[Dean Landolt]

On Wed, Feb 15, 2012 at 12:35 PM, Matt <hel...@gmail.com> wrote:

On Wed, Feb 15, 2012 at 12:01 PM, Chris Scribner <scr...@gmail.com> wrote:

On the topic of "why use node to service static files"...

Because if you don't need to understand, configure, and maintain an
extra piece of software in your stack, things get simpler.

On a very basic level yes, but there are more advantages to using a front-end nginx server than there are this one downside of "it's another piece of software".

 

That's a value judgement that you can't possibly make for other people. Nor are you in any position to understand what is good enough for other people's use cases.

If node can get 10-50% faster at serving static files, then that's X
number of more deployments that don't need to complicate their
infrastructure more than it needs to be.

I can almost guarantee you there are no deployments where this is a limiting factor.

 

What? Serving static files? Exactly.

 

I just tested the simple file server at https://gist.github.com/701407 serving up a ~7kB png file. It gets around 2000 requests per second on my ageing macbook with Core 2 Duo CPUs.

That's enough to serve 172 MILLION requests per day.

Matt.

--

[Matt]

On Wed, Feb 15, 2012 at 3:03 PM, Dean Landolt <de...@deanlandolt.com> wrote:

On Wed, Feb 15, 2012 at 12:35 PM, Matt <hel...@gmail.com> wrote:

On Wed, Feb 15, 2012 at 12:01 PM, Chris Scribner <scr...@gmail.com> wrote:

On the topic of "why use node to service static files"...

Because if you don't need to understand, configure, and maintain an
extra piece of software in your stack, things get simpler.

On a very basic level yes, but there are more advantages to using a front-end nginx server than there are this one downside of "it's another piece of software".

 

That's a value judgement that you can't possibly make for other people. Nor are you in any position to understand what is good enough for other people's use cases.

I'm trying to stick to facts. But sure, if you weighed the pros of running a proxy front end extremely low against the  con of having 1 more piece of software running, then you may come out with having a proxy as a negative. But I would hope people are aware of all the benefits before making that choice. Having one more piece of software in place is a con you have to deal with once. The rest of the benefits you gain occur all the time:

- Ability to proxy to multiple different backends as your site expands (e.g. maybe even mixing technologies, such as Rack/Sinatra and Node).

- A well tested, battle hardened static file server with support for all of HTTP already coded in (e.g. will your Node file server do If-Modified-Since? ETags? Gzip compress on the fly? Support sendfile()? Log correctly when scaled to multiple CPUs? Support Accept headers?)

- When you restart or take down your Node process for maintenance of some sort, do your users see a spinny beachball or a nice "Site is down for Maintenance" page which nginx can easily deliver?

There's probably a couple more I forgot, but those are the biggies.

 

If node can get 10-50% faster at serving static files, then that's X
number of more deployments that don't need to complicate their
infrastructure more than it needs to be.

I can almost guarantee you there are no deployments where this is a limiting factor.

 

What? Serving static files? Exactly.

Right. I've never once said Node is too slow at serving static files. Perhaps you misread my posts?

Matt.

[Mark Hahn]

FWIW, I was quite surprised at how easy it was to set up nginx to front-end node.  It took a couple of hours instead of the days I expected.  You can think like you do with an apache config but use a much simpler config language that just works.

[Dean Landolt]

On Wed, Feb 15, 2012 at 3:27 PM, Matt <hel...@gmail.com> wrote:

On Wed, Feb 15, 2012 at 3:03 PM, Dean Landolt <de...@deanlandolt.com> wrote:

On Wed, Feb 15, 2012 at 12:35 PM, Matt <hel...@gmail.com> wrote:

On Wed, Feb 15, 2012 at 12:01 PM, Chris Scribner <scr...@gmail.com> wrote:

On the topic of "why use node to service static files"...

Because if you don't need to understand, configure, and maintain an
extra piece of software in your stack, things get simpler.

On a very basic level yes, but there are more advantages to using a front-end nginx server than there are this one downside of "it's another piece of software".

 

That's a value judgement that you can't possibly make for other people. Nor are you in any position to understand what is good enough for other people's use cases.

I'm trying to stick to facts. But sure, if you weighed the pros of running a proxy front end extremely low against the  con of having 1 more piece of software running, then you may come out with having a proxy as a negative. But I would hope people are aware of all the benefits before making that choice. Having one more piece of software in place is a con you have to deal with once. The rest of the benefits you gain occur all the time:

- Ability to proxy to multiple different backends as your site expands (e.g. maybe even mixing technologies, such as Rack/Sinatra and Node).

Proxying and load balancing, eh? That's wonderful if you can succinctly express all your logic here in a clunky declarative config file. Yes, it's better than apache, but let's not kid ourselves -- couldn't a lot of that be made substantially more efficient with a more suitable tool for this logic? So if node makes a good static file server it makes an even better proxy. Perhaps moreso for LB.

 

- A well tested, battle hardened static file server with support for all of HTTP already coded in (e.g. will your Node file server do If-Modified-Since? ETags? Gzip compress on the fly?

It could. And I suspect there'll be a lib for that (done in c) some day. Your use of the word "battle hardened" seems to preclude the use of anything new. That's your prerogative, and perhaps sound advice, but you're wielding it like a sledgehammer. There's room for nuance here.

 

Support sendfile()?

Perhaps eventually.

 

Log correctly when scaled to multiple CPUs? Support Accept headers?)

Of course node supports accept headers. You're just thinking purely terms of static files -- there's not a lot of win in conneg there. But since you mention it yes, we really should be doing conneg. You need a more complex content processing pipeline to support this properly. Something like what node can give you.

 

- When you restart or take down your Node process for maintenance of some sort, do your users see a spinny beachball or a nice "Site is down for Maintenance" page which nginx can easily deliver?

 

And you need nginx for this?

 

There's probably a couple more I forgot, but those are the biggies.

And each one could be accomplished with a module or a small amount of javascript.

 

If node can get 10-50% faster at serving static files, then that's X
number of more deployments that don't need to complicate their
infrastructure more than it needs to be.

I can almost guarantee you there are no deployments where this is a limiting factor.

 

What? Serving static files? Exactly.

Right. I've never once said Node is too slow at serving static files. Perhaps you misread my posts?

No, you've consistently said that Node will never be as fast as nginx and implied that because of that you should of course have nginx in front of node. It's this implication I take issue with.

I'm not saying we should all use node for this. What I'm replying to is your consistent, belligerent insistence that node will never be good enough for these use cases (in spite of the evidence, including your admission that static files aren't the bottleneck). 

That's ridiculous.

[Matt]

On Wed, Feb 15, 2012 at 4:48 PM, Dean Landolt <de...@deanlandolt.com> wrote:

On Wed, Feb 15, 2012 at 3:27 PM, Matt <hel...@gmail.com> wrote:

On Wed, Feb 15, 2012 at 3:03 PM, Dean Landolt <de...@deanlandolt.com> wrote:

On Wed, Feb 15, 2012 at 12:35 PM, Matt <hel...@gmail.com> wrote:

On Wed, Feb 15, 2012 at 12:01 PM, Chris Scribner <scr...@gmail.com> wrote:

On the topic of "why use node to service static files"...

Because if you don't need to understand, configure, and maintain an
extra piece of software in your stack, things get simpler.

On a very basic level yes, but there are more advantages to using a front-end nginx server than there are this one downside of "it's another piece of software".

 

That's a value judgement that you can't possibly make for other people. Nor are you in any position to understand what is good enough for other people's use cases.

I'm trying to stick to facts. But sure, if you weighed the pros of running a proxy front end extremely low against the  con of having 1 more piece of software running, then you may come out with having a proxy as a negative. But I would hope people are aware of all the benefits before making that choice. Having one more piece of software in place is a con you have to deal with once. The rest of the benefits you gain occur all the time:

- Ability to proxy to multiple different backends as your site expands (e.g. maybe even mixing technologies, such as Rack/Sinatra and Node).

Proxying and load balancing, eh? That's wonderful if you can succinctly express all your logic here in a clunky declarative config file. Yes, it's better than apache, but let's not kid ourselves -- couldn't a lot of that be made substantially more efficient with a more suitable tool for this logic? So if node makes a good static file server it makes an even better proxy. Perhaps moreso for LB.

Yes, if you want to write it all yourself, you can do this today in Node. There's some options on npm to do some of it that may also suit your needs.

 

 - A well tested, battle hardened static file server with support for all of HTTP already coded in (e.g. will your Node file server do If-Modified-Since? ETags? Gzip compress on the fly?

It could. And I suspect there'll be a lib for that (done in c) some day. Your use of the word "battle hardened" seems to preclude the use of anything new. That's your prerogative, and perhaps sound advice, but you're wielding it like a sledgehammer. There's room for nuance here.

No I'm not, I'm stating the advantages of putting nginx in front of a node backend today as your deployment method, so that people here can make professional choices based on sound advice.

 

- When you restart or take down your Node process for maintenance of some sort, do your users see a spinny beachball or a nice "Site is down for Maintenance" page which nginx can easily deliver?

 

And you need nginx for this?

No but you need a front end proxy. How are you going to deliver a page if node is doing all the work and you stop the process. I'm just saying nginx is good for that.

 

There's probably a couple more I forgot, but those are the biggies.

And each one could be accomplished with a module or a small amount of javascript.

Of course - I absolutely don't deny that. Is that the best option for you? If so, go ahead.

 

 

If node can get 10-50% faster at serving static files, then that's X
number of more deployments that don't need to complicate their
infrastructure more than it needs to be.

I can almost guarantee you there are no deployments where this is a limiting factor.

 

What? Serving static files? Exactly.

Right. I've never once said Node is too slow at serving static files. Perhaps you misread my posts?

No, you've consistently said that Node will never be as fast as nginx

"at serving static files". And it won't be. And we should be OK with that.

 

and implied that because of that you should of course have nginx in front of node. It's this implication I take issue with.

Well I think this is partly why people are so mad at me here, because what I'm trying to say is nuanced and hard to express, so let me try again.

1) please don't benchmark Node against Nginx for static file serving unless you're actively working to make it faster - that is a fools game as Nginx will always be faster.

2) having a front end proxy has many advantages, beyond just being "faster at serving static files", as I expressed above. A FEP should be used in all serious professional installations, in my (strongly stated) opinion because they provide a better UX, and a better operations environment.

3) nginx makes a great front end proxy because of features like try_file, among others.

 

I'm not saying we should all use node for this. What I'm replying to is your consistent, belligerent insistence that node will never be good enough for these use cases (in spite of the evidence, including your admission that static files aren't the bottleneck). 

That's ridiculous.

Because I've been misinterpreted, but that's probably my fault. If someone wants to write an awesome front end proxy in Node that does all those great features, and people want to use it over Nginx (despite it being slower, have fewer real world users, etc) that's absolutely their prerogative, but if you use that, see point #1 above.

Matt.

[Matt]

On Wed, Feb 15, 2012 at 5:12 PM, Matt <hel...@gmail.com> wrote:

1) please don't benchmark Node against Nginx for static file serving unless you're actively working to make it faster - that is a fools game as Nginx will always be faster.

I should have added in here, so more people don't get mad at me (sheesh, I'm taking a beating today!), that I don't mean making Node faster is a fools game. That's a wonderful thing to do and may you be rewarded with many beers (or your choice of non-alcoholic beverage).

[Tim Caswell]

On Wed, Feb 15, 2012 at 4:14 PM, Matt <hel...@gmail.com> wrote:

On Wed, Feb 15, 2012 at 5:12 PM, Matt <hel...@gmail.com> wrote:

(or your choice of non-alcoholic beverage).

Sweet! ;) 

[Chris Scribner]

Maybe I'm dumb, but I ran into a few issues setting up nginx the first
time...

It rejected file uploads greater than 1 meg
It timed out requests after some short period of time
It rejects multi-part form uploads that don't have a proper content
length, and doesn't support chunked encoding. We had to bypass it for
these uploads

Maybe we'll run into other issues, maybe not. The gzip has worked out
really well anyway.

Chris

[billywhizz]

Matt - just wanted to address a couple of points you made:

1. With node.js clustering you get what is effectively a layer 4 load
balancer across the available cpu's on a single box. this is all being
handled by the OS and is insanely fast
2. Restarting a server gracefully is pretty easy with a little bit of
work
3. Recommending nginx as a reverse proxy in front on node.js for all
scenarios seems a bit OTT to me. reverse proxies add quite a lot of
latency to each request and introduce another point of failure into
the pipeline. if you want to use nginx for serving static files, then
you'd be better to serve your static from a separate host or port and
go directly to node.js for anything non-static. imho, a reverse proxy
should always be a last resort if no other solution is possible, and
this is partly why i'd like to node.js performance improve, so we can
put together simpler solutions that don't have multiple points of
failure
4. you mentioned issues with logging when clustering over multiple
cpu's? what exactly is the issue here and what does nginx do to solve
it?

here's a short list of what i'd like to see node.js become really good
at:

1) simple and fast static file serving out of the box
2) reverse proxying for TCP and HTTP
3) fastcgi for talking to PHP or AN/Other FastCGI backend (see:
https://github.com/billywhizz/node-fastcgi-parser)

anyway - i'll post some more benchmarks up here when i get the chance
and hopefully we can all make node.js better together.

[Matt]

On Thu, Feb 16, 2012 at 4:25 AM, billywhizz <apjo...@gmail.com> wrote:

Matt - just wanted to address a couple of points you made:

1. With node.js clustering you get what is effectively a layer 4 load
balancer across the available cpu's on a single box. this is all being
handled by the OS and is insanely fast

Calling require('cluster') a load balancer seems a really skewed way of looking at things to me. I think you'd be hard pressed convincing anyone looking for a load balancer that this was an alternative. But anyway not one of the posts I wrote mentioned load balancing, so I don't see the point.

 

2. Restarting a server gracefully is pretty easy with a little bit of
work

But taking it down because you've got something seriously broken in your application that is breaking the database (happens ALL the time in software companies) and can't leave the server up for another second is a scenario this doesn't deal with.

It's a shame this "little bit of work" isn't in core yet - hopefully soon.

 

3. Recommending nginx as a reverse proxy in front on node.js for all
scenarios seems a bit OTT to me. reverse proxies add quite a lot of
latency to each request

Benchmarks please. I bet the overhead is nowhere near enough to make you worry.

 

and introduce another point of failure into
the pipeline.

They've been used for over a decade at some of the largest sites you've heard of.

 

if you want to use nginx for serving static files, then
you'd be better to serve your static from a separate host or port and
go directly to node.js for anything non-static.

There are issues with that for certain types of sites, but yes this is certainly another option for static file serving - but didn't address the other issues I raised.

 

imho, a reverse proxy
should always be a last resort if no other solution is possible, and
this is partly why i'd like to node.js performance improve, so we can
put together simpler solutions that don't have multiple points of
failure

I bet the performance is probably just fine for you.

 

4. you mentioned issues with logging when clustering over multiple
cpu's? what exactly is the issue here and what does nginx do to solve
it?

It very much depends on how you are logging. It's possible for multiple processes writing to the same file (or stdout) to end up with overlapping lines. Obviously if you're using syslog this isn't an issue, but I know how much you hate additional pieces of software.

Matt.