Why NodeJS' performance is much lower than PHP5-raw's and even behind PHP-raw in the chart?

[Anders Ohlson]

Hello,

I look at the chart (https://www.techempower.com/benchmarks/) and can't seem to understand why NodeJS is a loser to PHP (a scripting framework oppresses and discourages OOP practices, not many modules around to support development as node_modules, no real console/terminal like the npm) ?

why NodeJS is a loser to PHP?

Thanks,

@Zetek

[Dmitry]

Yeah, it would be nice to know how many instances did they run and what load balancer did they use.

[Adam Chlipala]

In case it wasn't clear: all the benchmarking is run from the source code freely available here:
    https://github.com/TechEmpower/FrameworkBenchmarks
Every configuration detail is there.

[Dmitry]

From the github:

The tests were run with:

• Node.js v0.12.2

• Node MySQL 2.7.0

• Sequelize 3.1.1
• Node MongoDB Driver 2.0.33
• Mongoose 4.0.4
• Node Redis 0.12.1
• Hiredis 0.4.0 (C lib for Redis)

There is no load balancer mentioned here. Node.js is single threaded. You wrote you used a dual Xeon E5 v2. Worst case scenario it's 2 * 4 cores * 2 threads CPUs. Which means that Node used 1/16 of the power of that server. Well not exactly 1/16 because if only 1 core is used it will work in turbo mode (in higher frequency), and SQL runs on the same machine too.

https://www.nginx.com/blog/5-performance-tips-for-node-js-applications/#load-balancer

Here is a blog post about running Node.js behind nginx proxy for load balancing.

I don't see how your current set up is representative of the actual performance. You need to run at least like 8-12 instances of Node.js to utilize that server properly.

[Adam Chlipala]

Another piece of advice: there is not a central team hard at work implementing all English suggestions for making particular tests more realistic.  Most likely you need to submit a pull request with your proposed changes, if you want to see them adopted.

[Mike Smith]

I am not entirely sure I follow.

In Round12, nodejs beat out both versions of php (5 and 7) in the json test (~225k rps vs. ~175k rps). On the plaintext test where raw throughput is measured, nodejs also finishes on top (~314k vs. ~195k).

Ignoring the fact that the complaint's premise is wrong, there are some decent mentions in this thread about improving nodejs (and presumably, many of the frameworks running on nodejs); we welcome any pull requests to the open source project here: https://github.com/TechEmpower/FrameworkBenchmarks/

Additionally, if a pull request is too involved, you can open an issue instead and simply list out all the suggestions you have aimed at improving NodeJS (or any test) to be more production-realistic.

[Yun Zhi Lin]

@Dimitry if you take a closer look at the node.js example, you can see that it already leverages multi-core via the use of https://github.com/LearnBoost/cluster:

if (cluster.isMaster) {
  // Fork workers.

  for (var i = 0; i < numCPUs; i++) {
    cluster.fork();
  }

Otherwise it would be very difficult to reach 200~300k rps on a single thread :)

I don't think load balancers are relevant here. The benchmarks are not about how well nginx vs haproxy does round robin load balancing. They have their own benchmarks for that: https://github.com/observing/balancerbattle

Regarding @Zetek's original question: I think as technology professionals, we all need to be careful not to fall into religious camps that blindly believes "NodeJs must be faster because it's async non-blocking with lots of npm modules!" Or "Scala must be faster because it's functional and has true monads!". Be pragmatic, use the right language/framework for the right job, and don't be afraid to look into the code when in doubt.

[Daniel Sagenschneider]

@Dmitry  

The long story short.  I would actually suggest adding the following to the startup of the NodeJS server to improve its performance:

process.env.UV_THREADPOOL_SIZE = 20;  // or some higher number as suitable based on tuning for performance

BACKGROUND

Having looked at synchronous threaded architectures vs asynchronous event architectures in detail, it is not always obvious at surface level why performance issues.  In developing OfficeFloor, it uses a combination of both to improve performance.  I will though avoid going into depth here as it is a very detailed topic.  But in basic superficial summary, asynchronous is not always faster due to thread context switching overheads of event handling when using multiple threads to better utilise multi-core architectures (yes excuse the mouth full of words).

However, I was similarly intrigued that developers have left the ease of sequential threaded code and swung to the other side of event driven coding of NodeJS for the scaling benefits that an asynchronous event based architectures provides.   I was similarly intrigued like yourself why NodeJS does not provide this performance improvement of thread-per-request architectures (even granted it is interpreted).   So I did a little background reading on what goes on under the hood of NodeJS, and it seems the "single threaded event loop" is an abstraction and NodeJS does actually use multiple threads under the hood (mainly to do I/O).  I won't go into detail here, but below is links to two articles explaining this (and come with pictures and code examples so better explanation I can give in words :p ):

 - https://www.future-processing.pl/blog/on-problems-with-threads-in-node-js/

 - http://www.journaldev.com/7462/node-js-architecture-single-threaded-event-loop

From going on the architectures explained in these articles, NodeJS still suffers the same performance problem of thread-per-request architectures of a thread listening to I/O and then having to context switch to a worker thread to handle the request.  It actually even seems further worse, in that I/O with the database requires further thread context switching - possibly explaining why it is not higher up in the results.

Note the real problem I see in squeezing out performance is in the management of executing the code.  For the Plain Text and JSON tests there is no I/O involved.  So you want the thread listening to the socket to also process the results (as the cost to service the request being some CPU instructions is less than a thread context switch overheads).  Rapidoid is an example of doing this.  However, when it comes to the tests using database interaction (other tests) the socket thread can not be tied up waiting for the database call to complete.  In this case, the context switch is a lower cost than servicing the request so using multiple threads improves throughput.  But too many threads creates significant overheads and therefore tuning is required.  Hence, why I suggest the above fix for NodeJS to find the I/O thread pool appropriate size for optimal performance.  But I'm no NodeJS expert, and will always defer to them (and put a big disclaimer on this in case I have the NodeJS architecture wrong or outdated).  But I happily help good competition between frameworks to ensure we continue to see improvements and advancements in application/web server architectures :)

[Daniel Sagenschneider]

@Dmity

Also, beyond the long explanation of the details within a single process of NodeJS.   NodeJS in cluster, I believe, runs as separate processes.   This requires interprocess communication overheads that is typically significantly slower compared to threads interacting with shared memory (and the thread context switching overheads involved)

[Andrey Sidorov]

Altering UV_THREADPOOL_SIZE does not make network IO better, it lives in a single thread. It could be helpful for node-libmysqlclient but benchmark is based on node-mysql which does not use thread pool at all.

[Andrey Sidorov]

On Monday, 8 August 2016 14:52:41 UTC+10, Daniel Sagenschneider wrote:

@Dmity

Also, beyond the long explanation of the details within a single process of NodeJS.   NodeJS in cluster, I believe, runs as separate processes.   This requires interprocess communication overheads that is typically significantly slower compared to threads interacting with shared memory (and the thread context switching overheads involved)

Only at startup - master process sends file handle to workers so that they can listen on the same port. There is no overhead afterwards

[Daniel Sagenschneider]

Thanks Andrey. Yes, I've had a brief look at Node's source and does seem the case.

I guess, it intrigues me further then, why node is not more performant in the results.

Rather than reading more of Node's source, you might be able to clear one thing up for me.

Node runs its own queue of tasks within an event loop. Does the thread waiting on Node's tasks in the queue, also wait on socket events?

Or does a separate thread wait on socket events, and then loads relevant tasks onto Node's queue of tasks to be executed by the main event loop?