[ANN] thread.js - a high concurrent ipv4/v6 server of node.js addon

[Robert L]

Hi everybody,

I have worte a node.js addon, to support multi thread with v8, and act as a ipv4/v6 server, please give it a try.

Your feedback is very welcome.

Thread.js support:

1. Act as a network layer of node.js.

2. Register a file to thread - you then run javascript file within the thread along with the network layer. Thread.js support require(), console.log, events.Emitter, net.createConnection and typedarray in thread mode for now.

Benchmark is done by using weighttp on a Ubuntu 12.04 LTS 64bits server with a i7 2.67G 4cores cpu.

Performance of a simple 'Hello world' server:

60K+ concurrent connections per second.

150K~200K concurrent requests per second.

And it's scale mainly by cpu core number. You can get higher benchmark by adding more cpu power.

Throughout of 50Kbytes response server:

using typedarray: 1.7Gbytes per second.

using javascript string: 290Mbytes per second.

Stability:

Each connection object is reused, thus the memory allocation is just at new age, don't need to face gc problem, and it can run days to weeks without restart the program.

You can prove it by running node with v8 flag '--trace_gc --trace_gc_verbose'

Detail benchmark output and test script is supported.

Because it's written with the support of pthread and epoll , thread.js is Linux only.

And please support me if it make some useful for your application. Thanks very much.

homepage: http://rob333.github.com/thread.js/

github: https://github.com/rob333/thread.js

[Micheil Smith]

Hi Robert,

How's this compare to a Node.js server running with Cluster (API in 0.8)?

I expect it to certainly be better than just a single node process running a net.Server,
but if that process has been setup with Cluster, then you'd have one server per core
to deal with the shared socket (or, at least, that's my understanding of it).

Kind Regards,
Micheil Smith
--
BrandedCode.com

> --
> Job Board: http://jobs.nodejs.org/
> Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
> You received this message because you are subscribed to the Google
> Groups "nodejs" group.
> To post to this group, send email to nod...@googlegroups.com
> To unsubscribe from this group, send email to
> nodejs+un...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/nodejs?hl=en?hl=en

[Kevin Jones]

This looks very cool, appears to fit in nicely between threads-a-go-go (threads without event loops) and the approach I took of modifying node to get threads with event loops but not installable as a module. My main interest was in reducing communication overheads between cooperating tasks for some other work I want to get to. I take it from the examples you are more concerned with achieving high throughput without cluster or is there a larger objective here as well?

Regards

Kev

[Robert L]

Hi Micheil,

You have mentioned me what I have miss.

I was only focused on the performance of http service.

After I have done the benchmark with cluster using `net`. I was surprised that `net` are not that slow as`http`.

Here is the result: (with node v0.8.8)

>ab -n 200000 -c xxxx 127.0.0.1:8080/

200client 1000client 2000client

node_cluster 21001.89 19000.00   18717.28

tj wite node 23792.30 23175.49   22707.17

tj simple 24220.45 23182.52   22709.86

>ab -n 200000 -c xxxx -k 127.0.0.1:8080/ 

200client 1000client 2000client

node_cluster 134403.36 136077.29  116258.52

tj wite node 124507.96 109830.13  103300.93

tj simple 124113.37 105600.24  100807.72

>weighttp -n 200000 -c xxxx -t 8 127.0.0.1:8080/

200client 1000client 2000client

node_cluster 48644 41211    43047

tj wite node 78133 76878    72929

tj simple 78061 73478    72209

>weighttp -n 200000 -c xxxx -t 8 -k 127.0.0.1:8080/

200client 1000client 2000client

node_cluster 168867 163490          161464

tj wite node 179399 195751          178095

tj simple 289362 281941          256841

Thread.js get a bigger advantage when benchmark with weighttp that get multi thread support.

Currently I implement a simply http service in javascript and it can achieve 60,000+  both connections/sec and requests/sec.

I will try to implement a method for programs to register their protocol. When that time, I'll do another benchmark base on http service.

Sincerely,

Robert

Micheil Smith於 2012年8月24日星期五UTC+8上午5時20分08秒寫道：

[Micheil Smith]

So, correct me if I'm wrong, but your initial benchmark results were skewed
because of how you've implemented your HTTP module.

That said, you do see to get some performance boost over the standard net
module, I'm guessing this may be because you're not communicating over
STDOUT/STDIN pipes to child processes, but instead directly to the
individual threads?

– Micheil

On 24/08/2012, at 4:05 PM, Robert L wrote:

> Hi Micheil,
>
> You have mentioned me what I have miss.
> I was only focused on the performance of http service.
> After I have done the benchmark with cluster using `net`. I was surprised that `net` are not that slow as`http`.
>
> Here is the result:

> >ab -n 200000 -c xxxx 127.0.0.1:8080/
> 200client 1000client 2000client
> node_cluster 21001.89 19000.00 18717.28
> tj wite node 23792.30 23175.49 22707.17
> tj simple 24220.45 23182.52 22709.86
>
>
> >ab -n 200000 -c xxxx -k 127.0.0.1:8080/
> 200client 1000client 2000client

> node_cluster 21001.89 136077.29 18717.28
> tj wite node 23792.30 109830.13 22707.17
> tj simple 24220.45 105600.24 22709.86

>
>
> >weighttp -n 200000 -c xxxx -t 8 127.0.0.1:8080/
> 200client 1000client 2000client
> node_cluster 48644 41211 43047
> tj wite node 78133 76878 72929
> tj simple 78061 73478 72209
>
>
> >weighttp -n 200000 -c xxxx -t 8 -k 127.0.0.1:8080/
> 200client 1000client 2000client
> node_cluster 168867 163490 161464
> tj wite node 179399 195751 178095
> tj simple 289362 281941 256841
>
>
> Thread.js get a bigger advantage when benchmark with weighttp that get multi thread support.
> Currently I implement a simply http service in javascript and it can achieve 60,000+ both connections/sec and requests/sec.
> I will try to implement a method for programs to register their protocol. When that time, I'll do another benchmark base on http service.
>
> Sincerely,
> Robert
>
>
>
> Micheil Smith於 2012年8月24日星期五UTC+8上午5時20分08秒寫道：

[Robert L]

Hi Kevin,

Actually I was facing a problem of sending larger response to client at extreme low throughout.

It is caused by cast javascript string to char* with big string, then I found out that by using typedarray (external data in v8), will outperform every web server that we common know. It is a surprised, too.

I want to achieve good performance by threading and take out of gc freeze.

After benchmarking the cluster with `net`, I think I need to adjust the roadmap of thread.js.

It seems libuv is fine, but there is something between libuv and the javascript. I need to think about it.

And someone has mentioned about a game engine, v8 as a script engine, I think that is very cool, too.

Sincerely,

Robert

Kevin Jones於 2012年8月24日星期五UTC+8下午4時18分57秒寫道：

[Robert L]

I need to study on this, why `tj simple` get that much compares to `tj with node` before I can be certain about it.

But I can say, after writeing thread.js, c/c++ module make the most difference by comparing to javascript code and this is relate to using threads. Then, reusing javascript objects is second the most issue to take care of.

Best Regards,

Robert

Micheil Smith於 2012年8月24日星期五UTC+8下午11時12分20秒寫道：

[Robert L]

Hello everyone,

I'm glad to announce a new release of thread.js.

I have done two major feature in this release.

First, I have implemented a service handler interface that people can write their own protocol and user can install it via npm, and I have done a simple http service handler and run benchmark with it.

Second, I thought event listener may divide into two categories - dynamic event and static event.

Dynamic event listeners mostly used by client applications that they need a array to capacity a event and may need a customized event.

On the other hand, almost all server side event listeners should be static, they always get only one callback function with a event and never need a customized event.

Base on this assumption, I then implement a event listener cache. This save a lot of time when event was emitting and go through the object lookup process then call the function and it's really gain a huge performance boost.

Here is the new benchmark with net module and http module both get a connection/sec and a keep-alive request/sec test.

*threadjs uses the simple_http service handler in http module test.

net connection/sec

      node_cluster   node with tj   threadjs

 200  48352          81343          83121

 500  47978          78007          81971

 750  40199          77905          76840

1000  47170          73458          78905

1250  41250          72063          76738

1500  44624          73598          74492

1750  40386          72416          75769

2000  35174          73918          75174

net keepalive request/sec

      node_cluster   node with tj   threadjs

 200  173069         297282         383607  

 500  174376         311152         376256  

 750  177343         315716         374010  

1000  178202         296393         365689  

1250  174102         283776         345388  

1500  174480         276573         336369  

1750  171244         266155         319608  

2000  170675         267328         313555  

http connection/sec

      node_cluster   node with tj   threadjs

 200  31552          82054          81518

 500  29468          78882          80104

 750  30008          75741          75938

1000  29278          71373          77577

1250  14400          72631          75945

1500  13859          76791          75165

1750  12417          72147          74491

2000  12667          73394          74089

http keepalive request/sec

      node_cluster   node with tj   threadjs  

 200  70151          213382         329009

 500  70231          230153         328236

 750  71910          223123         329234

1000  58294          216859         321123

1250  69058          212391         307097

1500  69118          210384         298151

1750  66007          205793         287315

2000  69120          203913         279819

Do you think it is useful? Please tell me what do you need or what do you think about it.

I was planned to implement timer and process module next.

Cheers.

-Robert

[1] github: https://github.com/rob333/thread.js 

[2] homepage: http://rob333.github.com/thread.js/

[Nuno Job]

Can you provide a better explanation of what the project his, and what the benchmarks refer to?

I was curious to understand but didn't really grasp it with the email you sent alone. :)

Nuno

[Mark Hahn]

Yes, an explanation of how it is implemented would help a lot.

--

[Robert L]

Hi Nuno,

You can think of threadjs as a cluster module that works on threads instead of processes and focus on high performance networking.

Currently, it just provides a limited module system to run on the thread.

But you can dispatch works to nodejs and that leaves threadjs just a network engine.

As of the benchmarks, the line

net connection/sec

means "use net module and benchmark in one request per connection mode i.e. http/1.0 method"

net keepalive request/sec 

means "use net module and benchmark in one connection many requests mode i.e. http/1.1 keepalive method" 

and

http connection/sec 

http keepalive request/sec 

get the same meaning but just use http module for the benchmarks.

node_cluster: means to use nodejs cluster module for the benchmark.

node with tj: means to use threadjs but pass javascript code back to nodejs.

threadjs: means to use threadjs and runs javascript on threads.

And the first column of the data means the concurrent connections number. i.e. the -n xxxx parameter of the benchmark tool.

Hope this helps.

Thanks,

Robert

[Robert L]

Hi Mark,

I have draw a diagram to simply illustrate the system flow.

Thanks,

Robert.