Question about threads in node.js

[Alberto Gori]

Is this guy confused about threads in node.js?

http://letsnode.com/example-of-what-node-is-really-good-at

Recaping, he says that traditional thread programming incurs in performance penalties because of cost of context switching and thread creation. 

AFAIK, it's true that node.js is hiding threads from end developer, but it's still using them internally. For example an HTTP call to a website uses a thread. Node.js is not magic, otherwise how can it execute  portion of code asynchronously without using any threads?

Am I wrong?

[mscdex]

On Nov 7, 5:00 pm, Alberto Gori <forzaton...@gmail.com> wrote:
> Am I wrong?

The only place the thread pool in node is used (AFAIK) is for async
file system calls. So, network I/O does not use extra threads as you
suggest. There is one main thread where the event loop lives and where
everything happens (except async file system calls as previously
noted).

[Isaac Schlueter]

Node uses threads for file system IO and for some slow CPU-intensive
operations, and for system calls that are not available
asynchronously, and for spawning child processes (since you can't
actually do that without a fork call.)

It does *not* use threads for async network IO, because it's
unnecessary, and it certainly does not spawn a thread for each request
to an HTTP server, or for each outbound HTTP request it makes.

> --
> Job Board: http://jobs.nodejs.org/
> Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
> You received this message because you are subscribed to the Google
> Groups "nodejs" group.
> To post to this group, send email to nod...@googlegroups.com
> To unsubscribe from this group, send email to
> nodejs+un...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/nodejs?hl=en?hl=en

[Alberto Gori]

Very interesting. So asynchronous I/O usually relies on specific kernel procedures, and only in few situations it's emulated using threads.

[Marco Rogers]

It took me a while to get a good conceptual understanding of the node async networking layer (I still don't have a concrete understanding). I just did a quick search around and didn't really find anything. I think it's really interesting and instructive to get a handle on how different OSes enable asynchronous networking and the choices that were made in node. And how all these eventually coalesced into libuv. There's a great resource on libuv here http://nikhilm.github.com/uvbook/index.html. But it doesn't go into the underlying implementation much.

If someone has better resources, I encourage them to post here.

:Marco

[Ben Noordhuis]

On Thu, Nov 8, 2012 at 12:04 AM, Isaac Schlueter <i...@izs.me> wrote:
> Node uses threads for file system IO and for some slow CPU-intensive
> operations, and for system calls that are not available
> asynchronously, and for spawning child processes (since you can't
> actually do that without a fork call.)
>
> It does *not* use threads for async network IO, because it's
> unnecessary, and it certainly does not spawn a thread for each request
> to an HTTP server, or for each outbound HTTP request it makes.

Sorry Isaac, I'm afraid I have to amend that a little. :-)

* Child processes are spawned synchronously. (But afterwards, process
management is event-driven.) That may change if it becomes an issue
but that hasn't been the case so far.

* While HTTP traffic is event-driven, the initial DNS lookup is done
from inside the thread pool. That's because we use getaddrinfo(3)
instead of c-ares, mostly because the latter doesn't support things
like mDNS (think Bonjour, Avahi, etc.). I may switch it to
getaddrinfo_a(3) on Linux someday but other platforms will keep on
using the thread pool.

[Bert Belder]

On Thursday, November 8, 2012 12:45:25 AM UTC+1, Ben Noordhuis wrote:

* While HTTP traffic is event-driven, the initial DNS lookup is done
from inside the thread pool.  That's because we use getaddrinfo(3)
instead of c-ares, mostly because the latter doesn't support things
like mDNS (think Bonjour, Avahi, etc.).  I may switch it to
getaddrinfo_a(3) on Linux someday but other platforms will keep on
using the thread pool.

Hey Ben, let's amend that a little more.

I also plan to use the overlapped variant of GetAddrInfoEx on version 8 / server 2012 of a particular operating system. This would make DNS lookups fully async.

- Bert 

[Matt]

On Wed, Nov 7, 2012 at 6:45 PM, Ben Noordhuis <in...@bnoordhuis.nl> wrote:

While HTTP traffic is event-driven, the initial DNS lookup is done
from inside the thread pool.  That's because we use getaddrinfo(3)
instead of c-ares, mostly because the latter doesn't support things
like mDNS (think Bonjour, Avahi, etc.).  I may switch it to
getaddrinfo_a(3) on Linux someday but other platforms will keep on
using the thread pool.

Didn't I see somewhere a Node DNS library written in pure Node? That might be useful...

[Nuno Job]

github.com/iriscouch/named ?

Its used to run npm

nuno

On Fri, Nov 9, 2012 at 3:50 PM, Matt <hel...@gmail.com> wrote:

ee somewhere a Node DNS

[Nuno Job]

btw i would love to have the dns server (named) and indutny's spdy implementation in core *hint* :P

[Ben Noordhuis]

You mean [1]? It has the same issue as c-ares in that it doesn't
integrate with platform-specific services. mDNS is one but there are
others, like NIS or NSS.

[1] https://github.com/tjfontaine/node-dns

[Matt]

Yeah that was the one. I thought mDNS was just DNS packets sent over UDP multicast? With a library that can parse DNS packets we should be able to see mDNS packets in userspace, no?

[Bradley Meck]

Correct but people use factory functions based on string lookups in most of these modules for some forsaken reason (udp/tcp do not have same api basically).