net.createServer

[Mark Volkmann]

I'm trying to understand a detail about net.createServer().

In the code below, how is it that Node guarantees that no data will be lost if a client immediately writes to its socket when it connects to this server?

Is something buffering writes from the client until the callback function passed to net.createServer completes?

Does the server not read from its socket until that callback completes because something that enables reads is waiting on the event queue?

var net = require('net');

var server = net.createServer(function (socket) {

  // Could the client write to the socket before the next call completes?

  socket.on('data', function (data) {

    console.log('received "' + data + '"');

  });

});

server.listen(8123, function () {

  console.log('listening');

});

--
R. Mark Volkmann
Object Computing, Inc.

[Jorge]

My understanding is that node won't/shouldn't emit/dispatch any 'data' events to the socket 'socket' until *after* having called cb(socket), 'cb' being the callback function passed on to .createServer(cb).
--
Jorge.

[Mark Volkmann]

On Thu, Sep 6, 2012 at 4:40 PM, Jorge <jo...@jorgechamorro.com> wrote:

On 06/09/2012, at 23:11, Mark Volkmann wrote:

 

My understanding is that node won't/shouldn't emit/dispatch any 'data' events to the socket 'socket' until *after* having called cb(socket), 'cb' being the callback function passed on to .createServer(cb).

I suspect you are correct. I'd like to understand how Node implements that. It would be great if this is already documented somewhere and I could just read that to understand it. 

[Hsu Ping Feng]

https://github.com/joyent/node/blob/master/lib/net.js 

It is a good way to understand through source code.

2012/9/7 Mark Volkmann <r.mark....@gmail.com>

--
Job Board: http://jobs.nodejs.org/
Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nod...@googlegroups.com
To unsubscribe from this group, send email to
nodejs+un...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en

--
AUFKLÄRUNG ist der Ausgang des Menschen aus seiner selbstverschuldeten Unmündigkeit. Unmündigkeit ist das Unvermögen, sich seines Verstandes ohne Leitung eines anderen zu bedienen. Selbstverschuldet ist diese Unmündigkeit, wenn die Ursache derselben nicht am Mangel des Verstandes, sondern der Entschließung und des Mutes liegt, sich seiner ohne Leitung eines andern zu bedienen. Sapere aude! Habe Mut, dich deines eigenen Verstandes zu bedienen! ist also der Wahlspruch der Aufklärung.

[ribao wei]

It is not about node.js, it is a Javascript thing. 

Socket will not receive any data util it "connect". 

[Mark Volkmann]

I don't doubt you. I just want to see that documented somewhere or clearly see how that is enforced in the code. I have looked at net.js. It's not the easiest code to follow. If it's clear from that code, I could use some pointers on where to look.

I'll try to make my concern more clear below with very simple code for server.js and client.js that really runs. Run "node server" in one window and "node client" in another.

The numbered comments indicate the order in which I expect the lines to execute.

It seems clear to me that 4 could run before 5. If that happens, how is it that the server still gets that message? Is the message being buffered until connListener completes?

server.js

var net = require('net');

function dataListener(data) {

  console.log('received', data.toString());

}

function connListener(socket) {

  socket.on('data', dataListener); // 5

}

var server = net.createServer(connListener); // 1

server.listen(8019); // 2

client.js

var net = require('net');

var socket = net.connect(8019); // 3

socket.write('Is this lost?'); // 4

[Ben Noordhuis]

The client queues write requests until the connection handshake is
done. Said handshake is complete when the server accepts the
connection, which node does right before it calls your connListener
function.

[ribao wei]

In this case I think the data will be buffered in the kernel (outside of Node), so you don't have to worry about that too much. 

--

[Mark Volkmann]

On Fri, Sep 7, 2012 at 9:09 AM, Ben Noordhuis <in...@bnoordhuis.nl> wrote:

The client queues write requests until the connection handshake is

done. Said handshake is complete when the server accepts the
connection, which node does right before it calls your connListener
function.

If that is true then why is the first message from the client lost if I do this?

Note that on my machine (OSX) the message is lost if the timeout is 2 ms or more, but not if it is less than 2.

Maybe the client queues write requests until the connListener function completes.

server.js

var net = require('net');

function dataListener(data) {

  console.log('received', data.toString());

}

function connListener(socket) {

  setTimeout(function () {

    socket.on('data', dataListener);

  }, 2);

}

var server = net.createServer(connListener);

server.listen(8019);

client.js

var net = require('net');

var socket = net.connect(8019);

socket.write('Is this lost?');

[ribao wei]

Well, in this code connListener finishes execution without dataListener being registered. So I guess that is why the first message is lost. 

--

[Mark Volkmann]

If the server accepts the connection BEFORE connListener is called, then it seems possible that the client could send a message before the server registers for data events. That's why it seems that the client queues messages until AFTER connListener completes. If I'm wrong about this, I'd like to understand why.

[mscdex]

On Sep 7, 10:21 am, Mark Volkmann <r.mark.volkm...@gmail.com> wrote:

> On Fri, Sep 7, 2012 at 9:09 AM, Ben Noordhuis <i...@bnoordhuis.nl> wrote:
>
> The client queues write requests until the connection handshake is
>
> > done. Said handshake is complete when the server accepts the
> > connection, which node does right before it calls your connListener
> > function.
>
> If that is true then why is the first message from the client lost if I do
> this?

Probably because the accept and incoming data happen within the same
tick. This is why you should add a data handler right away, or at
least pause() the socket right away.

[Mark Volkmann]

Sounds reasonable. I'm currently teaching a class on Node.js. I'd like to say something better to the students than "This code probably works because ...". That's why I'm trying to either find a place where this is documented or find where the queuing of messages and releasing of them actually happens in the code.

 

[Jimb Esser]

It sounds like you might be misunderstanding something fundamental about node/Javascript - it's all single threaded, and (at least the networking stuff) is event-based (specifically a stream of sequential events, since there's no concurrency).  In your example, while it's running line 5, there's no way it's going to miss anything because that's the only code that is running, nothing could possibly be pumping the event loop.  Until your function returns from handling the event, node is not going to handle the next event coming from the socket.  Even if the data has been sent and buffered in the kernel for your server process, it's not going to fire any events until node tries to read from it/pumps the event loop.  Lots of node APIs rely on this kind of behavior (e.g. HTTP request/response stuff similarly lets you set up event listeners in a connection event before it could possibly fire any data events).  Hope this helps!

  Jimb Esser

[Mark Volkmann]

On Fri, Sep 7, 2012 at 1:16 PM, Jimb Esser <wast...@gmail.com> wrote:

It sounds like you might be misunderstanding something fundamental about node/Javascript - it's all single threaded, and (at least the networking stuff) is event-based (specifically a stream of sequential events, since there's no concurrency).  In your example, while it's running line 5, there's no way it's going to miss anything because that's the only code that is running, nothing could possibly be pumping the event loop.  Until your function returns from handling the event, node is not going to handle the next event coming from the socket.  Even if the data has been sent and buffered in the kernel for your server process, it's not going to fire any events until node tries to read from it/pumps the event loop.  Lots of node APIs rely on this kind of behavior (e.g. HTTP request/response stuff similarly lets you set up event listeners in a connection event before it could possibly fire any data events).  Hope this helps!

What I think you're saying is that the server will stop queuing messages AND execute the connection listener in the same synchronous tick. So there's no way a queued message will be delivered until after the connection listener runs. That makes sense.

BTW, if I didn't understand that Node is single-threaded, I really shouldn't be attempting to teach a class on it. I do understand that part. ;-)

 

[Jorge]

An off-the-shelf node can't execute *JavaScript* code in parallel (*), because *only* *one* of its *multiple* threads runs a (single) v8 JS VM.

But a node with the threads-a-gogo module can create as many (up to thousands) additional V8 JS VMs as needed, and execute javascript code truly in parallel, in additional threads:

<https://github.com/xk/node-threads-a-gogo>

(*) But in every node process there's always a bunch of threads running... so node's *not* single threaded.
--
Jorge.

[Shripad K]

On Fri, Sep 7, 2012 at 7:05 PM, Mark Volkmann <r.mark....@gmail.com> wrote:

I don't doubt you. I just want to see that documented somewhere or clearly see how that is enforced in the code. I have looked at net.js. It's not the easiest code to follow. If it's clear from that code, I could use some pointers on where to look.

I'll try to make my concern more clear below with very simple code for server.js and client.js that really runs. Run "node server" in one window and "node client" in another.

The numbered comments indicate the order in which I expect the lines to execute.

It seems clear to me that 4 could run before 5. If that happens, how is it that the server still gets that message? Is the message being buffered until connListener completes?

 

4 always runs before 5. The data event cannot be triggered unless something is written to the socket. This is provided the server.js file is run first and then client.js. The other way round will not work as it will throw a ECONNREFUSED on connect() at line 3 (provided the server has not done a listen() yet).

Here is the flow:

Behind the scenes, the event loop is notified of a new connection. It accept()'s the connection. Once accept()ed the connListener callback is triggered and the particular socket (wrapper to the raw socket) is passed as an argument. Listeners for various events (such as "data", "end", "chunk" etc) are setup. Then, a read event handler is registered on the raw socket's FD (UV_READ). When the file descriptor becomes readable, the data is recv()d and is stored in a buffer and "chunk" event is emitted for each data received (recv() returns the size of the data and "errno" is EAGAIN until all data is read into the buffer). When the buffer is full (recv() returns a -1), the "data" event is emitted with the buffer as argument. If recv() returns 0, the socket is deemed to have closed on the other end. The read event handler is then stopped, a full shutdown of the raw socket is performed and the file descriptor is detached from the event loop with the "end" event being emitted.  (i have simplified it a lot. this is how its generally done).

 

--

[Shripad K]

On Sat, Sep 8, 2012 at 12:51 PM, Shripad K <assortme...@gmail.com> wrote:

On Fri, Sep 7, 2012 at 7:05 PM, Mark Volkmann <r.mark....@gmail.com> wrote:

I don't doubt you. I just want to see that documented somewhere or clearly see how that is enforced in the code. I have looked at net.js. It's not the easiest code to follow. If it's clear from that code, I could use some pointers on where to look.

I'll try to make my concern more clear below with very simple code for server.js and client.js that really runs. Run "node server" in one window and "node client" in another.

The numbered comments indicate the order in which I expect the lines to execute.

It seems clear to me that 4 could run before 5. If that happens, how is it that the server still gets that message? Is the message being buffered until connListener completes?

 

4 always runs before 5. The data event cannot be triggered unless something is written to the socket. This is provided the server.js file is run first and then client.js. The other way round will not work as it will throw a ECONNREFUSED on connect() at line 3 (provided the server has not done a listen() yet).

Here is the flow:

Behind the scenes, the event loop is notified of a new connection. It accept()'s the connection. Once accept()ed the connListener callback is

Oops. Should be, the server is notified of a new connection via the event loop. The server accept()'s the connection....