dumb question about asynchronous function execution

[José de Zárate]

I don't really know how node knows when to run a function asynchronously or not.

Let's say you have this piece of code.

function gen_url(str) {
  return 'http://' + str;
}

full_url = gen_url('www.google.com');

In that case I'm obviously counting on the function gen_url being executed synchronously.

Now If I have

function hit_mysql(query, callback) {
  var mystuff = oldstuff * system_variable;
  client = require('mysql').Client();
  client.query(query, callback);
}

var one = 'one';
hit_mysql('select ...', function(){console.log('done');});
var two = 'two';


I would expect hit_mysql to run asynchronousy.


How does node figure this out? I'm no "marking" any function as "sync" or "async". Does node througfuly examine the function's body to see if somewhere in the function body (or in the body of the functions defined in the function body) there is some well recognized native node async call, like setTimeout, or http.listen(), or whatever? 
--
uh, oh.

[Mark Hahn]

>  I would expect hit_mysql to run asynchronousy. 

It doesn't.  It runs and returns synchronously (although the require is blocking and will slow up your server).  It is only later that you get the callback. 

2012/1/17 José de Zárate <jza...@gmail.com>

--
uh, oh.

--
Job Board: http://jobs.nodejs.org/
Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nod...@googlegroups.com
To unsubscribe from this group, send email to
nodejs+un...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en

[Scott González]

This is an oversimplification, but it's probably all you're looking for:

The implementation of the actually async functions, such as setTimeout, inform node that they are async.

2012/1/17 José de Zárate <jza...@gmail.com>

I don't really know how node knows when to run a function asynchronously or not.

--
uh, oh.

[Mark Hahn]

Every function always returns quickly, even if it has async calls in it.  The only thing that is async is the callback.

2012/1/17 Scott González <scott.g...@gmail.com>

[Scott González]

I could be wrong, but I don't think he's confused about that. I understood the question to be "How does node know when there are pending async operations so that it can keep the process alive?"

[JoeZ99]

That's precisely my point. I know the "callback" is what is run asynchronously, but , looking at the responses so far, I think the only functions that are executed asynchronously are the one node natives function that are defined as async. node executes every instruction synchronously until it finds a node native async function, at which point it creates a new stack (or whatever the term is. I'm no expert).

That would mean that I can not make an async function unless it contains some of the node async functions within (or at least within some of the nested functions). Simply defining a "callback" is not guaranteed to create async processing.

Hope you will correct me if I'm wrong.

[Dobes]

On Jan 18, 6:29 am, José de Zárate <jzar...@gmail.com> wrote:
> I don't really know how node knows when to run a function asynchronously or
> not.

At some point in your call to client.query() there will be eventually
be a call to one of node's internal socket functions. That call will
accept a callback and return control to its caller immediately,
calling the callback later when the socket operation is complete. It
won't be a javascript function at that point, it'll be a "native"
method which tells node's I/O mechanisms what to watch for and what
callback to call when it sees it.

Thus, functions that are built using asynchronous calls are
asynchronous and function that do not use asynchronous calls
synchronous.

Note that in some cases functions that take a callback may run
synchronously and invoke their callback before they return. For
example the forEach() method on an array invokes the callback before
returning. Sometimes calls that are normally asynchronous will invoke
the callback before returning if the desired result can be gotten
without making any asynchronous calls (such as when the data is
fetched from a cache that operates synchronously).

Also note that when you call an asynchronous method the code after
that call continues to run. You could invoke several asynchronous
operations in a row and they would run in parallel from the
perspective of your invoking code. Only the invokation of the
callback is delayed until the operation is complete, and that
invokation takes place in a new context where the original funciton is
no longer running (although its local variables are still available).

Hope that helps!

Dobes

[José de Zárate]

yep!! txs. Sometimes a little dumbiness is good.

--
Job Board: http://jobs.nodejs.org/
Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nod...@googlegroups.com
To unsubscribe from this group, send email to
nodejs+un...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en

--
uh, oh.

[t...@creationix.com]

This is exactly what process.nextTick is for. It defers your callback to a new stack.

[José de Zárate]

Ok, then a user can build async functions without using any of the native async calls of node, just by calling process.nextTick(callback).

is that right??
I love it when an initial questions gets answers that goes way beyond its initial scope

--
uh, oh.

[Scott González]

Yes, process.nextTick(callback) will execute the callback asynchronously. If all you want to do is cause the function to be executed asynchronously, process.nextTick() is better than setTimeout().

It's worth mentioning that this is really important to avoid a scenario that Dobes mentioned:

"Sometimes calls that are normally asynchronous will invoke
the callback before returning if the desired result can be gotten
without making any asynchronous calls (such as when the data is
fetched from a cache that operates synchronously)."

You should always avoid having functions that are sometimes synchronous and sometimes asynchronous. If you have to be async sometimes, use process.nextTick() when you can be sync, to ensure that you're consistently async. Having a function that changes behavior like this can cause unexpected bugs to popup on a cache hit.

2012/1/18 José de Zárate <jza...@gmail.com>

[Dobes]

Hi,

"process.nextTick()" is one of node's built-in async calls, it's
simpler than others in that it calls its callback very quickly in a
new call stack without doing any external I/O.

Note, however, that asynchronous functions should only be used where
it is beneficial to do so. If you aren't doing any I/O then you can
make your function synchronous instead. Synchronous function calls
are easier to read in the code and easier to debug.

I wouldn't recommend using process.nextTick() in a situation where a
synchronous function could do the job UNLESS you are implementing an
API that is designed to support asynchronous implementations.

For example, you might have created an asynchronous API for reading a
record from a database. However, for testing purposes you create an
in-memory database that is synchronous. Since you want the API to be
the same as the asynchronous one you can pretend to be asynchronous by
using process.nextTick() to send back the result asynchronously. The
other case I already mentioned is when there's a cache that can return
the result from memory without performing external I/O - in this case
process.nextTick() is helpful in avoid bugs where the caller assumed
the result would always come asynchronously.

Hope that helps!

Dobes

> uh, oh <http://www.youtube.com/watch?v=GMD_T7ICL0o>.
>
> <http://windows7sins.org/>

[Scott González]

Computationally expensive functions that can be synchronous should often be asynchronous as well.

[José de Zárate]

Just for the record. I don't intend of forcing anything into being async, I just wanted to know how that works. Now it's cristal clear

thank you!!!

2012/1/19 Scott González <scott.g...@gmail.com>

--
uh, oh.

[Tim Caswell]

Not to nitpick, but JavaScript in node never "executes" asynchronously.  It is single threaded.  Only one stack is ever executed at once.  What's parallel is the time between events.  I can open 10 files in parallel and node will spawn threads internally to open them for me.  My callbacks for these 10 files will gets called at some later date when the disk finishes with it's work.  However, once one of my callbacks gets called, it owns the process.  No other work will get done until I return.  This all works great as long as you never block the event loop.

If you have some cpu intensive task that takes a long time, then that's bad for node.  All javascript executin is blocking.  This is normally ok because we're just setting up other background tasks (listening for packets, starting FS operations, starting timers, etc..) and returning right away.  If I were to count to a million a million times in two nested for loops, that would be very bad. Using process.nextTick won't help here because it will just defer your expensive task to a later task. But once your callback gets called, the event loop is blocked till you return.  Usually it's best to put these kinds of tasks in different processes (probably even using not node, it's not the best for CPU intensive tasks).  If you must do a long running CPU task in node, you'll have to manually break it into small enough chunks and execute each one on it's own stack using recursive process.nextTick calls.

2012/1/19 José de Zárate <jza...@gmail.com>

[Scott González]

On Thu, Jan 19, 2012 at 3:39 PM, Tim Caswell <t...@creationix.com> wrote:

If you have some cpu intensive task that takes a long time, then that's bad for node.  All javascript executin is blocking.  This is normally ok because we're just setting up other background tasks (listening for packets, starting FS operations, starting timers, etc..) and returning right away.  If I were to count to a million a million times in two nested for loops, that would be very bad. Using process.nextTick won't help here because it will just defer your expensive task to a later task. But once your callback gets called, the event loop is blocked till you return.  Usually it's best to put these kinds of tasks in different processes (probably even using not node, it's not the best for CPU intensive tasks).  If you must do a long running CPU task in node, you'll have to manually break it into small enough chunks and execute each one on it's own stack using recursive process.nextTick calls.

Yes, I should have been more clear. I was trying not to go into detail about the various ways to properly handle computationally expensive tasks. I was not implying that just delaying the expensive task to the next tick was a good idea, that it may have come across that way with my terse response.

[Daniel Kuffner]

Just to get sure that I understand that correctly. 

"process.nextTick" will not really do an async execution. All it does is to delay the execution to the next tick in the same thread. In other words you give all other queued callbacks a chance to be executed before your code gets executed. So there is no benefit to delay a lightweight operation to the nextTick. But as soon you have a heavy operation you can think about it to break it down into smaller batches.

does that makes sense? Or did I understand nextTick wrong?

2012/1/19 José de Zárate <jza...@gmail.com>

[Tim Caswell]

That's correct.  As mentioned earlier in the thread. The main use for process.nextTick is to defer some action till a later tick as cheaply as possible.  For example, a function that does a database query finds the result in an in-memory cache and knows the result on the first tick.  If you call the callback right away, this could break the caller's code if they had assumed it was an async function.

Nothing in node is parallel except for I/O and timer waits. JavaScript is always single threaded and blocking.  Which is nice, you don't have to worry about getting pre-empted except at continuation boundaries (callbacks).

[Mark Hahn]

>  If you call the callback right away, this could break the caller's code if they had assumed it was an async function. 

But it would be the caller's fault.  No one should assume anything about callbacks. I can't think of any use case that can't be coded properly for this.

[Paddy Byers]

Hi,

It might be more helpful to think of node callbacks - nextTick included - as asynchronous but non-preemptive. Asynchronous in the sense that the event is not synchronous relative to any other code you are running,  including the calls that may have triggered the event. They are also asynchronous in the sense that they can occur in any idle window, at any time, and in any order.

However, they are non-preemptive in the sense that they will never cause a current thread of execution to be pre-empted; once a javascript function has been entered from the event loop, it will continue to completion unconditionally until it has returned fully (even if that's by an exception) or the process has exited.

Paddy

[Richard Miller-Smith]

Well, it might be considered the caller's fault, but it is an issue that's caught me out a couple of times and had me digging around other's modules to check whether a function with a callback is asynchronous (which, due to the function signature you tend to assume) or it calls the callback within itself. I've even come across some modules where the documentation says a function is async when it blatantly isn't.

The typical kind of issue you see is things like this give unexpected behaviour:

var state = "none" ;

function foo () {

  iThinkThisIsAsync(function () {

    state = "done" ;

  }) ;

  state = "started" ;

}

On 19 January 2012 22:40, Mark Hahn <ma...@hahnca.com> wrote:

>  If you call the callback right away, this could break the caller's code if they had assumed it was an async function. 

But it would be the caller's fault.  No one should assume anything about callbacks. I can't think of any use case that can't be coded properly for this.

[Mark Hahn]

This  should clearly be ...

function foo () {

  state = "started" ;

  iThinkThisIsAsync(function () {

    state = "done" ;

  }) ;

}

Coding it your way is a clear bug.

[Mike Pilsbury]

What about this (from Node docs)?

var net = require('net');
var client = net.connect(8124, function() { //'connect' listener
  client.write('world!\r\n');
});

This pattern is very common in Node. If net.connect were to become synchronous, the client variable would not be set when the 'connect' callback is called. I can't think how this could be re-written to work.

[pushpinder rattan]

Th call back mechanism of nodejs executes the functions in asynch mode.nodejs is event driven scripting laguage with callback mechanism, though it is not multithreaded like java.

2012/1/18 José de Zárate <jza...@gmail.com>

I don't really know how node knows when to run a function asynchronously or not.

--
uh, oh.

[Tim Caswell]

There is no one right way on the issue of sometimes calling a callback
synchronously. There are pros and cons to both sides.

Supposing I'm writing a database query (or any I/O related request
really) and I have an in-memory cache. In the first line of my
function (or near it), I will often have code like this:

var cache = {};
function loadResource(key, callback) {
if (cache.hasOwnProperty(key)) {
callback(null, cache[key]);
return;
}
...
}

This is fine and works, except if the caller had assumed that the
function was always async. Simply saying it's caller's fault is
really dumb. What you can do that's responsible is document in your
signature for this function that the callback may be called
synchronously. Then hope that all users of your library actually read
the docs and see this warning. Then it's on them if they don't guard
against this hazard.

The other option is to not have the hazard! For example:

var cache = {};
function loadResource(key, callback) {
if (cache.hasOwnProperty(key)) {
var value = cache[key];
process.nextTick(function () {
callback(null, value);
});
return;
}
...
}

Here the caller can safely assume that their callback won't be called
in the same tick that they set it up. (Though it's still good to
document this fact, especially if the API is mixed in style). This is
a very powerful assumption and makes life much easier for the caller.
The downside is your code is slightly slower. The nextTick mechanism
is pretty fast, but not as fast as a synchronous call. And remember,
performance is all relative. This only matters in certain hot path
cases.

I've written libraries exposing both styles. Sometimes I really need
the performance of a synchronous call and had to adjust my calling
code to take this into account. It made my code significantly harder
to read and often a landmine field of potential bugs. My
recommendation is that whenever you can afford the slight performance
cost, make your API consistent and always call the callback
asynchronously. Especially in the case of a in-memory cache in front
of a expensive resource, this is a no-brainer. It's already several
orders of magnitude faster than actually making the call.

-Tim Caswell

[Isaac Schlueter]

If an API is synchronous, it must always be synchronous, no exceptions.
If an API is asynchronous, it must always be asynchronous, no exceptions.

We have process.nextTick if there's no actual async work to do. If
Server.listen becomes synchronous ever, then the cb will be processed
on the next tick.

[Jorge]

On 21/01/2012, at 18:43, Isaac Schlueter wrote:

> If an API is synchronous, it must always be synchronous, no exceptions.

Events are asynchronous, but EventEmitter.emit() emits synchronously.
--
Jorge.

[Isaac Schlueter]

On Sat, Jan 21, 2012 at 09:50, Jorge <jo...@jorgechamorro.com> wrote:
> Events are asynchronous, but EventEmitter.emit() emits synchronously.

No. *IO* is asynchronous. EventEmitter.emit() is synchronous, always.

If this was not the case, then you wouldn't be able to properly handle
http requests, because you'd randomly miss the first "data" event some
of the time.

By the same token, fs.open(file, cb) is asynchronous (it asks the
system to do some IO, and then call the some time cb in the future),
but cb(er, result) is synchronous (it calls the function NOW.)
EventEmitter.emit() is like calling the function. It might be called
at some time in the future, but *when it's called*, it's called
immediately.

[Bruno Jouhier]

What about caching? Functions that cache results are async when cache
is missed and sync otherwise. Why force such functions to go through
process.nextTick when the result is obtained from cache?

[Isaac Schlueter]

On Sat, Jan 21, 2012 at 10:03, Bruno Jouhier <bjou...@gmail.com> wrote:
> What about caching? Functions that cache results are async when cache
> is missed and sync otherwise. Why force such functions to go through
> process.nextTick when the result is obtained from cache?

Because you'll run into strange edge cases where code like this works
one way part of the time, and another way when cached:

var bar
var foo = createServer(arg, function (er, result) {
// can you access foo in here?
bar = 10
})
// is bar 10 or undefined here?

So far, every time I've broken this rule, it's been a disaster
eventually. You just have to start to have a pain-aversion reaction
to APIs that are not contractually obligated to be either async or
sync, all the time.

Great explanation with better real-world examples:
http://blog.ometer.com/2011/07/24/callbacks-synchronous-and-asynchronous/

[Jorge]

On 21/01/2012, at 19:01, Isaac Schlueter wrote:
> On Sat, Jan 21, 2012 at 09:50, Jorge <jo...@jorgechamorro.com> wrote:
>> Events are asynchronous, but EventEmitter.emit() emits synchronously.
>
> No. *IO* is asynchronous. EventEmitter.emit() is synchronous, always.
>
> If this was not the case, then you wouldn't be able to properly handle
> http requests, because you'd randomly miss the first "data" event some
> of the time.

What are you saying Isaac ?

You would not miss anything if .emit() was not synchronous, it would simply emit a turn (of the event loop) later (that is: asynchronously).
--
Jorge.

[C. Mundi]

I apologize for prolonging this thread, but the discussion is very good. 

Tell me if I have these points right. 

0. Callbacks are no more "magical" in node than in any language with first-class functions.

1. JavaScript/V8 is single-threaded and never gets prempted. 

2. process.nextTick simply defers/enques a function to the next cycle of the (one and only) event loop.

3. All of the above is 100% synchronous and non-premptive, but...

4. Node provides JavaScript hooks to access a shared thread pool in the node process.  These hooks are used by the authors of slow io libraries, for example to open files or access disk-based databases.

5. If I need a function to execute asynchronously or ...more precisely... be nonblocking, then I need to make use of that shared thread pool, either by piggybacking on an existing node function which is nonblocking or by learning to use the hooks myself.

Question: Are the hooks exposed in JavaScript, or do I need to write native C code?

The application I have in mind to write does compute-intensive tasks, which is the CPU analog of slow disk.

Thanks!

[Scott González]

If emitting were asynchronous, it would be useless in synchronous code. That would be really bad.

[Bruno Jouhier]

Your createServer function is unusual in that it has a callback
parameter AND it also returns a result. A typical function that
fetches something asynchronously (with or without cache) will only
have a callback parameter, and the code that handles the result will
always be in the callback.

If the async function does not "return", the question about foo goes
away. And, regarding bar, I would ban code that accesses bar after the
createServer call (except if the intent is precisely to test if
createServer executed synchronously or not).

We have a large body of node.js code, with lots of functions that do
caching and invoke their callback synchronously when the cache is hit.
So far I've never run into any bad surprises with this design.

On Jan 21, 7:12 pm, Isaac Schlueter <i...@izs.me> wrote:

> Great explanation with better real-world examples:http://blog.ometer.com/2011/07/24/callbacks-synchronous-and-asynchron...

[Isaac Schlueter]

Jorge,

Consider an upload server that did what you're suggesting:

http.createServer(function (req, res) {
process.nextTick(function () {
var stream = fs.createWriteStream(filename)
req.on("data", function (c) {
// write data to file
stream.write(c)
})
req.on("end", function () {
// request is over
// close file
stream.end()
res.writeHead(200, {})
res.end("got it!")
})
})
}

Because you're not responding to the "request" event synchronously,
you're not guaranteed to be able to attach the "data" event handler
before the first "data" event fires. The parser calls the "onrequest"
function, which emits the "request" event in the server, and then
continues right on through whatever data it has. There is no time to
waste, and you're betting that the body and the headers will never
arrive in the same TCP chunk.

In fact, if you tried to do this in production, you wouldn't *just*
get lots of corrupted files missing their first chunk: since files
smaller than a single packet might be uploaded in one data event, you
might also miss the "end" event, and never send a response! So, users
uploading *small* files would be more likely to see it hang forever
when they try.

.emit("event") is synchronous, always. It's an in-memory operation.
It loops through an array of functions, calling them one after the
other. (Plus some serious optimizations for certain common use
cases.)
.on("event", fn) is synchronous, always. It puts the function in the
list of functions to call when "event" is emitted.
fs.read(..., cb) is asynchronous, always. It schedules the cb to be
called once the IO is completed.

[Jorge]

Here you're delaying the setting of the listeners/event handlers to the nextTick. Why ? This has nothing to do with .emit() being synchronous.

The point is that the server you create with .createServer() (the EventEmitter) .emit()s *asynchronously*, unlike what happens when you call .emit() on an EventEmitter (e.g. on it, on said server).

That's why I pointed to you in my previous email, that .emit() is synchronous ( -> when you call it explicitly) but it's asynchronous when the server returned by .createServer() calls it.

You said: "If an API is synchronous, it must always be synchronous, no exceptions"

And here we have an example of the opposite: the server emits asynchronously, but calling .emit() on it makes it emit synchronously.
--
Jorge.

On 21/01/2012, at 20:25, Isaac Schlueter wrote:

> Jorge,
>
> Consider an upload server that did what you're suggesting:
>
> http.createServer(function (req, res) {
> process.nextTick(function () {
> var stream = fs.createWriteStream(filename)
> req.on("data", function (c) {
> // write data to file
> stream.write(c)
> })
> req.on("end", function () {
> // request is over
> // close file
> stream.end()
> res.writeHead(200, {})
> res.end("got it!")
> })
> })
> }

> <snip>

[Jorge]

On 21/01/2012, at 19:45, C. Mundi wrote:

The application I have in mind to write does compute-intensive tasks, which is the CPU analog of slow disk.

For that you need the threads-a-gogo module. It lets you move compute-intensive tasks out of the event loop, to background JS threads.

-- 

Jorge.

[C. Mundi]

Ah!  Thank you. I will check it out.

--

[Mark Hahn]

> This pattern is very common in Node.

var net = require('net');
var client = net.connect(8124, function() { //'connect' listener
    client.write('world!\r\n');
});

Yes, this is not the caller's fault.  I was wrong.

However, I consider this a node design fault.  One should never return data both synchronously and asynchronously.  It is inherently wrong.

[C. Mundi]

I'm a noob, but I don't see anything wrong by design in letting a function which (synchronously) returns a value also accept a callback which happens to perform asynchronously. 

I fail to see why the programmer should be denied this flexibility, which after all is really just syntactic sugar for wrapping the async call not returning a value inside a sync call returning a value.  Deny me this, and I will reinvent it.

Am I missing something?

Thanks.

--

[Mark Hahn]

>   which after all is really just syntactic sugar for wrapping the async call  

I couldn't parse that sentence but it is not just sugar.  As the example above shows things will break if the callback is synchronous.  You should be able to write code that knows nothing about whether the callback is sync or async.

Even if you document that a callback is always async, it locks you into that and you don't have the option to change how a function works in the future.  So optimization like caching (memoization) is impossible.

[C. Mundi]

I agree that you should be able to write code to whatever API contract you want.  But please don't take away my ability to do the same.  Regardless of language, anyone can write code that is fragile or breaks.

--

[Isaac Schlueter]

On Sat, Jan 21, 2012 at 10:45, C. Mundi <cmu...@gmail.com> wrote:
> 0. Callbacks are no more "magical" in node than in any language with
> first-class functions.

True.

> 1. JavaScript/V8 is single-threaded and never gets prempted.

True, mostly. Your JavaScript program never has to worry about
*another* javascript program tampering with the state of its
JavaScript objects.

However, the contents of a Buffer are stored outside the v8 heap. So,
a function on the thread pool might modify it. The rule of thumb is,
once you pass your buffer to another function, it's not your buffer
any more, so don't touch it.

Also, Isolates introduces the ability for more than a single v8
environment to be running in a single *process*, but they're still
each in a completely separate heap space from one another, so the
semantics of the program don't change much. (I mean, process.pid will
be the same, and process.tid will tell you which thread you're on, but
otherwise, it looks just like a child process.)

> 2. process.nextTick simply defers/enques a function to the next cycle of the
> (one and only) event loop.

True.

> 3. All of the above is 100% synchronous and non-premptive, but...

False. process.nextTick is not synchronous. It happens at the start
of the next tick, not the end of the current one. Most of the time,
that's the same thing, but it's not *guaranteed* to run before
anything else.

> 4. Node provides JavaScript hooks to access a shared thread pool in the node
> process.  These hooks are used by the authors of slow io libraries, for
> example to open files or access disk-based databases.

True.

> 5. If I need a function to execute asynchronously or ...more precisely... be
> nonblocking, then I need to make use of that shared thread pool, either by
> piggybacking on an existing node function which is nonblocking or by
> learning to use the hooks myself.

True. Or you can use setTimeout, or process.nextTick, but that won't
run your javascript in parallel, it'll run it *later*. The only way
to run JavaScript in parallel with other JavaScript is as a child
process or Isolate.

> Question: Are the hooks exposed in JavaScript, or do I need to write native
> C code?

You have to write native C code.

> The application I have in mind to write does compute-intensive tasks, which
> is the CPU analog of slow disk.

Use child_process.fork, and communicate with the child via IPC.

// parent.js
var child_process = require("child_process")
var path = require("path")
var child = child_process.fork(path.resolve(__dirname, "child.js"))
child.on("message", function (x) {
if (x === "ping") child.send("pong")
else console.log(x)
})

// child.js
process.on("message", function (x) {
console.log("received: "+x)
process.exit(0)
})
console.log("child starting work")
// some cpu intensive stuff...
for (var i = 0; i < 1e10; i ++) ;
console.log("pinging parent")
process.send("ping")

[Isaac Schlueter]

On Sat, Jan 21, 2012 at 11:50, Jorge <jo...@jorgechamorro.com> wrote:
> Here you're delaying the setting of the listeners/event handlers to the nextTick. Why ? This has nothing to do with .emit() being synchronous.

Jorge, what would it mean for emit to be "asynchronous"? Can you
explain that? I feel like we're passing by one another, and probably
making more confusion in the process.

This is what I mean when I say that emit is synchronous:

var x = 5
foo.on("bar", function () { x = 10 })
foo.emit("bar")
console.log(x) // should be 10

You can still have a synchronous action is scheduled for some later
time, because it's called by something asynchronous.

var x = 5
foo.on("bar", function () { x = 10 })
process.nextTick(function () { foo.emit("bar") }) // or fs or net or whatever
console.log(x) // emit hasn't happened yet. Still 5.
setTimeout(function () { console.log(x) }, 100) // will be 10 later

[Isaac Schlueter]

On Sat, Jan 21, 2012 at 13:13, Mark Hahn <ma...@hahnca.com> wrote:
>>   which after all is really just syntactic sugar for wrapping the async
>> call
>
> I couldn't parse that sentence but it is not just sugar.

It's sugar. (Not syntactic sugar, it's API sugar, but I'm hair-splitting.)

var request = createRequest(options, function (response) {
request.whatever(...)
})

// is sugar for:

var request = createRequest(options)
request.on("response", function (response) {
request.whatever(...)
})

Again, none of this is a problem if you make sure that every callback
is guaranteed to be either sync always, or async always.

[C. Mundi]

Thanks Isaac.  That's exactly what I meant by sugar.

[Mark Hahn]

I never knew that callbacks were implemented as events.  You learn something new every day.

 

[Isaac Schlueter]

Mark,

Callbacks that get an error as the first argument are not events, usually.

I guess you could also say it's sugar for:

createServer(options).on("request", function (request) { ... })

so we just remove the `).on("request"` bit by extending the API
slightly. I guess it does increase size and make for some confusion,
but it's probably too costly to change now.

On Sat, Jan 21, 2012 at 13:48, Mark Hahn <ma...@hahnca.com> wrote:
> I never knew that callbacks were implemented as events.  You learn something
> new every day.
>
>

[Mark Hahn]

I'm not proposing changes.  I'm just expressing my opinion and suggesting that new code pay attention to this.

[Jorge]

On 21/01/2012, at 22:29, Isaac Schlueter wrote:
> On Sat, Jan 21, 2012 at 11:50, Jorge <jo...@jorgechamorro.com> wrote:
>> Here you're delaying the setting of the listeners/event handlers to the nextTick. Why ? This has nothing to do with .emit() being synchronous.
>
> Jorge, what would it mean for emit to be "asynchronous"? Can you
> explain that? I feel like we're passing by one another, and probably
> making more confusion in the process.

Events happen at any time (asynchronously). A timer, a packet that arrived, a DOM mutation, a click, a keypress, etc.

Events have event listeners that run in response to events.

But the event listeners should not interrupt your code: they should only run when the program flow returns to the event loop.

An async .emit('event') should simply create an event that will be dispatched to its listener(s) as soon as possible, when the program flow returns to the event loop.

A sync .emit('event') goes a step further, and calls the listener(s) instead right on the spot, immediately, interrupting your program flow.

DOM mutation events are synchronous (1)
Async XHRs are botched eventemitters, because the first onreadystatechange event is synchronous but the rest up to readystate === 4 are asynchronous. (2)
DOM Mouse, keyboard, etc events are always asynchronous: they'll never interrupt your code.

(1)
(function () {
var i= 0;
document.body.addEventListener("DOMNodeInserted", function listener () { i++ }, false);
console.log(i);
document.body.appendChild(document.createElement('p'));
document.body.appendChild(document.createElement('p'));
document.body.appendChild(document.createElement('p'));
console.log(i);
})();

Outputs -> 0, 3

(2)
(function () {
var i= 0;
var xhr= new XMLHttpRequest;
xhr.onreadystatechange= function listener () { i++ };
xhr.open('GET', 'nonexistent', true);
xhr.send(null);
console.log(i);
})();

Outputs -> 1

[Scott González]

On Sun, Jan 22, 2012 at 12:11 PM, Jorge <jo...@jorgechamorro.com> wrote:

DOM mutation events are synchronous (1)
Async XHRs are botched eventemitters, because the first onreadystatechange event is synchronous but the rest up to readystate === 4 are asynchronous. (2)
DOM Mouse, keyboard, etc events are always asynchronous: they'll never interrupt your code.

No. All events are synchronous.

XHRs are not botched. The first event occurs immediately because the state changes immediately. The rest occur in the future, when the state has actually changed. The state changes asynchronously, but the events associated with the state change occur synchronously at the time the state changes.

DOM Mouse, keyboard, etc. events are always synchronous. They never interrupt your code because the actions a user takes don't occur while your code is running. You can create and fire a native event to verify this if you want.

[Bruno Jouhier]

"Synchronously/asynchronously" by themselves don't mean much.
"Synchronously/asynchronously with respect to ..." are more
meaningful.

Events are synchronous with respect to the state change that triggers
them.

Events are generally asynchronous with respect to the code that
creates their source. But there are cases (DOM mutations and first XHR
event examples given by Jorge) where they are synchronous with respect
to their source creation. Looks a bit odd to me but not enough to make
a big fuss about it.

[Jorge]

On 24/01/2012, at 00:10, Scott González wrote:

On Sun, Jan 22, 2012 at 12:11 PM, Jorge <jo...@jorgechamorro.com> wrote:

DOM mutation events are synchronous (1)
Async XHRs are botched eventemitters, because the first onreadystatechange event is synchronous but the rest up to readystate === 4 are asynchronous. (2)
DOM Mouse, keyboard, etc events are always asynchronous: they'll never interrupt your code.

No. All events are synchronous.

XHRs are not botched. The first event occurs immediately because the state changes immediately. The rest occur in the future, when the state has actually changed. The state changes asynchronously, but the events associated with the state change occur synchronously at the time the state changes.

If you were right, this would not block:

(function () {

  var i= 0;

  var xhr= new XMLHttpRequest;

  xhr.onreadystatechange= function listener () { i++ };

  xhr.open('GET', 'nonexistent', true);

  xhr.send(null);

  while (i === 1) ;

})();

DOM Mouse, keyboard, etc. events are always synchronous. They never interrupt your code because the actions a user takes don't occur while your code is running.

Nor this:

(function () {

  var i= 0;

  window.onclick= window.onkeyup= function listener () { i++ };

  while (i === 0) ;

})();

<snip>

-- 

Jorge.

[Scott González]

I completely fail to see your reasoning for either of the cases below. Neither code block below introduces any new arguments or reasoning to this conversation. They're merely slight changes to previous blocks of code, with the addition of known infinitely blocking code blocks.

--

[Jorge]

On 26/01/2012, at 01:43, Scott González wrote:

> I completely fail to see your reasoning for either of the cases below. Neither code block below introduces any new arguments or reasoning to this conversation. They're merely slight changes to previous blocks of code, with the addition of known infinitely blocking code blocks.

You said:

>> DOM Mouse, keyboard, etc. events are always synchronous. They never interrupt your code because the actions a user takes don't occur while your code is running.

If you were right, this code:

> (function () {
> var i= 0;
> window.onclick= window.onkeyup= function listener () { i++ };
> while (i === 0) ;
> })();

would only block until a click or a keyup, but it blocks forever, because mouse and keyboard events never interrupt your code.

You said:

>> XHRs are not botched. The first event occurs immediately because the state changes immediately. The rest occur in the future, when the state has actually changed. The state changes asynchronously, but the events associated with the state change occur synchronously at the time the state changes.

If that were true, this:

> (function () {
> var i= 0;

> var xhr= new XMLHttpRequest;
> xhr.onreadystatechange= function listener () { i++ };
> xhr.open('GET', 'nonexistent', true);
> xhr.send(null);
> while (i === 1) ;
> })();

would only block until "the future, when the state has actually changed", but it blocks forever, because *only* the first readystatechange event is synchronous (interrupts your code).
--
Jorge.

[Scott González]

I said "they never interrupt your code". What is so confusing about this? JavaScript is single threaded. Events are always synchronous. Your examples are the equivalent of:

process.nextTick(function() {
    console.log( "what is going on?" );
});
while (true) {}

--
Jorge.

[Bruno Jouhier]

> >> DOM Mouse, keyboard, etc. events are always synchronous. They never interrupt your code because the actions a user takes don't occur while your code is running.

No. The user's actions *happen* at any time. I'm free to click on my
mouse at any time. I'm not constrained in any way by what the program
is doing.

The user's actions are *emitted as events* directly (and
synchronously) from the event loop. So they never interrupt a function
which is running JS user code.

Saying that mouse events are synchronous seems a bit far-fetched. From
a human observer's standpoint, the mouse actions are truly
asynchronous (the user is free to decide when he clicks). From the
event loop's standpoint, the dispatching of events is a synchronous
operation.

My 2 cents.

[Scott González]

On Fri, Jan 27, 2012 at 8:59 AM, Bruno Jouhier <bjou...@gmail.com> wrote:

Saying that mouse events are synchronous seems a bit far-fetched. From
a human observer's standpoint, the mouse actions are truly
asynchronous (the user is free to decide when he clicks). From the
event loop's standpoint, the dispatching of events is a synchronous
operation.

This is clearly a very technical discussion. I think we can agree that mouse event means the actual event, e.g., a click event, which is different than the user's action, e.g., a click of a mouse button. I feel like you and I are actually in complete agreement. This distinction is the entire purpose of the majority of this thread.

[Jorge]

On 27/01/2012, at 13:44, Scott González wrote:

> I said "they never interrupt your code". What is so confusing about this?

You said more than that:

"They never interrupt your code because the actions a user takes don't occur while your code is running"

Then I sent you a code snippet in which the user actions *do* occur while your code is running, to demonstrate that the reason why these listeners never interrupt your code is *not* that the events "don't occur while your code is running".

DOM mutation events happen while your code is running, and their listeners interrupt your code.

DOM Mouse, keyboard, etc events (may) happen (as well) while your code is running, but their listeners won't ever interrupt your code

> <snip>

--
Jorge.

[Scott González]

On Mon, Jan 30, 2012 at 5:05 AM, Jorge <jo...@jorgechamorro.com> wrote:

On 27/01/2012, at 13:44, Scott González wrote:

> I said "they never interrupt your code". What is so confusing about this?

You said more than that:

"They never interrupt your code because the actions a user takes don't occur while your code is running"

Yes, I said that. It's a fundamental rule of JavaScript. Your code is never interrupted.
 

Then I sent you a code snippet in which the user actions *do* occur while your code is running, to demonstrate that the reason why these listeners never interrupt your code is *not* that the events "don't occur while your code is running".

They occur while your code is running, because you caused them to occur. The user did not perform an action that interrupted your code.
 

DOM mutation events happen while your code is running, and their listeners interrupt your code.

If *your code* modifies the DOM, then yes, the events will fire in your event loop. Those events are not interrupting your code, they're merely firing from inside your code. Your argument is essentially the same as saying "When I use .forEach() on an array, my code is interrupted because the function is executed in the same event loop." That's nonsense.

DOM Mouse, keyboard, etc events (may) happen (as well) while your code is running, but their listeners won't ever interrupt your code

They may fire in the same event loop. For example, if you invoke the click() method on a DOM element. But a *user action* will never interrupt your code. User actions don't occur while your code is running, just like node's nextTick or setTimeout callbacks will never occur while your code is running.

[Ilya Shaisultanov]

Pardon for resurrecting such old thread but I have a question about process.nextTick: why/how does it eliminate the current stack? What happens behind the scenes?

On Tuesday, January 17, 2012 5:29:13 PM UTC-5, JoeZ99 wrote:

I don't really know how node knows when to run a function asynchronously or not.

Let's say you have this piece of code.

function gen_url(str) {
  return 'http://' + str;
}

full_url = gen_url('www.google.com');

In that case I'm obviously counting on the function gen_url being executed synchronously.

Now If I have

function hit_mysql(query, callback) {
  var mystuff = oldstuff * system_variable;
  client = require('mysql').Client();
  client.query(query, callback);
}

var one = 'one';
hit_mysql('select ...', function(){console.log('done');});
var two = 'two';


I would expect hit_mysql to run asynchronousy.


How does node figure this out? I'm no "marking" any function as "sync" or "async". Does node througfuly examine the function's body to see if somewhere in the function body (or in the body of the functions defined in the function body) there is some well recognized native node async call, like setTimeout, or http.listen(), or whatever? 
--
uh, oh.

[Rick Waldron]

On Fri, Nov 8, 2013 at 3:57 PM, Ilya Shaisultanov <ilya.sha...@gmail.com> wrote:

Pardon for resurrecting such old thread but I have a question about process.nextTick: why/how does it eliminate the current stack? What happens behind the scenes?

It doesn't eliminate the current stack. The callback is "scheduled" and subsequently executed in the next execution turn.

Rick

[Ilya Shaisultanov]

Ok, I think I'm thinking about process.nextTick as it was in node <0.10, where, if I'm not mistaken

function f() {

  process.nextTick(f)

}

f()

would loop infinitely, whereas an immediate recursive call throws RangeError.

I tried running that in v0.10 and got RangeError. However,

function f() {

  setImmediate(f)

}

f()

puts the function in "background" and makes node process consume 100% CPU. What happens there?

[Rick Waldron]

On Fri, Nov 8, 2013 at 5:29 PM, Ilya Shaisultanov <ilya.sha...@gmail.com> wrote:

Ok, I think I'm thinking about process.nextTick as it was in node <0.10, where, if I'm not mistaken

function f() {

  process.nextTick(f)

}

f()

would loop infinitely, whereas an immediate recursive call throws RangeError.

I tried running that in v0.10 and got RangeError. However,

function f() {

  setImmediate(f)

}

f()

puts the function in "background" and makes node process consume 100% CPU. What happens there?

I believe this is a bug that's specific to  0.10.x and was fixed in 0.11.x and up. (Sorry, I don't have free cycles to look up the relevant github issues at the moment)

Rick

On Friday, November 8, 2013 5:20:21 PM UTC-5, Rick Waldron wrote:

On Fri, Nov 8, 2013 at 3:57 PM, Ilya Shaisultanov <ilya.sha...@gmail.com> wrote:

Pardon for resurrecting such old thread but I have a question about process.nextTick: why/how does it eliminate the current stack? What happens behind the scenes?

It doesn't eliminate the current stack. The callback is "scheduled" and subsequently executed in the next execution turn.

Rick

--

--
Job Board: http://jobs.nodejs.org/
Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nod...@googlegroups.com
To unsubscribe from this group, send email to
nodejs+un...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en
 

---

You received this message because you are subscribed to the Google Groups "nodejs" group.

To unsubscribe from this group and stop receiving emails from it, send an email to nodejs+un...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Ilya Shaisultanov]

No worries.

Is the bug that it doesn't throw RangeError or that the execution happens in background?

[Rick Waldron]

https://github.com/joyent/node/issues/5798

[Ilya Shaisultanov]

Interesting discussion, thank you.

I'm still unclear, though, what exactly makes infinite recursion work when recursive call is wrapped in setImmediate. What happens to the stack?