Replace Promises with Deferreds
Felix Geisendörfer
why do today what you can do tomorrow?
Currently node.js implements a simple & elegant system for dealing
with non-blocking operations, the Promise API. You request an
operation to be performed, and you get a promise object which allows
you to track the fulfillment of this operation by specifying one or
more callbacks / error handlers.
As neat as this concept is, anybody familiar with it will have noticed
its shortcomings:
Promises are hard to build upon. If you want to provide a high-level
wrapper for some low-level Promise-based API, right now you are stuck
with creating a second promise and delegating / processing the
original promise events by hand.
Exception handling is manual. Promises may fail in unexpected ways
causing the main event loop to crash and node to exit. You can defend
against this by manually wrapping all of your logic in try..catch
blocks, but you are out of luck if the issue exists in a 3rd party
library. The last line of defense, process.addListener
('uncaughtException'), is suitable to keep your process from crashing,
but any kind of failure recovery logic is rather hard to implement on
top of it.
Promises are hard to chain. While we might be able to do everything in
parallel with node, there are tons of use cases that require certain
operations to be performed in a certain sequence. Using the current
promise API, this will usually lead to very deep nesting and high
cyclomatic complexity.
Given the critical role that Promises have in the current node.js API,
it seems only fair to make them capable enough to deal with most of
the issues that first time users as well as power users will end up
struggling with. Luckily, there is some great prior API in this field,
namely the Deferred API as, AFAIK, first seen in Twisted. Most of my
knowledge about it however, is drawn from the Deferred implementation
in Dojo.
So, I have spend some time porting Dojo Deferreds to node.js, and I
think they would be an excellent replacement for the current Promise
API. For those not familar with Deferreds, I highly recommend the
documentation of the dojo implementation as a first introduction:
http://api.dojotoolkit.org/jsdoc/1.2/dojo.Deferred
Basically, they are just like Promises with a few differences:
Callback return values are important. The value returned from a
callback is the value passed to the next callback. This makes it easy
to build upon Deferreds by adding callbacks that manipulate the
initial results.
Exceptions are automatically handled. I actually propose a slight API
change here, but let me explain. The default behavior of Deferreds is
that any callback invocation is wrapped in a try..catch blok. If you
do something stupid in your callback, the Deferred automatically
switched its state to "error" and continues by executing all "errback"
callbacks. What I like about this, is that you can rest assured that
3rd party Deferreds may never crash your application, but rather
invoke a "errback" callback instead. However, I think this behavior
should be disabled by default. My initial experience has been that it
is too easy to oversee exceptions that are being thrown, which is very
bad during development. Therefor my current port of dojo Deferreds
diables this behavior by default. If you want to use it, you just
call .trycatch() on any Deferred you receive to enable it
transparently.
Last but not least, Deferreds are very easy to chain in sequence. Any
callback may return a new Deferred itself, which will cause any
remaining callbacks to be put on hold until that Deferred has
completed. This makes it trivial to express sequences of asynchronous
operations.
My current port of (dojo) Deferreds can be seen here:
http://github.com/felixge/node-deferred/
At this point it is essentially identical to dojo Deferreds, except
that I removed all dependencies to dojo and made the mentioned change
to the try..catch behavior.
I herby would like to ask all of you to comment on replacing node.js
Promises with Deferreds. It certainly would make working with async
operations easier, and backwards compatibility is pretty good for most
parts.
If nobody sees major issues, I would volunteer to provide the
necessary patch for node.js.
-- Felix Geisendörfer aka the_undefined
PS: I am also working on syntax sugar and concepts that make it easier
to aggregate the results from multiple deferreds running in parallel
or sequence, but those are not quite ready yet and would only
complicate the discussion.
Dean Landolt
Tim Caswell
Benjamin Thomas
> This all sounds very similar to narwhal's Promise API (which use a dojo-like
> Deffered under the hood) -- it may be worth having a look.
http://wiki.commonjs.org/wiki/Promises
And then recently on the Narwhal and Jack mailing list there is a
discussion of Node and its relation to Narwhal:
http://groups.google.com/group/narwhaljs/msg/aa9f29ac9118ad00
There they suggest that in order to bring Node's Promises inline with
the Promises from the proposal,
Basically to extend what already exists in Node, I think you would do:
Promise.prototype.then = function(successCallback, errorCallback){
var promise = new process.Promise();
this.addCallback(function(){
successCallback.call(this, arguments);
promise.emitSuccess.call(promise, arguments);
});
this.addErrback(function(){
errorCallback.call(this, arguments);
promise.emitError.call(promise, arguments);
});
return promise;
}
So, I wonder if instead of implementing a new API we should go this route...
For what it's worth I like Twisted's and Dojo's Deferred API better
than the Promise proposal. "addCallback(function)" is much more
intuitive and easy to grasp than "then(function, function)"
I don't know Node's position on how compliant it wants to be with the
CommonJS "standardizations".
Benjamin Thomas
Felix and I have already discussed this via e-mail, but - though I
think that this is a great addition to Node - here is my concern about
his implementation:
Felix said,
> 3rd party Deferreds may never crash your application, but rather
> invoke a "errback" callback instead. However, I think this behavior
> should be disabled by default. My initial experience has been that it
> is too easy to oversee exceptions that are being thrown, which is very
> bad during development. Therefor my current port of dojo Deferreds
> diables this behavior by default. If you want to use it, you just
> call .trycatch() on any Deferred you receive to enable it
> transparently.
about handling this error problem. I agree that errors shouldn't be
getting swallowed up but it seems like making it so that the errback
chain only gets called for errors that you return and not those that are
thrown makes the title 'errback' inaccurate. The errback function should
get any error that is raised/returned/thrown, not some of them. I don't
like the idea of having to opt-in to more semantic (and thorough)
functionallity.
I think this can be addressed in a different way. Right now, the way
Dojo's Deferreds work is that they wrap all callbacks in a try/catch
block so that they can catch the error and pass it to the errback
functions. however if the last errback function ends in an error then
Dojo does nothing. This is why errors get "overseen". What a deferred
implementation should do is throw the error again if there isn't an
errback function to handle it.
Now all an errback function would have to do is look at the error that is passed
to it and make sure it is an error it knows how to handle, and if it isn't
just throw it again or return it and the Deferred will pass it along.
What do people think?
I've implemented this idea (plus some other small changes) in a fork
of Felix's here: http://github.com/bentomas/node-deferred
Felix Geisendörfer
> I don't know Node's position on how compliant it wants to be with the
> CommonJS "standardizations".
solutions that solve our problems. If a specification out there does
that, fine - let's not reinvent the wheel. CommonJS is very awesome,
but I think they are more of the idealists who want to make everything
really pretty and perfect while node operates on the "Make it work,
then make it worker better" mode. In the long run, I hope node will
join back together with CommonJS on as mainy topics as possible, but
at this point there is just more interest in action than there is
discussion (I can only speak for myself and my observations here).
> What a deferred implementation should do is throw the error again if there isn't an errback function to handle it.
callback or errback returned/raised an Error and there is no more
errback handler to deal with it?
I like that. The only shortcoming I see is that callbacks and errbacks
can be attached at any point, even if the Deferred callback chain has
already reached its end. This is interesting for API's that implement
caching and thus may "fire" their Deferred immediately, that means
before it is even being returned to the caller of the function. So far
we had to use a setTimeout() hack to make sure that the receiver of an
immediately firing Promise is able to attach callbacks, but with
Deferreds we would not have to do this.
This behavior however will not be very useful if the exception can
occur before the first errback listener can be attached. You would
essentially rob yourself of any means to catch it.
Unfortunately, my trycatch() approach suffers from the same logical
flaw. So I guess we are back to implementing Deferreds exactly like
dojo, or accepting the setTimeout() hack and live with it.
New ideas would be very welcome!
-- Felix Geisendörfer aka the_undefined
Rakesh Pai
--
You received this message because you are subscribed to the Google Groups "nodejs" group.
To post to this group, send email to nod...@googlegroups.com.
To unsubscribe from this group, send email to nodejs+un...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/nodejs?hl=en.
--
Rakesh Pai
Mumbai, India.
http://piecesofrakesh.blogspot.com/
http://twitter.com/rakesh314
http://www.google.com/profiles/rakeshpai
Felix Geisendörfer
> the ability to attach handlers to the deferred even after the event has occurred
explanation is better. It's an awesome feature.
> If you've cloned dojo's deferreds, you must have already got this capability
in your implementation
is still under discussions - what are your thoughts on it?
-- Felix Geisendörfer aka the_undefined
> > .
Dean Landolt
Rakesh,
Yip, I tried explaining this in my previous message but your
> the ability to attach handlers to the deferred even after the event has occurred
explanation is better. It's an awesome feature.
This is one of the things I meant when I suggested investigated narwhal's Promise implementation -- it does this as well.
Ryan Dahl
> namely the Deferred API as, AFAIK, first seen in Twisted. Most of my
> knowledge about it however, is drawn from the Deferred implementation
> in Dojo.
>
> So, I have spend some time porting Dojo Deferreds to node.js, and I
> think they would be an excellent replacement for the current Promise
> API.
to be worked out before bringing this into node:
- In at least one function, posix.read(), the success callback has
multiple parameters. Is this allowed in Deferreds, if so how are they
returned from the callback? If not, how would you change the success
parameters for posix.read()?
- What happens if one does not return a value from the callback? That
is, if the callback returns undefined? It seems like the default
behavior should either be
1. error out with a helpful message or
2. assume the user wants to return the same result that they received.
Which do you think is better?
- EventEmitters also have chains of callbacks. For a consistent
interface, if we change Promises to Deferreds, so that callbacks
should each return the result, then I think EventEmitters should act
similarly. Ideally Deferreds should be instances of EventEmitters.
- Regarding catching errors, what if we go with the dojo.Deferred
behavior, in that if a success callback has an error, then the state
is changed to error, and the error callbacks chain is then issued -
but with a twist - if there are no error handlers then the exception
is allowed to propagate down to the event loop (invoking the
uncaughtException event, if any).
Benjamin Thomas
> multiple parameters. Is this allowed in Deferreds, if so how are they
> returned from the callback? If not, how would you change the success
> parameters for posix.read()?
{
data: "..."
bytes_read: 3
> - What happens if one does not return a value from the callback? That
> is, if the callback returns undefined? It seems like the default
> behavior should either be
> 1. error out with a helpful message or
> 2. assume the user wants to return the same result that they received.
> Which do you think is better?
you to have to return the same value seems like a pain.
> - EventEmitters also have chains of callbacks. For a consistent
> interface, if we change Promises to Deferreds, so that callbacks
> should each return the result, then I think EventEmitters should act
> similarly. Ideally Deferreds should be instances of EventEmitters.
but would you add an errback handler for a specific event?
emitter.addListener("event1", listener);
emitter.addErrback("event1", listener);
emitter.addListener("event2", listener);
emitter.addErrback("event2", listener);
Or would you add an errback handler for all events?
emitter.addListener("event1", listener);
emitter.addListener("event2", listener);
emitter.addErrback(function(eventName, err) { //handle error });
But then what would happen if the errback function cancels the error?
Would you go back to processing the same event? I think I prefer
adding errback handlers to specific events...
Rakesh Pai
multiple parameters. Is this allowed in Deferreds, if so how are they
returned from the callback? If not, how would you change the success
parameters for posix.read()?
--
You received this message because you are subscribed to the Google Groups "nodejs" group.
To post to this group, send email to nod...@googlegroups.com.
To unsubscribe from this group, send email to nodejs+un...@googlegroups.com.
--
Rakesh Pai
Isaac Z. Schlueter
> 2009/12/12 Felix Geisendörfer <fe...@debuggable.com>:
>
> > Luckily, there is some great prior API in this field,
> > namely the Deferred API as, AFAIK, first seen in Twisted. Most of my
> > knowledge about it however, is drawn from the Deferred implementation
> > in Dojo.
>
> > So, I have spend some time porting Dojo Deferreds to node.js, and I
> > think they would be an excellent replacement for the current Promise
> > API.
these two use cases I've run into:
1. I need to do these 15 things in order, and just want to know when
the list is completed
2. I need to do these 15 things in no particular order, and want to
know when they're all done
It seems that something like Deferreds can make these two generic
cases easier to handle. However, I'm not convinced that both can't be
solved with a generic utility built on top of the Promise API.
Since pretty much 100% of nodejs code out there today makes heavy use
of Promises, I'd be wary about "replacing" them, at least on ry's
master, any time soon.
--i
Connor Dunn
> Since pretty much 100% of nodejs code out there today makes heavy use
> of Promises, I'd be wary about "replacing" them, at least on ry's
> master, any time soon.
>
do it now before the API is meant to be frozen, rather than later.
Brian Hammond
I was in the same boat so I created PromiseChain and PromiseGroup.
I am just on my phone ATM but I think you can find them under my gists
on github (fictorial).
Felix Geisendörfer
sorry I couldn't answer earlier. Two things were brought up here that
deserve a discussion of their own.
a) Managing multiple Promises/Deferreds that run in parallel or
sequence
-> I have started think about concepts for this myself already, but
nothing ready to present yet. I do however believe that anything we
might want to do here could be done regardless of the actual Deferred/
Promise implementation we decide on, so I suggest picking up this
topic again once Promise vs. Deferreds has been battled out.
b) Clarifications on when to use Promise, EventEmitter or pure
callbacks.
-> This is a discussion we should probably have rather sooner than
later as it may lead to a bigger change in node's current API than
Deferreds would. One idea that came up in IRC is that node should
provide the most low-level functionality via plain callbacks, and most
stuff should get a high-level wrapper based on Promises/EventEmitters.
I would love if we could come up with something like a "Node Interface
Guideline" that applies to all APIs used within node and promoted to
developers to use in their projects as well.
So anybody who feels passionate about either of those two topics
should probably go ahead and start a separate thread for them.
Meanwhile back to the issues pointed out with the current plan of
replacing Promises with Deferreds.
> Ideally Deferreds should be instances of EventEmitters
will get fired if the previous listener returned an error or raised an
exception.
Finally, I think I have the solution for the excerption handling. We
simply make the setTimeout hack part of Deferreds themselves. So when
you call myDeferred.callback() (aka Promise.emitSucces()), it will do
a setTimeout(firstCallback, 0) internally. This will make sure that
its always possible to attach an error handler to a Deferred, even if
it fires before it is being returned from the function that creates
it.
This way we can use Benjamins proposal of re-throwing any exceptions
occurring in callbacks, if there are no more errback handlers
registered. This will solve the problem of having hard to debug &
silent failures, while giving great flexibility and power for dealing
with exceptions.
So unless there are any objections I'd like to work on a patch
replacing Promises with Deferreds. I propose to keep the current API /
naming where possible. Once this patch is ready, I'd like to do
another patch for EventEmitters as suggested for Ryan. I don't want to
try doing both of those refactorings at the same time, that would just
become messy. I will also test the patch against my node project
(transload.it), so I should be able to provide some feedback on how
much stuff broke and how long it took me to fix it : ).
-- Felix Geisendörfer
> Since pretty much 100% of nodejs code out there today makes heavy use
> of Promises, I'd be wary about "replacing" them, at least on ry's
> master, any time soon.
but we certainly should be brave enough to do some changes at this
point. (I certainly feel the pain, trust me, our node codebase is
fairly big).
Rakesh Pai
nothing ready to present yet. I do however believe that anything we
might want to do here could be done regardless of the actual Deferred/
Promise implementation we decide on, so I suggest picking up this
topic again once Promise vs. Deferreds has been battled out.
later as it may lead to a bigger change in node's current API than
Deferreds would.
simply make the setTimeout hack part of Deferreds themselves. So when
you call myDeferred.callback() (aka Promise.emitSucces()), it will do
a setTimeout(firstCallback, 0) internally. This will make sure that
its always possible to attach an error handler to a Deferred, even if
it fires before it is being returned from the function that creates
it.
--
You received this message because you are subscribed to the Google Groups "nodejs" group.
To post to this group, send email to nod...@googlegroups.com.
To unsubscribe from this group, send email to nodejs+un...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/nodejs?hl=en.
Felix Geisendörfer
> chain immediately, or forget trapping events. So for example, if you want to
> decide if you want to throw an error based on the outcome two different
> async events, you are stoked. I'd rather that this be cleaner. As you
> mentioned yourself, this is a hack, and only works when you have one async
> thing you are doing, and all event handlers are attached without being in
> setTimeouts themselves. You never can assume that, and you are in trouble if
> you have to explain that in API docs.
returning to the event loop again. If you don't want to decide whether
to handle errors at this point, you can always just attach and errback
and returned a 2nd Deferred in that. Once all of your pre-conditions
are met to decide on handling the exception or not, you fire off the
2nd Deferred. If that Deferred returns / triggers an exception and
there is no further errback registered to the original event yet, an
exception is raised. But you have any amount of time you need to
decide on that, as long as you don't fire off the 2nd Deferred. Also:
I am only suggesting to add setTimeout() to the emitSuccess/emitError
functions, so only the first callback is affected.
Anyway, what about making the setTimeout hack an optional parameter
when creating the Deferred? Something like "{delay: true}" or "{queue:
true}". This would allow us to hide the setTimeout hack from any
function that uses Deferreds. But unless somebody has a new idea,
setTimeout just has to come into play somewhere (for Deferreds that
may fire immediately).
-- Felix Geisendörfer aka the_undefined
> > .
Felix Geisendörfer
> this point, you can always just attach and errback and
> returned a 2nd Deferred in that
s/returned/return/
Siegmund Führinger
2009/12/13 Felix Geisendörfer <fe...@debuggable.com>:
>
> sorry I couldn't answer earlier. Two things were brought up here that
> deserve a discussion of their own.
>
> a) Managing multiple Promises/Deferreds that run in parallel or
> sequence
>
> -> I have started think about concepts for this myself already, but
> nothing ready to present yet. I do however believe that anything we
> might want to do here could be done regardless of the actual Deferred/
> Promise implementation we decide on, so I suggest picking up this
> topic again once Promise vs. Deferreds has been battled out.
all of them have fired, than DeferredList is the tool for the job.
btw. thank you Felix for driving this Deferred issue forward! a few
weeks ago i brought this topic up and intended to port the API over to
node's promise implementation, but so far i didn't have the time yet
:)
i do use twisted a lot for server side stuff and would like to switch
to node.js at some point and this would make it a lot easier!
cheers,
sifu
Felix Geisendörfer
> all of them have fired, than DeferredList is the tool for the job.
Anyway, different discussion : ).
Still waiting for feedback on the exception handling. For those new to
the thread the quick summary again:
a) Should we re-throw exceptions if there are no further errback
handlers to deal with them (rather than silently ignoring them like
dojo)?
b) Should we provide a setTimeout(function(){myDeferred.callback()},
0) wrapper via the Deferred constructor, like {delay: true}.
-- Felix Geisendörfer aka the_undefined
Rakesh Pai
--
You received this message because you are subscribed to the Google Groups "nodejs" group.
To post to this group, send email to nod...@googlegroups.com.
To unsubscribe from this group, send email to nodejs+un...@googlegroups.com.
--
Rakesh Pai
Iván Montes
desirable to re-throw them, perhaps what could be done is to just log
them to stderr instead and emit a global event so that the developer can
manage them at the application level.
I like though the use of a "delay" option. I would pass it in to the
callback/errback method instead of the constructor. Even special methods
could be added (delayCallback/delayErrback) which accept an optional
second argument indicating the number of msecs for the timeout. This way
it can also be useful for other use cases besides giving the chance to
attach the callbacks.
/imv
On 12/14/09 10:38 AM, Felix Geisend�rfer wrote:
>> if by "managing multiple deferreds" is meant, to get a callback after
>> all of them have fired, than DeferredList is the tool for the job.
>>
> I looked at it, but I'm not convinced from an API point of view.
> Anyway, different discussion : ).
>
> Still waiting for feedback on the exception handling. For those new to
> the thread the quick summary again:
>
> a) Should we re-throw exceptions if there are no further errback
> handlers to deal with them (rather than silently ignoring them like
> dojo)?
> b) Should we provide a setTimeout(function(){myDeferred.callback()},
> 0) wrapper via the Deferred constructor, like {delay: true}.
>
>> 2009/12/13 Felix Geisend�rfer<fe...@debuggable.com>:
>>
>>> Hey guys,
>>>
>>
>>> sorry I couldn't answer earlier. Two things were brought up here that
>>> deserve a discussion of their own.
>>>
>>
>>> a) Managing multiple Promises/Deferreds that run in parallel or
>>> sequence
>>>
>>
>>> -> I have started think about concepts for this myself already, but
>>> nothing ready to present yet. I do however believe that anything we
>>> might want to do here could be done regardless of the actual Deferred/
>>> Promise implementation we decide on, so I suggest picking up this
>>> topic again once Promise vs. Deferreds has been battled out.
>>>
>> if by "managing multiple deferreds" is meant, to get a callback after
>> all of them have fired, than DeferredList is the tool for the job.
>>
>> btw. thank you Felix for driving this Deferred issue forward! a few
>> weeks ago i brought this topic up and intended to port the API over to
>> node's promise implementation, but so far i didn't have the time yet
>> :)
>> i do use twisted a lot for server side stuff and would like to switch
>> to node.js at some point and this would make it a lot easier!
>>
>> cheers,
>> sifu
>>
Ryan Dahl
>> all of them have fired, than DeferredList is the tool for the job.
>
> I looked at it, but I'm not convinced from an API point of view.
> Anyway, different discussion : ).
>
> Still waiting for feedback on the exception handling. For those new to
> the thread the quick summary again:
>
> a) Should we re-throw exceptions if there are no further errback
> handlers to deal with them (rather than silently ignoring them like
> dojo)?
> b) Should we provide a setTimeout(function(){myDeferred.callback()},
> 0) wrapper via the Deferred constructor, like {delay: true}.
>
To be clear we're talking about two types of exceptions:
A) When the promise action normally errors out. E.G.
posix.stat("/does_not_exist")
B) When a callback or errback attached to a promise itself throws an exception.
Type-A exceptions should be sent to errbacks. If no errbacks are
attached, then it should be thrown too - because this is a programming
error. Ignoring the a type-A exception should be done explicitly.
Type-B exceptions should not be caught. They are programming errors.
Felix Geisendörfer
to wrap their callback/errback into setTimeout() themselves?
> B) When a callback or errback attached to a promise itself throws an exception
no way to "catch" them unless our Deferred implementation provides
some means to do so. If I use a 3rd party module that returns
Deferreds, I would very much like to be able to defend against
mistakes in the authors code whenever possible.
> I'd go with silently ignoring them. If someone wants to handle exceptions,
> they can add a errback handler.
think its going to create a ton of buggy code and debugging pain, so I
wouldn't go that route unless you can rally up arguments & community
support for this.
> perhaps what could be done is to just log
> them to stderr instead and emit a global event so that the developer can
> manage them at the application level
there is no handler scenario better.
-- Felix Geisendörfer aka the_undefined
Joe Bowman
pass by them, then you open the door for lots of buggy code that may
never be caught. Application halting due to error is the surefire way
to raise attention to a problem.
You could possibly extend the process for the http server to always
catch and throw a 500 though.
Ryan Dahl
>
> Ok, so you are saying that Deferreds that fire immediately will have
> to wrap their callback/errback into setTimeout() themselves?
callbacks are added to deferreds after they fire, that the callback
will immediately be called. (The deferred saves a reference to the
result.)
>> B) When a callback or errback attached to a promise itself throws an exception
>> Type-B exceptions should not be caught. They are programming errors.
>
> I'm not sure I agree. Yes, they are programming errors, but - there is
> no way to "catch" them unless our Deferred implementation provides
> some means to do so. If I use a 3rd party module that returns
> Deferreds, I would very much like to be able to defend against
> mistakes in the authors code whenever possible.
modules that suck)
Felix Geisendörfer
> modules that suck)
> Well, one of the nice properties of deferreds, afaik is that if
> callbacks are added to deferreds after they fire, that the callback
> will immediately be called. (The deferred saves a reference to the
> result.)
handler before the beginning of the Deferred anymore.
I guess that's all I need to know. We can always pick up this issue
again if we find out that the map wasn't the territory. For now I'll
go with the hard-crashing Deferreds.
So Ryan, unless you have any remaining concerns I'll start working on
a patch that:
- Replaces Promises with Deferreds
- Keeps the Promise naming for all intersections of both API's, uses
Deferred naming for new methods (like addBoth)
- Provides some unit tests
- Keeps the current wait() implementation
I will push my work in progress here: http://github.com/felixge/node/tree/deferreds
However, please don't clone from it, I might rebase, squash or
otherwise rewrite history on that branch.
-- Felix Geisendörfer
Benjamin Thomas
> callbacks are added to deferreds after they fire, that the callback
> will immediately be called. (The deferred saves a reference to the
> result.)
decided that the normal callback chain of a deferred will do this. The
Deferred implementation will save the result and whenever a callback
is added, pass it to the callback. BUT we have decided on slightly
different functionality for the errback chain. If an error is thrown
or returned then the error is not saved for possible later errback
functions, but is instead thrown if there are no errback handlers to
pass it to.
So we have decided that the two chains of action (callbacks and
errbacks) will operate in slightly different ways.
This is normally not an issue. But the one that Felix has brought up,
is the case where you have something like this:
function returnsDeferred(startValue) {
var d = new Deferred();
if( /* some check here that depends on startValue */ ) {
d.callback(value);
}
else {
d.errback(value);
}
return d;
}
In this case I don't think it can be argued that this function is
poorly written. The returnsDeferred function wants to allow you to
chain callbacks on the result of some computation, but might go into
the errback chain depending on the value you give it.
but if I do this code:
var d = returnsDeferred(someValue);
d.addErrback(function(err) {
// do something with the error
});
the errback function I have added never gets added because the
deferred would have already thrown. Are we saying right now that the
returnsDeferred function should use the setTimeout "trick" or that
maybe deferreds aren't the right functionality to use in this case?
I'm okay with saying that, I think it makes sense, but I think we
should be explicit about the way to address the problem Felix has
raised.
> So unless there are any objections I'd like to work on a patch
> replacing Promises with Deferreds. I propose to keep the current API /
> naming where possible. Once this patch is ready, I'd like to do
> another patch for EventEmitters as suggested for Ryan. I don't want to
> try doing both of those refactorings at the same time, that would just
> become messy. I will also test the patch against my node project
> (transload.it), so I should be able to provide some feedback on how
> much stuff broke and how long it took me to fix it : ).
EventEmitters first and then do Promises. Since Promises are based on
EventEmitters, if you start with Promises it seems like you are going
to have to program the chaining of the callbacks twice, but if you
start with EventEmitters, then you'd only have to do it once.
Felix if you need any help, from manually testing things, to writing
unit tests, to another pair of eyes, I'd be glad to help!
Felix Geisendörfer
> or returned then the error is not saved for possible later errback
> functions, but is instead thrown if there are no errback handlers to
> pass it to.
difference to dojo Deferreds will be the exception catching.
Exceptions will not be caught and passed to the errback chain. If your
deferred raises an exception, it will crash your process unless you
define a global "uncaughtException" handler. If however, you return an
instanceof Error, it will be passed to the errback chain.
> I wonder if maybe it would be easier in the long run to start with
> EventEmitters first and then do Promises.
already implemented and has all behavior specified than to build an
extension on top of it right away.
I will certainly make Promise extend EventEmitter later on in order to
not duplicate any logic. But for now I'd rather leave that code
untouched.
> Felix if you need any help, from manually testing things, to writing
> unit tests, to another pair of eyes, I'd be glad to help!
you are more than invited to write failing unit tests and add
suggestions to improve the code! If we collaborate on that, maybe you
could lead the next effort of refactoring EventEmitter?
-- Felix Geisendörfer aka the_undefined
Ryan Dahl
> Just to make sure we are clear on all of this, thus far we have
> decided that the normal callback chain of a deferred will do this. The
> Deferred implementation will save the result and whenever a callback
> is added, pass it to the callback. BUT we have decided on slightly
> different functionality for the errback chain. If an error is thrown
> or returned then the error is not saved for possible later errback
> functions, but is instead thrown if there are no errback handlers to
> pass it to.
This is only a problem if the function issuing the deferred calls
errback() before returning. That means they haven't performed any I/O,
so they could just raise an exception themselves from the function. EG
x.connect = function (ip) {
this.doConnection(function (hadError) {
if (hadError)
d.errback();
else
d.callback();
});
return d;
};
Ryan Dahl
> a patch that:
>
> - Replaces Promises with Deferreds
> - Keeps the Promise naming for all intersections of both API's, uses
> Deferred naming for new methods (like addBoth)
> - Provides some unit tests
> - Keeps the current wait() implementation
might require changing how arguments are propagated to listeners
(should each listener also return the value they receive?). Certainly
it can be done without EventEmitter, but it seems a shame not to reuse
what is nearly the same thing... I suppose putting them under one roof
could be deferred until a later date. (I hope to move EventEmitter
totally into JS at some time, but it probably won't be for a few weeks
still.)
Also, how will multiple arguments be dealt with? (posix.read)
Felix Geisendörfer
EventEmitter into an über-Deferred will raise many questions of its
own. That's why I suggested to take a small bite first, and re-unite
Deferred & EventEmitter afterwards. If this is not what you want, let
me know and I'll start thinking about the implications for including
EventEmitter in this refactoring.
> Also, how will multiple arguments be dealt with? (posix.read)
Deferred and just offer a callback. For the refactoring I would
probably make all Promises, that currently provide several arguments,
emit an object with several keys on it.
The Deferred API is limited to a single parameter. It brings the
convenience of the return value at the price of a single argument. I
can live with the tradeoff, but if you think its a problem I could
investigate alternatives.
> Ah! okay, thank you for clarifying. I was missing that.
returned by a Deferred callback will behave exactly the same way as
they do in dojo. That means they are memorized and available if
errbacks are attached later on.
-- Felix Geisendörfer aka the_undefined
Benjamin Thomas
> returned by a Deferred callback will behave exactly the same way as
> they do in dojo. That means they are memorized and available if
> errbacks are attached later on.
that the program can expect and unexpected errors.
Rakesh Pai
that the program can expect and unexpected errors.
--
You received this message because you are subscribed to the Google Groups "nodejs" group.
To post to this group, send email to nod...@googlegroups.com.
To unsubscribe from this group, send email to nodejs+un...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/nodejs?hl=en.
Ryan Dahl
>
> I think so too. But the devil is in the details and turning
> EventEmitter into an über-Deferred will raise many questions of its
> own. That's why I suggested to take a small bite first, and re-unite
> Deferred & EventEmitter afterwards. If this is not what you want, let
> me know and I'll start thinking about the implications for including
> EventEmitter in this refactoring.
Felix Geisendörfer
> Another note on this issue: Dojo has this development time flag that can be
> turned on called "debugAtAllCosts: true"
like dojo anymore. If we did something like that I would probably
check if the NODE_DEBUG environment variable is set.
-- Felix Geisendörfer aka the_undefined
Felix Geisendörfer
Good news! I just pushed my node/deferred branch which passes all test
cases:
http://github.com/felixge/node/tree/deferred
I only hit one major wall with promises that currently emit multiple
parameters. Refactoring all of those seemed daunting, and knowing my
procrastination skills, I usually try to not leave my code in a broken
state for hours at a time. So I decided to try adding support for
multiple arguments to dojo Deferreds, and it was surprisingly easy.
Initially I just saw it as a temporary hack to get from A to B, but
now that I have got it working, I wonder if we may want to keep it.
But you can't "return" multiple arguments in a callback I hear you
say. Well, it turns out that you actually can. You just have to do
this within your callback:
var childPromise = new process.Promise();
childPromise.emitSuccess('arg1', 'arg2', ...);
return childPromise;
Sure it's not the prettiest thing in the world, but really - I don't
think it will be an issue. Most promises just emit a single parameter,
so those are not affected. Promises that emit multiple parameters are
fine as well, as long as you don't try to modify the parameters (you
just return undefined). And in those cases where you do want to modify
a callback with multiple parameters, I think you usually will end up
"flattening" those to a single more meaningful parameter anyway. For
all other cases the above solution should suffice.
My branch isn't quite ready to merge yet, but I do want to propose to
keep the multiple argument support as it works right now. It's very
convenient and should not cause any issues, the behavior of passing a
promise along with other arguments to emitSuccess will be
intentionally left undefined.
Let me know what you think!
--- Part B: Backwards compatibility
I have not yet tested the impact of my branch on our production app
(transload.it), but I can already spread the good news that I did not
have to modify any tests in the whole suite! Yes, I did remove the
promise-test-timeout/promise-test-cancel tests, and I modified one
test case that used the timeout() method. I would even bring the
timeout method back if people are interested. But for anybody who has
not been using timeout() or cancel(), this should be a smoother
upgrade than one would expect it to be.
--- Part C: Interested in helping?
I know ben offered help, and maybe others are interested as well. The
main thing left todo at this point is to complete the unit tests to a
point, where they can reasonably be used to define the behavior of our
modified dojo.Deferred implementation. I already started a test case
that shows the multi-argument support:
http://github.com/felixge/node/blob/deferred/test/mjsunit/test-promise.js
I'm thinking we could probably describe most promise behavior by just
building a long callback chain. Either way, I can finish the test
cases myself, but it would be even cooler if somebody else would do it
as he'll not suffer from the same kind of blindness as I do (it's hard
to test code once it already exists and YOU know how it works).
If anybody is interested, please let me know so we can coordinate
things, otherwise I'll go ahead and finish the tests by the end of the
week.
-- Felix Geisendörfer aka the_undefined
Rakesh Pai
--
You received this message because you are subscribed to the Google Groups "nodejs" group.
To post to this group, send email to nod...@googlegroups.com.
To unsubscribe from this group, send email to nodejs+un...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/nodejs?hl=en.
Felix Geisendörfer
Ryan, brought up the idea that Promises are the non-blocking
counterpart to functions in IRC, and therefor should they should only
have a single return value. This really made a lot of sense to me, and
does even more so now that I have gone ahead and applied a patch for
it to my branch:
http://github.com/felixge/node/commit/b4862a5a1c3eabe751b0d3dfddb06bdcadc9c33a
The most interesting part is error handling. If you are now do
emitError({myKey: myVal}) then this object will be turned into an
Error. All properties will mixed into the error object. If you specify
a "message" property, it will overwrite the original message of the
error. This is very cool because you are now able to do this:
.addBoth(function(r) {
if (r instanceof Error) {
// Handle error
}
// Handle success
});
--- Part B: Backwards compatibility II
This change of course had a bigger effect on BC than my initial one.
The semantics of posix.read and sys.exec will change as you can see
below:
posix.read:
emitSuccess(chunk, bytesRead)
-> emitSuccess({chunk: chunk, bytesRead: bytesRead})
sys.exec()
emitSuccess(stdout, stderr)
-> emitSuccess({stdout: stdout, stderr: stderr})
emitError(code, stdout, stderr)
-> emitError({code: code, stdout: stdout, stderr: stderr})
Additionally, if your applications has calls to emitSuccess or
emitError with more than 1 parameter in, this will throw an exception
now as its no longer supported.
--- Part C: Interested in helping?
need updating. Let me know if you are interested!
-- Felix Geisendörfer aka the_undefined
> > .
Rakesh Pai
To unsubscribe from this group, send email to nodejs+un...@googlegroups.com.
--
Rakesh Pai
Felix Geisendörfer
git clone git://github.com/felixge/node.git
cd node
git checkout deferred
./configure
make
sudo make install
Let me know if you need any help,
-- Felix Geisendörfer aka the_undefined
> Felix,
>
> How can I start using your code today?
>
> 2009/12/16 Felix Geisendörfer <fe...@debuggable.com>
>
>
>
> > --- Part A: Multiple parameters for Deferreds II
>
> > Ryan, brought up the idea that Promises are the non-blocking
> > counterpart to functions in IRC, and therefor should they should only
> > have a single return value. This really made a lot of sense to me, and
> > does even more so now that I have gone ahead and applied a patch for
> > it to my branch:
>
> > > > .
> > > > For more options, visit this group at
> > > >http://groups.google.com/group/nodejs?hl=en.
>
> > > --
> > > Rakesh Pai
> > > Mumbai, India.
> > --
>
> > You received this message because you are subscribed to the Google Groups
> > "nodejs" group.
>
> read more »
Karl Guertin
> Felix,
> How can I start using your code today?
git remote add felixge git://github.com/felixge/node.git
git fetch felixge
git checkout -b deferred felixge/deferred
./configure
make && sudo make install
Kris Zyp
significant discussions on CommonJS in regards to promises, and it
seems highly advantageous to the JS community to have convergence in
this area. In CommonJS we originally started with the Dojo promise API
(called Deferreds), and have altered significantly due to the inherent
security problems [1] with it and have arrived at the current proposal
[2], which provides a very simple API for promise providers to
implement, while still composing into much more sophisticated promise
handling [3]. There is good chance Dojo 2.0 may adopt the CommonJS
promise API due to the shortcomings of the current Deferred API
(although probably still use the promise synonym "Deferred"), so
following the Dojo API is not necessarily a good long-term solution.
Anyway, hopefully that is helpful, and I'd certainly invite you
discuss this issue in commonjs to encourage interoperability with
other promise implementing projects (node is certainly not the only
one).
[1] http://groups.google.com/group/commonjs/browse_thread/thread/e93f73ef97e88439/967367b7826aaf04
[2] http://wiki.commonjs.org/wiki/Promises
[3] http://github.com/280north/narwhal/blob/master/lib/promise.js
Kris
On Dec 11, 5:04 pm, Felix Geisendörfer <fe...@debuggable.com> wrote:
> Heya,
>
> why do today what you can do tomorrow?
>
> Currently node.js implements a simple & elegant system for dealing
> with non-blocking operations, the Promise API. You request an
> operation to be performed, and you get a promise object which allows
> you to track the fulfillment of this operation by specifying one or
> more callbacks / error handlers.
>
> As neat as this concept is, anybody familiar with it will have noticed
> its shortcomings:
>
> Promises are hard to build upon. If you want to provide a high-level
> wrapper for some low-level Promise-based API, right now you are stuck
> with creating a second promise and delegating / processing the
> original promise events by hand.
>
> Exception handling is manual. Promises may fail in unexpected ways
> causing the main event loop to crash and node to exit. You can defend
> against this by manually wrapping all of your logic in try..catch
> blocks, but you are out of luck if the issue exists in a 3rd party
> library. The last line of defense, process.addListener
> ('uncaughtException'), is suitable to keep your process from crashing,
> but any kind of failure recovery logic is rather hard to implement on
> top of it.
>
> Promises are hard to chain. While we might be able to do everything in
> parallel with node, there are tons of use cases that require certain
> operations to be performed in a certain sequence. Using the current
> promise API, this will usually lead to very deep nesting and high
> cyclomatic complexity.
>
> Given the critical role that Promises have in the current node.js API,
> it seems only fair to make them capable enough to deal with most of
> the issues that first time users as well as power users will end up
> struggling with. Luckily, there is some great prior API in this field,
> namely the Deferred API as, AFAIK, first seen in Twisted. Most of my
> knowledge about it however, is drawn from the Deferred implementation
> in Dojo.
>
> So, I have spend some time porting Dojo Deferreds to node.js, and I
> think they would be an excellent replacement for the current Promise
> API. For those not familar with Deferreds, I highly recommend the
> documentation of the dojo implementation as a first introduction:
>
> http://api.dojotoolkit.org/jsdoc/1.2/dojo.Deferred
>
> Basically, they are just like Promises with a few differences:
>
> Callback return values are important. The value returned from a
> callback is the value passed to the next callback. This makes it easy
> to build upon Deferreds by adding callbacks that manipulate the
> initial results.
>
> Exceptions are automatically handled. I actually propose a slight API
> change here, but let me explain. The default behavior of Deferreds is
> that any callback invocation is wrapped in a try..catch blok. If you
> do something stupid in your callback, the Deferred automatically
> switched its state to "error" and continues by executing all "errback"
> callbacks. What I like about this, is that you can rest assured that
> 3rd party Deferreds may never crash your application, but rather
> invoke a "errback" callback instead. However, I think this behavior
> should be disabled by default. My initial experience has been that it
> is too easy to oversee exceptions that are being thrown, which is very
> bad during development. Therefor my current port of dojo Deferreds
> diables this behavior by default. If you want to use it, you just
> call .trycatch() on any Deferred you receive to enable it
> transparently.
>
> Last but not least, Deferreds are very easy to chain in sequence. Any
> callback may return a new Deferred itself, which will cause any
> remaining callbacks to be put on hold until that Deferred has
> completed. This makes it trivial to express sequences of asynchronous
> operations.
>
> My current port of (dojo) Deferreds can be seen here:
>
> http://github.com/felixge/node-deferred/
>
> At this point it is essentially identical to dojo Deferreds, except
> that I removed all dependencies to dojo and made the mentioned change
> to the try..catch behavior.
>
> I herby would like to ask all of you to comment on replacing node.js
> Promises with Deferreds. It certainly would make working with async
> operations easier, and backwards compatibility is pretty good for most
> parts.
>
> If nobody sees major issues, I would volunteer to provide the
> necessary patch for node.js.
>
> -- Felix Geisendörfer aka the_undefined
>
> PS: I am also working on syntax sugar and concepts that make it easier
> to aggregate the results from multiple deferreds running in parallel
> or sequence, but those are not quite ready yet and would only
> complicate the discussion.
Kris Zyp
http://groups.google.com/group/nodejs/browse_thread/thread/1feab0309bd5402b
to keep the threading a little more sane. To be clear I agree with all
your hopes for a promise API, and I think the CommonJS promise API is
much closer to Dojo's (and yours) than you might think. We are
definitely on the same page that chaining is helpful and promises are
the non-blocking counterpart to functions. The primary behavioral
difference between Dojo's promises and the CommonJS proposal is that
actually imperative vs functional. With the Dojo Deferred API, the
promise is self-mutating, and the return value of the promise callback
has side effects on the result of the promise, whereas with CommonJS
promises the return value of a promise callback only affects a the
(new) promised returned from the callback addition. And the other
obvious difference is spelling ("then" vs "addCallback"). Also, it is
important to note that the CommonJS *only* defines what a promise
producer minimally needs to implement in order to be consumed by
others. It does not define how the promise should be created (Node has
a fine API for creating promises that works well), and it does not
prohibit additional convenience functions (like addCallback,
addErrback, etc.). In regards to the difference between imperative/
mutating promises and functional promises I'll illustrate with an
example:
CommonJS style:
var promise = asyncAction();
var promisePlusTwo = promise.then(function(value){return value + 2});
promise.then(function(value){print("value: " + value)});
promisePlusTwo.then(function(value){print("value plus two: " +
value)});
might print:
value: 5
value plus two: 7
Dojo style:
var promise = asyncAction();
var promisePlusTwo = promise.addCallback(function(value){return value
+ 2});
promise.addCallback(function(value){print("value: " + value)});
promisePlusTwo.addCallback(function(value){print("value plus two: " +
value)});
might print:
value: 7
value plus two: 7
The side-effects of mutating promises has been bitten Dojo badly too,
with earlier versions, if you didn't return a value in the callback,
then the return value (undefined) would be assigned to the value of
the promise, and future callbacks would get undefined as their passed
in value. We now have a magic check that avoids modifying the Deferred
for undefined return values; magic that shouldn't be necessary.
Mutating promises are bad for object capability systems as well, where
one should be able to pass a promise to untrusted code without
worrying about what it will do with it. Dojo's Deferreds can't be
passed to untrusted if you are adding future callbacks that need to
reliably received values.
In regards to try/catching callbacks, the reason this has to be done
is in case a future callback handler provides the error handler, they
should be able to do so and reliably catch the errors. However, the
problem with errors silently being caught is definitely very
frustrating, and I think the best solution is to have a timeout for
how long an unhandled promise error can go without being handled (by
an error handler callback, and errback in Deferred's terms).
Benjamin Thomas
> promises the return value of a promise callback only affects a the
> (new) promised returned from the callback addition.
> CommonJS style:
> var promise = asyncAction();
> var promisePlusTwo = promise.then(function(value){return value + 2});
> promise.then(function(value){print("value: " + value)});
> promisePlusTwo.then(function(value){print("value plus two: " +
> value)});
> might print:
> value: 5
> value plus two: 7
What happens when value is an object?
var promise = asyncAction();
var promisePlusTwo = promise.then(function(value) {
value.number += 2;
return value;
});
var promise.then(function(value) { sys.p(value); });
promisePlusTwo.then(function(value) { sys.puts('value plus two:');
sys.p(value); });
// outputs what?
What have done is taking a super simple API:
var deferred = new Deferred();
deferred.addCallback(func1).addCallback(func2);
deferred.addCallback(func3);
deferred.addCallback(func4);
Which is simple and intuitive. The functions will get called in this
order: func1, func2, func3, func4, and no matter what, they see the
results of the deferred (albeit maybe modified) in that order. With
the old API all the functions were called in the order that they were
added. Simple stuff.
And made it more ambiguous:
var promiseStart = new Promise();
var promise1 = promiseStart.then(func1);
var promise2 = promiseStart.then(func2);
promise1.then(func3);
promise2.then(func4);
promiseStart.emit(value);
This is called (I assume) in this order: func1, func3, func2, func4.
And it would seem like func2 should see exactly the value that either
promiseStart or func1 emits, but func3 has an opportunity to affect
this even though func2 is added to a different promise.
And then what happens to the course of action if I change the order around some?
var promiseStart = new Promise();
var promise1 = promiseStart.then(func1);
promise1.then(func3);
var promise2 = promiseStart.then(func2);
promise2.then(func4);
promiseStart.emit(value);
It seems like you have taken a simple chain:
func1 --> func2 --> func3 --> func4
and made it a tree:
func1 - --> func2
\---> func3 \---> func4
(I'm sorry if that gets mangled in email formatting).
And from that diagram I don't think func2 should be allowed to affect
the value func3 receives. And if it isn't allowed then you have to
deep copy objects which for simple cases would be easy but could be
quite costly for large objects. And if it is allowed then you have to
think about the value modification in breadth-first versus depth-first
terms which sounds like a headache.
I can see the benefits that you have mentioned about with untrusted
code but at this point I'm not convinced that this benefit outweighs
the added complexity and ambiguity.
> And the other
> obvious difference is spelling ("then" vs "addCallback").
I don't understand why the name change. It is already established in
both Dojo and Twisted as addCallback. Stick with what people know.
Plus, like the explicitness of this:
p.addCallback(callback);
p.addErrback(errback);
as opposed to this:
p.then(callback, errback);
That seems so much nicer. and this:
p.then(null, errback);
is just plain ugly.
But that isn't an important point since you are suggesting that
individual Promise implementations add their own "sugar" functions on
top of this. I still don't understand the name change though.
> The side-effects of mutating promises has been bitten Dojo badly too,
> with earlier versions, if you didn't return a value in the callback,
> then the return value (undefined) would be assigned to the value of
> the promise, and future callbacks would get undefined as their passed
> in value. We now have a magic check that avoids modifying the Deferred
> for undefined return values; magic that shouldn't be necessary.
I don't understand, I thought the Dojo API made it so if you didn't
return a value in the callback the previous value was passed to the
next callback. This seems pretty intuitive and not magical. Am I
missing something?
Further more the new Promise API seems to raise questions about this
exact issue. To quote from the proposal about the `then` function:
> This should return a new promise that is fulfilled when the given
> callback is finished. The value returned from the callback is the
> fulfillment value for the returned promise.
What if the callback doesn't return anything? What is used as the
fulfillment value of the returned Promise? Does the returned promise
get the fulfillment value from the first Promise?
In summary, I'm not convinced Node should adopt the CommonJS Promises
API. While I feel like Node should match CommonJS APIs where
appropriate, if the API doesn't feel right then Node should do what it
thinks is best (see this comment from Felix earlier in this thread:
http://groups.google.com/group/nodejs/msg/541bace648404044). And I
feel like the CommonJS Promise API is questionable enough to warrant
us experimenting with other possibilities.
Kris Zyp
I think we generally came to agreement on most of the issues (Felix,
please correct me if I am wrong). I think we agreed that immutable
promises do make sense (assuming they are chainable), and the CommonJS
API is a decent common denominator between projects and shouldn't be
onerous for Node if specified appropriately. Anyway, Felix brought up
a number of issues that needed clarification, so I updated
http://wiki.commonjs.org/wiki/Promises to be more in line with Felix's
suggestions.
Anyway, let me know if you see any issues with the update CommonJS
proposal, and thanks for the feedback.
In addition, if you are interested, perhaps it might be beneficial to
reuse/share the same module for promises that is used in Narwhal. I
know there is desire and effort going in to see more interoperability
between Narwhal and Node, that might be a good step. Of course, the
promise module could be updated to be more in line with expectations
of Node, including adding addCallback, addBoth, etc. functions on
promises. The promise module also provides a number of convenience
functions for working with promises, such as static promise handlers
that can handle normal values or promises (I use that very extensively
in my framework, Pintura).
Also, just to note, there were still a couple issues that we didn't
really decide about:
* Should errors thrown from callbacks be caught (and routed to the
error handler on the next promise)? - Originally Felix felt no, but it
seems we weren't sure. I specified that they should be in the CommonJS
proposal, since I think that is the most consistent with the
asynchronous form of function calls analogy, and it doesn't require
type checking return values from callbacks, but it seems reasonable
for promise implementors to provide a switch for this.
* Should calls to the callback handlers be put on the event queue or
executed immediately? - Currently unspecified, and I don't know for
sure what is best.
Kris
Felix Geisendörfer
> And made it more ambiguous:
>
> var promiseStart = new Promise();
> var promise1 = promiseStart.then(func1);
> var promise2 = promiseStart.then(func2);
> promise1.then(func3);
> promise2.then(func4);
> promiseStart.emit(value);
after talking to Kris in #commonjs I believe your example could be
rewritten to:
promiseStart = new Promise();
promiseStart
.then(func1)
.then(func2)
.then(func3)
.then(func3);
promiseStart.emit(value);
So as far as I am concerned, we wouldn't be loosing any of the
convenience provided by dojo. To me the callback chaining is a way to
compose new functions out of several async functions (promises). The
idea of immutable promises does make a lot of sense to me and I do
think that's what we should do in node. I don't really like the "then"
API, but if an alias to it is all that's required to be CommonJS
compatible I don't see the harm.
So I'm inclined to give the CommonJS proposal a chance, but I'm still
waiting on some comments from ryan about my initial patch.
-- Felix Geisendörfer aka the_undefined
Benjamin Thomas
> rewritten to:
>
> promiseStart = new Promise();
> promiseStart
> .then(func1)
> .then(func2)
> .then(func3)
> .then(func3);
>
> promiseStart.emit(value);
>
> So as far as I am concerned, we wouldn't be loosing any of the
> convenience provided by dojo. To me the callback chaining is a way to
> compose new functions out of several async functions (promises). The
> idea of immutable promises does make a lot of sense to me and I do
> think that's what we should do in node.
I agree that my example was convoluted and could be done in a better
way. But I still think there are unanswered issues when you change
promises from being a chain to being a tree. And if we are intending
Promises to be thought of and used as a chain then why are making them
a tree?
I think the idea of immutable promises makes a lot of sense. And when
I first read it I thought it was brilliant. But then I started working
through some examples and began to have questions.
Specifically the issue of different branches of the tree modifying the
same object. Am I getting hung up on something that doesn't matter?
Does no one else see an issue with this?
Felix Geisendörfer
memory/performance. But, node will probably not do a lot of this
internally and it shouldn't be an issue when properly used in user
land.
> Specifically the issue of different branches of the tree modifying the
> same object.
I can't think of a case where this would be an issue. If you want to
control sequence, use nesting. Otherwise make sure none of your
callbacks make any assumptions about the order in which they will
fire.
> Am I getting hung up on something that doesn't matter?
> Does no one else see an issue with this?
Not sure. I will have to play more with them myself. This stuff is
sufficiently complex to exceed my imagination at times ; ). If you
have come across problematic situations so far, I'd love to hear about
them.
-- Felix Geisendörfer aka the_undefined
Kris Zyp
couple comments:
Benjamin wrote:
> var promiseStart = new Promise();
> var promise1 = promiseStart.then(func1);
> var promise2 = promiseStart.then(func2);
> promise1.then(func3);
> promise2.then(func4);
> promiseStart.emit(value);
IMO, this code looks like the programmer is actually explicitly
wanting tree-style separation of promises, assigning them to different
variables even. Forcing a linear chain seems like it would be going
against the will of programmer, especially since they could easily add
then the promise handlers in straight chain as Felix demonstrated.
I think it is helpful (and intuitive) to think of promise handlers as
asynchronous counterparts to synchronous computations, taking an input/
parameter(s) value and returning/outputting a value. If you wrote:
var promiseStart = new Promise();
var promisePlusTwo = promiseStart.then(addTwo);
promiseStart.fulfill(4);
This should be asynchronous counterpart to:
valueStart = 4;
valuePlusTwo = valueStart + 2; // or addTwo(valueStart);
If the evaluation of (valueStart + 2) resulted both valuePlusTwo *and*
valueStart being incremented by two, that seems like it would not be
beneficial. Of course, in JavaScript valueStart remains 4 in this
example, and it seems like the analogous promise should behave the
same way.
On Dec 19, 12:09 pm, Felix Geisendörfer <fe...@debuggable.com> wrote:
> My only concern with tree-style promise composition vs chain-style is
> memory/performance. But, node will probably not do a lot of this
> internally and it shouldn't be an issue when properly used in user
> land.
That it is a fair criticism, but the very nature of promises is to
model evaluations that will take some time. It seems very rare that
the construction of a promise (a pretty lean JS object) would be
significant cost in comparison to the typical operations that a
promise is being used for (DB queries, file I/O, network I/O, etc).
> I don't really like the "then"
FWIW, I used "then" as a compromise between Dojo's Deferred API and
the ref_send API as these were the two different promise APIs that
were suggested in CommonJS discussions (ironically I was arguing for
Dojo's API and Tyler Close was arguing for ref_send). However, having
starting using this API more, I have actually have come to really like
"then". Not only is shorter, but it actually seems more meaningful.
"addCallback" properly suggests that a callback is being added, but
doesn't bring much suggestion of the fact that the callback is
participating in the computation, it's return value being used as the
fulfillment for the next promise. "then" actually seems to do a better
job of indicating the more general role of the callback, taking a
input promise and returning an output promise as the next computation
in the data flow. Plus, I still think it reads nicely:
doSomethingAsync().then(doTheNextComputation)
Kris
Benjamin Thomas
> valueStart = 4;
> valuePlusTwo = valueStart + 2; // or addTwo(valueStart);
Alright, I'm on board. My complaint was that you couldn't do this
(translated to Promises):
objectStart = { value: 4 };
objectPlusTwo = objectStart.value + 2;
// objectStart.value == 4 and objectPlusTwo.value ==6
But now that I see it like this I realize you can't do that in
javascript anyway. So, in the branch-able version of promises, if you
don't want the different branches stepping on each others toes, either
deep copy objects or use primitives like Numbers and Strings.
> Not only is shorter, but it actually seems more meaningful.
> "addCallback" properly suggests that a callback is being added, but
> doesn't bring much suggestion of the fact that the callback is
> participating in the computation, it's return value being used as the
> fulfillment for the next promise. "then" actually seems to do a better
> job of indicating the more general role of the callback, taking a
> input promise and returning an output promise as the next computation
Agreed.