Proposed alternative implementation for goog.async.Deferred
bolinfest
because of some of the subtle things that it does. To that end, I am
proposing an alternative implementation named goog.deferred.Deferred
whose behavior is more explicit:
http://bolinfest.com/deferred/
Please take a look and try it out. The differences between
goog.deferred.Deferred and goog.async.Deferred are summarized in the
fileoverview:
http://bolinfest.com/deferred/deferred.js
goog.deferred.Deferred also introduces a number of new instance
methods. You can look at the unit test to see how they are intended to
be used:
http://bolinfest.com/deferred/deferred_test.js
I'm curious to hear what others think of this approach. If there is
enough support for it, I would like to contribute this into the
Closure Library and have the Library use it in place of
goog.async.Deferred.
Cheers,
Michael
Garry Boyer
Ivan Kozik
> In using goog.async.Deferred, I have been bitten by some nasty bugs
> because of some of the subtle things that it does. To that end, I am
> proposing an alternative implementation named goog.deferred.Deferred
> whose behavior is more explicit:
>
> http://bolinfest.com/deferred/
1. What's the motivation behind not wrapping errors with Error?
2. Chaining is really useful. Why would you not want to do it
automatically when a callback/errback returns a Deferred?
3. Cancellers are really useful. Twisted added them after six years,
and it has simplified the design of many other components. Maybe the
goog.async.Deferred cancellation behavior is not ideal, in which case
better behavior should probably be stolen from Twisted.
Ivan
Shawn Brenneman
Ivan Kozik
> 3. I find the cancellation behavior, as written, basically useless. I would
> like to change it so that cancellation does not always trigger the error
> chain, since cancellation is not always an error. For example, we might want
> to stop downloading a feed because the user has closed the widget that
> requires it.
Right, cancellation is not always an error, and your canceller can
trigger the callback chain if it wants to. The implementation of
cancel in deferred.js does this after calling the canceller:
if (!this.hasFired()) {
this.errback(new goog.async.Deferred.CancelledError(this));
}
Is triggering the callback chain in the canceller enough to do what
you want? Not calling either chain would break the guarantees that
Deferreds provide, and I think it would make it harder to compose
Deferreds. I can talk more about this, but there people far more
qualified.
I'll point to these resources for Deferred design, in case anyone is interested:
1. The Twisted source code, which uses Deferreds in hundreds of
places, including some with cancellers.
2. Twisted's twisted/internet/defer.py, especially before
http://twistedmatrix.com/trac/ticket/411 was merged (which complicated
the code slightly).
3. #twisted on irc.freenode.net, which will happily talk about
Deferred design in general.
4. http://twistedmatrix.com/trac/ticket/990 , a general discussion of
how cancellers should work. It's a bit hard to wade through.
I'm not saying everything should work like their implementation, but
they have been thinking about the problem for 8 years, so it's wise to
leverage their output.
Ivan
Ivan Kozik
> I'll point to these resources for Deferred design, in case anyone is interested:
Forgot the mention the paper that introduced the Deferred,
"Generalization of Deferred Execution in Python":
http://www6.uniovi.es/python/pycon/papers/deferex/
as well as the modern documentation:
http://twistedmatrix.com/documents/10.2.0/core/howto/defer.html
http://ezyang.com/twisted/defer2.html
Ivan
bolinfest
them, but here goes:
(1) I don't think that Error should be considered special. Since you
can't "bind" deferred.errback to a catch block, I don't see how it
helps you do the right thing -- how are you passing an Error to the
callback when you meant to pass it to the errback? What's worse is
that there is no way to extract the data from the Error created by the
call to errback(). For example:
deferred.errback({idThatFailedToLoad: 42});
deferred.addErrback(function(obj) {
fixit(obj.idThatFailedToLoad); // This doesn't work and I can't
extract the data passed to the errback!
});
The workaround I have been using is:
var e = new Error();
e.stuffICareAbout = {idThatFailedToLoad: 42};
deferred.errback(e);
deferred.addErrback(function(e) {
fixit(e.stuffICareAbout.idThatFailedToLoad);
});
I could also create a subclass of Error and use that, though it seems
fairly ridiculous to create an entire class just to pass my one value
around. Though the easiest fix that would still work would be for
goog.async.Deferred to subclass Error with a class that takes
stuffICareAbout as a constructor parameter and then has a getter so I
can access it in the errback.
(2) I think treating a callback or errback that returns a Deferred as
special is clever enough to be dangerous. I think explicit chaining
via map() or bind() (those are defined in goog.deferred.Deferred) or
some other function would be better. Similar to how an event handler
can return false or call e.preventDefault() -- I think the former is a
subtlety that most developers are not aware of and makes it too easy
to get themselves into trouble. By comparison, e.preventDefault() is
explicit and unambiguous.
(3) Neither (1) nor (2) are publicly documented in
goog.async.Deferred, which is what I say contributes most to these
horrible surprises.
(4) I think the contract of a Deferred is stronger if there is no
cancel() method. I had not noticed that it calls all of the errbacks
(until Garry pointed it out) because that is not mentioned in the
JSDoc, either. Though Shawn, I have to disagree with you on this one
because I think the one good thing that cancel() does is call all of
the errbacks; otherwise, I think it would be too easy to get a memory
leak, no?
Maybe step 1 is just to clean up the JSDoc for goog.async.Deferred. I
think we are talking about two different types of chaining, so it
would be helpful if the contract of how chaining is meant to work were
specified better. Again, I think that thinks like map() and bind() as
I have defined them (again, I'm not committed to the names) make
things more explicit and are much simpler to explain.
On Dec 9, 6:46 pm, Ivan Kozik <i...@ludios.org> wrote:
Ivan Kozik
> I could also create a subclass of Error and use that, though it seems
> fairly ridiculous to create an entire class just to pass my one value
> around. Though the easiest fix that would still work would be for
> goog.async.Deferred to subclass Error with a class that takes
> stuffICareAbout as a constructor parameter and then has a getter so I
> can access it in the errback.
I agree. An Error subclass like this would be a good thing to have in
Closure Library.
> (4) I think the contract of a Deferred is stronger if there is no
> cancel() method. I had not noticed that it calls all of the errbacks
> (until Garry pointed it out) because that is not mentioned in the
> JSDoc, either. Though Shawn, I have to disagree with you on this one
> because I think the one good thing that cancel() does is call all of
> the errbacks; otherwise, I think it would be too easy to get a memory
> leak, no?
I don't think cancel() makes the contract any weaker. Cancellers
simply allow some code to suggest "stop doing stuff if possible",
which stops useless operations (because the result is no longer
needed). The Deferred still must go through either the callback or
errback chain.
FWIW, one way to think about Deferred features is to think of the
synchronous code analog. addErrback is like try {} catch {}. addBoth
is like try {} finally {}. A canceller in synchronous code would be
"if someone_stopped_me_early: abort()".
Ivan
Nathan Naze
last week, and also spent time researching the topic this evening. I
won't claim that deferreds/futures/promises are an area of expertise.
The current implementation is a branched and modified version of
Dojo's [1], which was based on MochiKit [2], which in turn was
inspired by Twisted [3]. These also informed the versions in jQuery
[4].
I'm not particularly fond of the current API -- the methods and names
are not intuitive ("addErrback", "addCallback" vs "addCallbacks",
etc), and it dispenses with patterns used elsewhere in Closure. At
some level, it's reimplementing observer/eventTarget, and I wonder if
it should just be one. Also, why is it not Disposable? It's a
powerful concept, but for it to be used widely, I'd want a
well-written basic class, which this isn't. Also, I've had some
painful experiences working with this version -- swallowing of runtime
errors being the worst of them.
I agree with Bolin on the confusion or returning a callback as a
special case. It conflates the two, and I'd rather chaining be more
explicit. What would help in all of these is sample usages of the new
API. I'd also like to see better tools for working with asynchronous
actions.
Finally, I am fond of jQuery's usage of "Promises" as a subset of
Deferreds, and this is a good candidate for a subclass -- read-only
views of futures. Most things should be returning "promises" that
can't be ended by outside code.
http://en.wikipedia.org/wiki/Futures_and_promises#Read-only_views
Nathan
[1] http://dojotoolkit.org/api/1.3/dojo/Deferred
[2] http://mochi.github.com/mochikit/doc/html/MochiKit/Async.html#fn-deferred
[3] http://twistedmatrix.com/documents/10.1.0/api/twisted.internet.defer.Deferred.html
[4] http://api.jquery.com/category/deferred-object/
Fredrik Blomqvist
Swallowing of runtime errors could potentially be handled using this
technique:
http://dean.edwards.name/weblog/2009/03/callbacks-vs-events/ (uses a
DOM-trick, thus unfortunately only usable in a browser environment).
Or, in some cases, by simply wrapping the callback in a setTimeout(),
assuming you don't
need to manipulate the value-chain.
Personally I call setTimeout wrapping a "listener" (addListener) and
try to use it when
I only need the result, the Deferred is still passed on.
For the record, a slight modification that reduces error hiding is
already implemented in the Closure fork of MochiKit code (similar in
Dojo) by piping the previous result if the callback returns undefined,
it was discussed here:
http://groups.google.com/group/closure-library-discuss/browse_thread/thread/bea5c42a8ae193ab/d1833bb3b7d6f9dd
// Fredrik Blomqvist
>
> Nathan
>
> [1]http://dojotoolkit.org/api/1.3/dojo/Deferred
> [2]http://mochi.github.com/mochikit/doc/html/MochiKit/Async.html#fn-defe...
> [3]http://twistedmatrix.com/documents/10.1.0/api/twisted.internet.defer....
> [4]http://api.jquery.com/category/deferred-object/
>
>
>
>
>
>
>
> On Fri, Dec 10, 2010 at 12:11 PM, Ivan Kozik <i...@ludios.org> wrote:
Nathan Weizenbaum
Garry Boyer
I've been using Deferred heavily recently, and I'm overall quite fond of the API. The error hiding is a problem, but I've found the rest of it to work quite well. I use it in very different circumstances from those in which I use events: a Deferred is a value that will materialize at some point in the future, whereas events are notifications that something has happened. The Deferred-specific functionality of chaining values, branching, cancelling, and sending errors through the chain has been very valuable to me.I would be saddened if chaining deferreds became substantially more verbose than it is today. I value the fluidity of being able to saydeferred.addCallback(function(val) { return val.foo(); }).addCallback(function(val) { return val.bar(); })....
Nathan Naze
> API. The error hiding is a problem, but I've found the rest of it to work
> quite well. I use it in very different circumstances from those in which I
> use events: a Deferred is a value that will materialize at some point in the
> future, whereas events are notifications that something has happened.
Right. "Materialization" of the value (or failure to materialize) is
the event in question.
I wouldn't want to impose a more-verbose usage, and it's probably
worth making shorthands for "attach a listener, pull the result, and
pass it to the next in line" because it's such a common operation.
jQuery calls these "filters" for use with "pipe()", or, as Garry
suggested "process".
Nathan
