CommonJS Promises and the Q API

1608 views
Skip to first unread message

Kris Kowal

unread,
Feb 20, 2010, 7:11:11 PM2/20/10
to Tyler Close, Kris Zyp, comm...@googlegroups.com, Mark Miller (Google), Ihab Awad, Tom Van Cutsem
Tyler,

Kris Zyp and I have been working to bring "promises" to CommonJS.
We've both studied and written variations of the ref_send Q API [1]
from Waterken.

I ported "ref_send" to Narwhal [2] by adapting it to our module system
[3] and factoring out the queuing API so that it could be implemented
differently for various ES engines both on the server and in browsers.
These changes, I think, could be easily back-ported to Waterken and
adapted to work both with CommonJS modules and with legacy browser
module systems [4], arranging for the "exports" to apply to an
autonomous "window.namespace.Q" if the "exports" free variable does
not exist, and using "setTimeout" for queuing if the "require" free
variable does not exist.

There are a few reasons we have opted not to adopt your implementation
of the Q API wholesale for all CommonJS engines.

The first, naturally, is that we're in the business of writing
specifications, not implementations. I am very interested in reverse
engineering a specification from your implementation. We would need a
specification for the Q API and also a specification for Promises,
which I have inferred are distinct; the Q API appears to be able to
work with a variety of conforming "promise" implementations. We are
very interested in making "duck-promises".

In "ref_send", a duck-promise appears to be a Function object that
accepts calls of the form:

promise("WHEN", ok(value), error(reason))
promise("GET", resolve, propertyName)
promise("POST", resolve, methodName, argsArray)
promise("PUT", resolve, propertyName, value)
promise("DELETE", resolve, propertyName)

Where "resolve" is a Deferred.resolve function that accepts a "more
fulfilled value" as its argument, meaning either a plain JavaScript
value, an unresolved promise, or a rejected promise.

Let me pose for proof by contradiction that the obvious alternative to
using a Function for promises would be polymorphic objects with
method names corresponding to the first argument of the former form:

promise.WHEN(ok(value), error(reason))
promise.GET(resolve, propertyName)
promise.POST(resolve, methodName, argsArray)
promise.PUT(resolve, propertyName, value)
promise.DELETE(resolve, propertyName)

It is my intuition that the reason why you opted to use "Functions"
for "Promises" instead of "Objects" was that if two mutually
suspicious, potentially malicious folks were to receive identical
references to a promise, they could interfere with one another by
monkey-patching the promise object with alternate methods.

var promise = yourImpregnableApi();
(function (oldThen) {
promise.then = function (ok, error) {
…interleaved lies, mischief…
};
})(promise.then);

At the time of your implementation, using a closure to maintain
control over the internal methods of a promise was the only option.
With ECMAScript 5, we have Object.freeze. Given that using objects
directly is more intuitive to a JavaScript programmer, and given that
there is no longer a reason to bend over backwards to maintain API
consistency in this way, I propose that the CommonJS specification for
Promise objects be this latter form with the caveat that, to defend
against suspicious users, a promise must be frozen. This verbiage is
delicate; there is a tension between performance, usability, and
security, especially since promises are useful for reasons apart from
security.

As for nomenclature, it is my intuition that you opted to use ALL_CAPS
method names to make obvious the relationship between the methods and
HTTP request codes, which have implications for caching on the wild
web. I think that in CommonJS, we should use oneHump case for method
names. This unfortunately would necessitate the renaming of "delete"
to something like "remove" or "del" since it is a reserved keyword and
not possible to use as a method name until ES5.

The utility of "WHEN" is now obvious to Kris Zyp and me, but Kris Zyp
(and countless others without a doubt) have reservations about "GET",
"PUT", "POST", and "DELETE". I think I can address this issue myself,
but please correct me if I'm wrong.

The reason for providing "GET", "PUT", "POST", and "DELETE" is to
permit a Q API to pipeline the corresponding actions on a chain of
promises, to reduce chatty network communication, and to enable the
corresponding property mutations and method calls on "far references".
A "WHEN" clause is sufficient for all communication among pure
JavaScript values in a single process, but you need "GET", "PUT",
"POST" and "DELETE to operate on remote objects, given the lack of
catch-alls (methodMissing, __getattr__) or proxies [5] in current
ECMAScript implementations. I think that we can safely "defer"
discussion of this portion of a Q and Promise API specification.

Given this information, Kris Zyp has proposed [6] and prototyped [7]
[8] Q API's around the assumption that it would be sufficient to have
a promise object with only the "WHEN" behavior as a method.

promise.WHEN(ok(value), error(reason));

However, in practice, it looks desirable to call the method "then".

a.then(b).then(c)

…which concisely illustrates the causal chain and potentially reduces
nesting. It's become my understanding that *using* the Promise API
directly in this fashion is hazardous for several reasons. Even
though the Promise API exposes the ability to invoke the "WHEN"
behavior directly in both cases, doing so introduces two hazards. To
illustrate the hazards, here is the unassailable equivalent pattern:

Q.when(a, function (a) {
return Q.when(b, function (b) {
return c(b);
};
});

The "Q.when(promise, ok, error)" function provides two important
invariants:

1.) that either "ok" or "error" will be called exactly once.
2.) that the "ok" or "error" callbacks will be called in a future
turn.

These are important because, unlike the promise returned by a
potentially malicious API (a, b, or c), the "ok" and "error" callbacks
close on state that is important to the integrity of your algorithm.
By calling:

var internalCounter = 0;
a("WHEN", function ok(value) {
internalCounter++;
}, function error(reason) {
});
ASSERT.equal(internalCounter, 0);

We are giving the "a" promise the ability to violate the invariant
that internalCounter will be 0 until a future turn, and never greater
than 1. This is possible because a malicious promise could call "ok"
in the current turn, or call "ok" multiple times in any number of
turns, at any time.

This is avoided by using the trustworthy Q.when(a, ok, error) because
internally, the "when" method constructs a new
"deferred.{promise,resolve}" object which can only be resolved in the
current turn, can only be resolved once, and can only enqueue an event
with your "ok" and "error" methods in a future turn.

The important point to glean from this is that the anti-pattern is in
the usage of the Promise API, not in the implementation of the Promise
API or Q API. In both cases, one could foolishly give a suspicious
promise the ability to fiddle with your algorithm's state. The
anti-pattern is furthermore virulent because it is syntactically
compelling. In most cases, promises will be trustworthy, so we are
putting our faith in the ability of every programmer to decide whether
to use Q.when(a) or a.then() based on their security issues.

I do not have a conclusion. There merely appears to be a tension
here.

In any case, I would like to invite you to attack some of the ideas in
Kris Zyp's and my variations of the "ref_send" API.

There's also the matter of attempting to assist folks migrating from
existing Deferred implementations in JavaScript, notably the Dojo API
[9], the MochiKit [10] API, and the now defunct NodeJS "Promise" API.
Kris Zyp's and my variations on "ref_send" [6,7,8,11] both support
some combination of:

addCallback(ok(value))
addErrback(error(reason))
addBoth(ok(value), error(reason))
addCallbacks(ok(value), error(reason))

Another "innovation" of these libraries is that we melded the ideas of
Promises and Deferreds to some degree. It is my understanding that
one distinction between Deferreds and Promises is that Deferreds have
methods for both resolving an eventual value and the ability to
observe the resolution of the value, whereas in Promise systems, the
ability to observe and resolve are separated into separate objects, so
the ability observe the resolution of an eventual value can be safely
given to two different and suspicious users without giving them the
ability to lie to one another about the resolution of the promise:

In camp 1:

var a = myImpregnableApi();
a.addCallback(function (a) {
// I trust that myImpregnableApi gave me
// the value "a"
};

In camp 2:

var a = myImpregnableApi();
a.resolve("lies, mischief");

In this case, camp 1 and 2 have both received the same eventual value,
but camp 2 has opted to use the promise to lie about the eventual
value to camp 1. Neither the impregnable API nor camp 1 have grounds
to defent themselves.

So the big technological advancement of Promises [12], is that it
separates the concerns of resolution and observation of eventual
values. Also, unless you're adapting a callback pattern to a promise
pattern, using "Q.when" elegantly obscures the separation most of
the time, so calls like these will seldom seen beyond adapter code:

defer() // as it exists in "ref_send"
new Deferred()
Deferred()

var delay = function (timeout) {
var deferred = Q.defer();
setTimeout(function () {
deferred.resolve();
}, timeout);
return deferrred.promise;
});

On a side node, this example illustrates another weakness of the
"ref_send" implementation in my experience. "undefined" is treated as
a special value to which a promise cannot eventually resolve. As Tom
van Cutsem has pointed out elsewhere, one of the nice things about a
Promise API is that an API that returns immediate values can easily be
adapted to an API that returns promises to resolve values eventually
because any return value is forwardable. For some functions, like
"array.forEach", to adapt them to an eventual resolution pattern, it
is necessary for them to resolve eventually to "undefined", to
postpone any actions that depend on its completion; we run into this
in adapting sync JSGI to async JSGI [14].

http://gist.github.com/293510

exports.ContentLengthMiddleware = function (app) {
return function (request) {
return Q.when(app(request), function (response) {
var stream = new ByteIO();
var write = function (line) {
stream.write(line.toByteString("utf-8"));
};
return Q.when(
response.body.forEach(write),
function () {
var content = stream.read();
response.headers['Content-length'] =
content;
response.body = [content];
return response;
}
);
});
};
};

Returning to the topic, we've (in our variations collectively
[7,8,11]) blended the Deferred API and this innovation of the Promise
API. The general notion coming from E is that there is a defer()
method that returns a [promise, resolver] pair. "ref_send" already
makes the leap to making an idiomatic JavaScript object instead of a
positional record: Q.defer() returns a {promise,
resolve(moreFulfilledValue)} record. We observe that this object is
like a Deferred in the sense that it contains both the API to resolve
a promise and to observe its resolution. So instead of separating the
Deferred from the Promise entirely, we have made them "separable".
The Deferred is the same, dangerously flawed object with the combined
ability to observe and resolve, but it contains a promise property
that contains only the ability to observe, which can be safely passed
to a suspicious user.

var deferred = new Q.defer(); // ref_send
var deferred = new Q.Deferred(); // extension

// resolution
deferred.resolve(moreFulfilledValue); // ref_send
deferred.reject(error); // for convenience

// migratory observation
deferred.then(ok, error); // ref_send-ish
deferred.addCallback(ok(value)); // dojo
deferred.addErrback(error(reason)); // dojo
deferred.addBoth(ok(value), error(reason)); // dojo
deferred.addCallbacks(ok(value), error(reason)); // mochi

// secure subset
deferred.promise.then(ok, error); // ref_send-ish
deferred.promise.addCallback(ok(value)); // dojo-ish
deferred.promise.addErrback(error(reason)); // dojo-ish
deferred.promise.addBoth(ok(value), error(reason)); // dojo-ish
deferred.promise.addCallbacks(ok(value), error(reason));

This comes from a tension between migration and security. I'm sure
you would argue that we should cull the ability to observe the
resolution from the deferred object to discourage the anti-pattern of
returning the whole deferred object to a suspicious user. At this
point, I would tend to agree; I'm okay with a clean break. Others
will not be so anxious.

In particular, my "events" module [11] experimentally attempts to
maintain a couple invariants using Object.freeze and Object.create
without compromising the reusability of constructor code. In the
course of maintaining these invariants, the code ended up looking very
unfamiliary and icky to the few people who reviewed it, and also
rather slow in comparison to less constrained variations. Those
invariants are:

1.) promises are immutable, so cannot be used as a cross-talk channel
among mutually suspicious recipients
2.) when an object-used-as-an-instance-of-a-type, it is guaranteed
that the method is not owned by the object, to distinguish it from
the property of an object-used-as-a-record. I use this invariant
in my "util" module [16] to distinguish a polymorphic method from
an owned property.

The implementation is an experiment with a layered architecture for
promises, starting with low-level event broadcasting emitters, then
observable object methods, then deferreds (promise, resolver objects
in our parlance), and promises.

I also observed that "valueOf" is an idiomatic variation of "near".
Having a "valueOf" method on promises, identical in behavior to
"Q.near", produces very nice run-time properties.

My last nits with regard to "ref_send" are that the "POST" method
takes an array of arguments; I think it would be more elegant to use
variadic arguments.

promise("POST", resolver, methodName, args);
promise.post(resolver, methodName, ...args);
// illustrated with "rest" notation in consideration for ES5+

This is an appeal for you and other experts in promises to work with
us in developing a CommonJS standard for implementations of the Q API
and the Promise API. Kris Zyp's proposal for promises has been
misleadingly occupying the root page for the Promises topic on the
CommonJS wiki [6]. We should move this to Promises/A, revise the
Promises page to, like other pages, introduce the topic and link to
the proposals, including Promises/B [15], which I hope you would be so
kind as to draft.

Kris Kowal

.. [1] http://waterken.sourceforge.net/web_send/
http://waterken.sourceforge.net/web_send/wsh/
"ref_send" Q API
.. [2] http://github.com/280north/narwhal/blob/master/lib/ref-send.js
Narwhal port of "ref_send"
.. [3] http://wiki.commonjs.org/wiki/Modules/1.1
CommonJS/Modules/1.1
.. [4] http://gist.github.com/281374
Isaac Schlueter's CommonJS/Browser adapter pattern
.. [5] http://wiki.ecmascript.org/doku.php?id=harmony:proxies
Tom van Cutsem and Mark Miller's proposal for Proxies for ES5+
.. [6] http://wiki.commonjs.org/wiki/Promises
Kris Zyp's CommonJS Promise Proposal
.. [7] http://github.com/280north/narwhal/blob/master/lib/promise.js
Kris Zyp's Narwhal Promise implementation
.. [8] http://github.com/kriszyp/node-promise
Kris Zyp's Node Promise implementation
.. [9] http://api.dojotoolkit.org/jsdoc/1.3/dojo.Deferred
Dojo Deferred API
.. [10] http://mochikit.com/doc/html/MochiKit/Async.html
MochiKit Deferred API
.. [11] http://github.com/280north/narwhal/blob/master/lib/events.js
my "events" module for Narwhal
.. [12] it is beyond scope of this discussion, but it has not escaped
our attention that Proxies [5] could be used to make
referentially transparent promises as they exist in E, with
caveat that it would only work for Objects, not Values.
.. [13] http://gist.github.com/305128
adapting setTimeout to a promise implementation
.. [14] http://wiki.commonjs.org/wiki/JSGI
JSGI
.. [15] http://wiki.commonjs.org/wiki/Promises/B
Yet to be Written Q and Promises API proposal
.. [16] http://github.com/280north/narwhal/blob/master/lib/util.js
Narwhal's "util" module.

Tyler Close

unread,
Feb 22, 2010, 8:52:26 AM2/22/10
to Kris Kowal, Kris Zyp, comm...@googlegroups.com, Mark Miller (Google), Ihab Awad, Tom Van Cutsem
Thank you for providing feedback on the ref_send API and inviting a
proposal for CommonJS. I look forward to defining / adapting it to
meet your needs.

One of the primary concerns expressed in your email is the oddity of
the interface to a promise itself. This interface is defined in the
source code at:

http://waterken.svn.sourceforge.net/viewvc/waterken/server/trunk/waterken/config/file/site/ref_send.js?view=markup#l_94

There are several important reasons for this seemingly odd design. The
encapsulation/freezing argument you note in your email is one reason.
Until object freezing becomes available on all the platforms that
CommonJS wishes to support, a function interface is the only way to
prevent a client from tampering with a promise. There are also a
couple other reasons to prefer the function interface.

As you note in your email, directly invoking methods on a promise is
an anti-pattern. Operations on a promise come with certain
expectations for the program's control flow. For example, you can
think of ref_send's when() function as like the asynchronous version
of JavaScript's if statement. It would be disorienting if the kind of
object used as the condition of an if statement affected when the if
block was executed and how many times it was executed. The programmer
expects the if block to execute immediately (if the condition is
truthy) and only once. Similarly the programmer should expect a when
block to only execute in a future event loop turn (if the promise is
resolved) and only once. It is impossible to ensure these semantics if
a when statement is expressed as a direct invocation on a promise
object. Instead, we express a when statement the same way we do an if
statement, as one that takes the condition as an argument, rather than
as an invocation target.

So the oddness of the interface to a promise itself is actually one of
the design goals. I want it to feel odd to write unsafe code. The Q
API is the one designed to be easy and familiar, and safe to use.

This style of the Q API also follows the convention in JavaScript of
using prefix notation for control flow statements, instead of infix
notation. In JavaScript, the conditional statements like if and loop
statements like while all use prefix notation. Prefix notation can be
almost as compact as infix notation. In your email, you were concerned
that the infix case:

a.then(b).then(c)

would have to be written as:

Q.when(a, function (a) {
return Q.when(b, function (b) {
return c(b);
};
});

but, it could just as well be written as:

Q.when(Q.when(a, b), c)

Another reason to prefer the function interface to a promise is that
it inherently supports catch-all semantics, whereas JavaScript objects
currently don't. For example, take a look at the implementation of a
deferred promise in ref_send:

http://waterken.svn.sourceforge.net/viewvc/waterken/server/trunk/waterken/config/file/site/ref_send.js?view=markup#l_142

Notice that it can queue up any kind of operation on a promise (WHEN,
GET, POST, ...) without knowing in advance what that operation is.
When the promise is resolved, all of these operations can be forwarded
on to the resolved reference. A class-based interface to a promise
would be unable to implement these semantics and so we would be unable
to define new kinds of operations on promises, since any intermediate
promise in a chain of promises might not have a corresponding method.

So we've got 4 good reasons to prefer the function-based interface to a promise:
1. It supports encapsulation
2. It discourages unsafe direct invocation on a promise.
3. It encourages following the JavaScript convention of using prefix
notation for control flow.
4. It supports pass-through of arbitrary operations on a promise.

In your email, you also express a desire to retain some level of
familiarity for users of Deferreds. I like the goal, but it seems hard
to achieve once you've decided on prefix notation for control flow
operations. I am hoping that the switch from infix to prefix won't be
that hard, especially if you've already grasped the idea of a deferred
execution, which is a bigger change.

As you note in your email, making the promise a separate object from
the resolver is an important safety measure. Less important, but also
useful, is making the resolver a separate object from the promise.
Sometimes, it's useful to give several different clients the resolver,
but not the promise. The first one to use the resolver determines the
value of the promise, but none of the other clients get to see this
value. It's like a race where none of the losers get access to the
prize.

In ref_send, I treat undefined and null as a kind of rejected promise
because that's typically what they mean. Consider code like:

Q.when(getOutputStream(), function (out) {
out.write(...);
},function (reason) {
log('no output stream provided: ' + reason);
});

In the above code, we only want the success branch of the when
statement to execute if we actually got an output stream. All other
cases should be routed to the error handler. If the output stream
resolves to undefined, that should go to the error handler, since we
can't write output to undefined.

In the case of array.forEach in your email, it returns an array of
selected values, so the error handler again seems like the right
choice if it instead returns undefined. If no values are selected, an
empty array should be returned.

Your suggestion to have promises support valueOf is interesting. I
need to study all the cases where valueOf gets called implicitly to
make sure this always makes sense. At first, this does seem like a
potentially valuable improvement. Does anyone have a cheat sheet for
all the cases where valueOf is implicitly invoked?

One of the reasons Q.post() takes a single "args" argument, that is
typically an array, instead of having a variadic signature, is to
support invocations that don't take an array. For example, lots of
JSON services take a POST operation that accepts a single JSON object
as the entity content. Others take an array of arguments. Using a
variadic signature for Q.post would prevent client code from
specifying how to invoke the remote service. For example, consider a
JSON service that takes a single argument. With a variadic signature,
how would you know whether to send a JSON text of:

[ { "foo" : 42 } ]

versus

{ "foo" : 42 }

With the current Q.post signature, the first is expressed as:

Q.post(target, 'someMethod', [ { foo: 42 } ])

and the second as:

Q.post(target, 'someMethod', { foo: 32 })

I'll add an entry to the CommonJS wiki for the current ref_send API,
but look forward to hearing your thoughts on all of the above.

Thanks,
--Tyler

--
"Waterken News: Capability security on the Web"
http://waterken.sourceforge.net/recent.html

Kris Walker

unread,
Feb 22, 2010, 10:01:28 AM2/22/10
to CommonJS
I've been mulling over http://github.com/280north/narwhal/blob/master/lib/events.js
for a few days since Kris posted it to a related discussion. I really
liked the layered approach, and wrote a re-implementation of it as for
a project I'm working on. In the past I've also wrote my own versions
of centralized event handlers for various projects, as well as worked
with Dojo and Mochikit deferreds.

One commonality that I see in all of this is that the higher order
logic always seems end up as a continuation passing scheme more than a
message passing scheme, as much as we would like to make it look
otherwise. Reasoning about a message passing system seems easier,
because that's they way we always think about method calls and HTTP
requests. However, breaking that cognitive barrier and thinking more
in terms of continuations, even regarding HTTP, allows us to create
better programs in my opinion.

Also, I think what we are really trying to accomplish with promises,
deferreds, events, and so on is the architecture that is so
brilliantly laid out in chapter 5 of the Fielding dissertation (http://
www.ics.uci.edu/~fielding/pubs/dissertation/rest_arch_style.htm),
Representational State Transfer. Correct me if I'm wrong, but I think
REST is at the heart of these proposals, although not necessarily in
the sense of describing the web or other networked architectures.

I would not shed a tear over the loss of addCallback(), addErrback(),
or addBoth(). They lead to nested callbacks and the breakage of the
continuation passing structure.

Also, although I've implemented and advocated deferreds and message
passing "channels", I think this idea should be removed from the
proposals. Instead, I would like to see this standard specify only
the safe subset of promises. I think this subset would be a good
restraint on the general idea, producing a scalable and robust
architecture.

For example, let's say I want to monitor a long running process on
another thread, possibly in an extension like XPCOM, NPAPI plugin, or
Rhino extension. Instead of allowing me to open a direct channel with
it, this other "processing vat" would return a promise to me. When
that promise is fulfilled or aborted, I just call the processing vat
and ask for another one. This is like long polling in Ajax apps.

I know this throws up a barrier to legacy code, but I would rather
trade off my legacy code for a better, well adopted, architecture. So,
I guess, I'm asking what some of the biggest arguments are against
dropping the deferred and callback pattern in the the CommonJS spec?
Kris Kowal mentioned migration, but that is not a strong enough
argument for keeping the bad stuff, in my opinion. Maybe a callback
pattern can be specified in a different spec if it is really needed.

-- Kris Walker

Kris Zyp

unread,
Feb 22, 2010, 10:19:59 AM2/22/10
to Tyler Close, Kris Kowal, comm...@googlegroups.com, Mark Miller (Google), Ihab Awad, Tom Van Cutsem

On 2/22/2010 6:52 AM, Tyler Close wrote:
> Thank you for providing feedback on the ref_send API and inviting a
> proposal for CommonJS. I look forward to defining / adapting it to
> meet your needs.
>
> One of the primary concerns expressed in your email is the oddity of
> the interface to a promise itself. This interface is defined in the
> source code at:
>
> http://waterken.svn.sourceforge.net/viewvc/waterken/server/trunk/waterken/config/file/site/ref_send.js?view=markup#l_94
>
> There are several important reasons for this seemingly odd design. The
> encapsulation/freezing argument you note in your email is one reason.
> Until object freezing becomes available on all the platforms that
> CommonJS wishes to support, a function interface is the only way to
> prevent a client from tampering with a promise. There are also a
> couple other reasons to prefer the function interface.
>
>

Fortunately, that day has come (or will very shortly) for CommonJS
implementors. Object or property freezing is available on most of the
primary server side environments that CommonJS implementors are using
(Rhino and V8, which represents the majority of users, I believe), and
probably will be available on all of them in a matter of months. I know
some are using CommonJS on the browser, but they can't implement all the
APIs anyway, and have to choose which ones (and to what degree). So I
don't think we need to worry about creating object-based promises that
can't be protected from tampering.

> As you note in your email, directly invoking methods on a promise is
> an anti-pattern. Operations on a promise come with certain
> expectations for the program's control flow. For example, you can
> think of ref_send's when() function as like the asynchronous version
> of JavaScript's if statement. It would be disorienting if the kind of
> object used as the condition of an if statement affected when the if
> block was executed and how many times it was executed. The programmer
> expects the if block to execute immediately (if the condition is
> truthy) and only once. Similarly the programmer should expect a when
> block to only execute in a future event loop turn (if the promise is
> resolved) and only once. It is impossible to ensure these semantics if
> a when statement is expressed as a direct invocation on a promise
> object. Instead, we express a when statement the same way we do an if
> statement, as one that takes the condition as an argument, rather than
> as an invocation target.
>
> So the oddness of the interface to a promise itself is actually one of
> the design goals. I want it to feel odd to write unsafe code. The Q
> API is the one designed to be easy and familiar, and safe to use.
>
>

.then() is too easy to use, ok.


> This style of the Q API also follows the convention in JavaScript of
> using prefix notation for control flow statements, instead of infix
> notation. In JavaScript, the conditional statements like if and loop
> statements like while all use prefix notation. Prefix notation can be
> almost as compact as infix notation. In your email, you were concerned
> that the infix case:
>

.then() is too hard to use? Doesn't this contradict the last point?

> a.then(b).then(c)
>
> would have to be written as:
>
> Q.when(a, function (a) {
> return Q.when(b, function (b) {
> return c(b);
> };
> });
>
> but, it could just as well be written as:
>
> Q.when(Q.when(a, b), c)
>
> Another reason to prefer the function interface to a promise is that
> it inherently supports catch-all semantics, whereas JavaScript objects
> currently don't. For example, take a look at the implementation of a
> deferred promise in ref_send:
>
> http://waterken.svn.sourceforge.net/viewvc/waterken/server/trunk/waterken/config/file/site/ref_send.js?view=markup#l_142
>
> Notice that it can queue up any kind of operation on a promise (WHEN,
> GET, POST, ...) without knowing in advance what that operation is.
> When the promise is resolved, all of these operations can be forwarded
> on to the resolved reference. A class-based interface to a promise
> would be unable to implement these semantics and so we would be unable
> to define new kinds of operations on promises, since any intermediate
> promise in a chain of promises might not have a corresponding method.
>

For far reference extended promises, one can easily add another method
representing arbitrarily named operations. There is nothing preventing
someone from polymorphically extending promises with this type of ability.

> So we've got 4 good reasons to prefer the function-based interface to a promise:
> 1. It supports encapsulation
> 2. It discourages unsafe direct invocation on a promise.
> 3. It encourages following the JavaScript convention of using prefix
> notation for control flow.
> 4. It supports pass-through of arbitrary operations on a promise.
>
>

These are mainly usability concerns, and the reality is that people
really are mainly interacting with trusted code, and many want to use
infix style promise interaction. You can try to convince everyone one
otherwise, but I don't think CommonJS is in a position to make such
opinionated APIs. The two aspects of the promise design:
Security - Object-based promises *are* securable on CommonJS platforms.
Usability - Some people *do* want to use infix style interaction.

I certainly have no objection to evangelizing the use of the Q API as a
more secure/predictable approach to using promises. However, I don't see
any reason to change the current design for the promises themselves
based on these objections. Thank you for clarifying these though.

I may have missed something in this argument, but forEach always returns
undefined when successful.

--
Thanks,
Kris

Kris Kowal

unread,
Feb 22, 2010, 11:02:48 PM2/22/10
to comm...@googlegroups.com
[+list] # normally I don't have to reply-all…weird. Please reply to this one.

---------- Forwarded message ----------
From: Kris Kowal <kris....@cixar.com>
Date: Mon, Feb 22, 2010 at 7:58 PM
Subject: Re: CommonJS Promises and the Q API
To: Tyler Close <tyler...@gmail.com>

On Mon, Feb 22, 2010 at 5:52 AM, Tyler Close <tyler...@gmail.com> wrote:
> So we've got 4 good reasons to prefer the function-based interface
> to a promise:
> 1. It supports encapsulation
> 2. It discourages unsafe direct invocation on a promise.
> 3. It encourages following the JavaScript convention of using prefix
> notation for control flow.
> 4. It supports pass-through of arbitrary operations on a promise.

I *personally* would not miss "p.then(ok, error)", as I would never
use it directly, and I think that providing a promise could be made
convenient in other ways.  Let me break down that value judgement:

1. I think the encapsulation argument is significantly weakened by
Object.freeze. (±0)

2. Discouraging direct use: given that we cannot prevent direct use of
promises, I think that making it awkward with the stratagems of
ALLCAPS methods in addition to making them Function objects is
probably a good fallback. (+.5)  However, I do not think that we
should encourage the *provision* of promises as a function with a
switch/case.  We should come up with a Promise constructor  with a
sane API.  For example:

return Promise({
   "when": function (resolve, reject) {},
   "get": function (resolve, name) {},
});

The Promise meta-objects could be inherited prototypically.  The
fallback to rejecting could exist implicitly.  Alternately, we could
make it possible to provide a catch-all explicitly:

return Promise({
   "progress": function (resolve) {
   }
}, function others (resolve) {
   resolve(Q.reject("reason"));
});

With this in mind, our specification could focus rather than on what
kinds of objects Promises are, but on how to work with them while
making no "promises" what they are or how they work directly.  We
might even in our implementations be able to prevent them from being
used directly.

3. Forcing "when" to appear like a control-flow block syntactically is
unconvincing. Prefix and infix notation communicate intent equally
well regardless of their appearance, and I don't think people would be
confused either way. I respect your aesthetic taste, but others
aesthetic tastes differ. (±0)

4. Without Tom's proxies, catch-all behavior for arbitrary message
forwarding is definitely possible with Functions and not possible with
Objects. (+1)

An important thing about this Promise constructor is that we could
have Promises that inherit default "get", "put", and "post" if the
provider elects not to provide pipe-lining.

> In your email, you also express a desire to retain some level of
> familiarity for users of Deferreds. I like the goal, but it seems
> hard to achieve once you've decided on prefix notation for control
> flow operations. I am hoping that the switch from infix to prefix
> won't be that hard, especially if you've already grasped the idea of
> a deferred execution, which is a bigger change.

I'm not married to Deferreds personally.  If Kris Walker's sentiment
on the issue is common, I think we should start anew.

> As you note in your email, making the promise a separate object from
> the resolver is an important safety measure. Less important, but
> also useful, is making the resolver a separate object from the
> promise.  Sometimes, it's useful to give several different clients
> the resolver, but not the promise. The first one to use the resolver
> determines the value of the promise, but none of the other clients
> get to see this value. It's like a race where none of the losers get
> access to the prize.

Very good.  Thanks for that insight.  We'll have to put that example
in the book.

> In ref_send, I treat undefined and null as a kind of rejected promise
> because that's typically what they mean. Consider code like:
>
> Q.when(getOutputStream(), function (out) {
>    out.write(...);
> },function (reason) {
>    log('no output stream provided: ' + reason);
> });
>
> In the above code, we only want the success branch of the when
> statement to execute if we actually got an output stream. All other
> cases should be routed to the error handler. If the output stream
> resolves to undefined, that should go to the error handler, since we
> can't write output to undefined.

Whether "undefined" is an error will varry by case.  It should be
possible for "undefined" to be a fully resolved value for the cases
where a function just needs time to arrange for invariants to be
satisfied, but has no computed value.  I think it would be better to
provide a utility to decorate a promise so that "undefined" implies
rejection than to make "undefined" and "null" implicitly rejections.

Q.when(Q.defined(getOutputStream()), function (out) {


   out.write(...);
},function (reason) {
   log('no output stream provided: ' + reason);
});

> In the case of array.forEach in your email, it returns an array of


> selected values, so the error handler again seems like the right
> choice if it instead returns undefined. If no values are selected,
> an empty array should be returned.

Array.prototype.forEach returns "undefined".

Array.prototype.map returns an Array of collected values.

> Your suggestion to have promises support valueOf is interesting. I
> need to study all the cases where valueOf gets called implicitly to
> make sure this always makes sense. At first, this does seem like a
> potentially valuable improvement. Does anyone have a cheat sheet for
> all the cases where valueOf is implicitly invoked?

x.valueOf() gets called by the following patterns:

   Number(x)
   +x
   -x
   x >>> 0
   '' + x

But not:

   String(x)
   !meta

A more vigorous poring over the spec for "[[Value" would seem to be in
order, I agree.

> One of the reasons Q.post() takes a single "args" argument, that is
> typically an array, instead of having a variadic signature, is to
> support invocations that don't take an array. For example, lots of
> JSON services take a POST operation that accepts a single JSON
> object as the entity content. Others take an array of arguments.
> Using a variadic signature for Q.post would prevent client code from
> specifying how to invoke the remote service. For example, consider a
> JSON service that takes a single argument. With a variadic
> signature, how would you know whether to send a JSON text of:
>
> [ { "foo" : 42 } ]
>
> versus
>
> { "foo" : 42 }
>
> With the current Q.post signature, the first is expressed as:
>
> Q.post(target, 'someMethod', [ { foo: 42 } ])
>
> and the second as:
>
> Q.post(target, 'someMethod', { foo: 32 })

I see, but I'm suspicious.  This indicates that there are two classes
of promise objects for which "post" is applicable.  The first class is
those that support JavaScript calling conventions, and then another
where the arguments are embodied by the root of an arbitrary JSON
object.  It strikes me that you would have to be concious of which
type of promise you're working with, which would make them less
generic.

I would think that invoking a method on a promise representing a
distant object mediated by HTTP would be subject to the same rules.

Perhaps the convention for mediating an arbitrary JSON REST system,
nominally not designed to be mediated by a JavaScript promise, would
conventionally provide and consume a single argument?

However, I could be convinced of the Array case simply because it's so
inconvenient to arrange for a variadic apply.

Q.post.apply(Q, [target, 'someMethod'].concat(args));

That being said, that kind of thing is normal in JavaScript until we
get "rest" notation, which I expect to be in the next couple years for
the server side, albeit a decade for browsers.

Q.post(target, 'someMethod', ...args);

Kris Kowal

Reply all
Reply to author
Forward
0 new messages