I stumbled upon CommonJS just recently, via CouchDB and Node.js. One
thing which are in my interests are the different control flow
methodologies handled via promises / deferreds. I read through the
Promises page on the wiki, and the two proposals, and their code. In the
end, I found I wasn't terribly happy with either of the proposals and
decided to make my own proposal. This e-mail is the prelude to that
proposal - if it gets a positive reception, I will write a full proposal
in the wiki.
* Background
I'm very new to actually writing proper JavaScript code, so I'm probably
doing a lot trivial things in an awfully complex way and will probably
miss some of the subtleties of the language. I have, however, read
through the ECMA-262 specification (both edition 3 and 5) and tried to
catch all the tricky things in JavaScript.
The one area I'm totally unfamiliar with is how hostile code can be
handled in JavaScript. The Promises requirements list a number of
"safely" and "defends" conditions, but I'm not really sure what I can
trust and what I can't. For example, if I call a non-trusted function,
what can it do besides loop forever and throw exceptions? Can it do some
nasty business to the calling code? Does it get something usable from
"callee"? Or, if I pass a function (closure) to non-trusted code - what
can it do besides call the function with all manner of arguments? Can it
access the function locals? What are the possibilities?
I have done quite a bit of Python, and there this kind of "defending"
would be insane - functions have access to a myriad of internal things,
such as call chains, locals, globals, bytecompiled code, etc. so there's
never a safe way to have hostile code in the same environment. That
hasn't obviously stopped people from trying to do this for years - but
there's always some little detail that gets overlooked. I'm hoping the
"defending" done here isn't similar - works for the common case, but
there's always a way around it.
* Promise security
Most of the requirements about promises seem to be centered around
making sure that programmers can not, even intentionally, cause errors
by not behaving as expected by some contract of a promise. Promises/A
seems to use "freeze" for some stuff, although I'm not sure if it was
mandatory for safety. Promises/B seems to create closures for functions
and use that to hide stuff - which is the approach I've taken here as
well. However, both seemed to complicate the issue by objects and
multiple operations and such - so I tried to simplify the whole promise
stuff to a bare minimum.
I came up with this:
- When reciving an untrusted input promise, there's now way to know if
it will adhere to any contract, or even if it is a promise at
all. Hence, to operate on the given in value in a promise-compatible
manner, it will always need to be wrapped in to something that
enforces the promise contract. This wrapping has to be done by a
function (or a constructor) exposed by the promises module. I am
calling this "when".
- The wrapping function need to have some way to provide the input
promise the ability to resolve the promise without exposing any of
it's internal state and keeping the internal invariant of not
resolving twice intact. For this, there should be a way to create a
function that will, when called, resolve the promise once and only
once with the values. I am calling this function "resolve".
- The wrapping function needs to return a value that fulfills the
promise contract and allows a callback to be attached that will be
called when the promise is resolved. The *only* operation needed for
this returned promise is the adding of a callback - if the promise
has already been resolved, the callback is resolved immediately - if
the promise has not yet been resolved, the callback is stored to be
called later. I am calling this function "then".
- For convenience, the callback wrapping function "then" should return
a new promise, that will be fulfilled when the value returned by the
callback is fully resolved. This allows for chaining of "then" calls
in a natural manner. However, the return value of the callback is
again possibly untrusted, and not necessarily a promise, so it
definitely needs to be wrapped with "when".
- For creating the promises, there is a need to get the "resolve"
function and the "then" function - this is separating the concerns
for "resolving" from "observing resolution". These functions always
appear in pairs as they need to reference the same resolution
state. Hence, there should be an exported function (or constructor)
to create a pair of "resolve" and "then" functions. I am calling this
function "pair" - with the meaning of "create a pair". This is to
emphasize the fact that the two functions do not are not really a
single object and they do not depend on the value of this - even
though my function returns them in a (simple) object for convenience.
* Interface
For callback convention, I chose to have callbacks of the form:
f(err, val)
That is, the first argument is a possible error - if it is not true,
then val is the returned value. The choice is easy to change, and it is
trivial to emulate other conventions on top of this (such as addCallback
vs. addErrback). I picked this, because I was writing this on top of
Node.js. It should be also pretty easy to modify this to have variable
length argument lists.
With this, I got:
- pair() -> {then, resolve}: a function that creates a pair of
functions representing the "observer" and "resolver" sides of a new
promise.
- then(callback) -> then: a function callable with a single argument,
that is the callback to be added. The return value is a new "then"
function as explained above. If the callback throws an exception,
this causes promise to be resolved with an error. For clarity, the
function has a property called "then" pointing to itself - this makes
it possible to actually include the word "then" in the code to signal
the reader what is going on.
- resolve(err, val) -> undefined: a function callable with two
arguments, giving the possible error or value. Returns nothing.
- when(value) -> then: a function returning a "then" function to which
callbacks will be added. If the value is not a promise, then "when"
will return an already resolved promise. If the value is a promise,
then "when" will attempt to add a "resolve" function as a callback to
that promise - if this adding attempt causes an exception, the
returned promise will be resolved with an error.
- ispromise(value) -> boolean: a function for determining if a value is
a promise. Right now it just checks for the existence of a "then"
property, but it can be made as lenient or as lax as wanted.
Taking this down to the absolute bare minimum, the only exported
function needs to be "when" - everything else can be derived from
that. In practice, exposing "pair" is useful for anybody wanting to
bridge in to any existing async interfaces.
* Example
var promise = require("./promised")
promise.when(3).then(function (err, val) {
return val*2;
}).then(function (err, val) {
console.log(val);
});
promise.when(5).then(function (err, val) {
var p = promise.pair();
setTimeout(p.resolve, 500, undefined, val*2);
return p.then;
}).then(function err, val) {
console.log(val);
}
Obviously for common usage this needs a bit more plumbing around it -
like wrappers providing for easy forwarding of error values, etc. - but
this is meant just as an example of the core functionality, the bare
minimum, to get it working.
* Implementation
Here's the full implementation - it's short enough to be included here
for discussion:
exports.ispromise = ispromise;
function ispromise(x) {
if (x && x.then) {
return true;
} else {
return false;
}
}
function run_cb(resolve, callback, err, val) {
var ret;
try {
ret = callback(err, val);
} catch (e) {
resolve(e, undefined);
return;
}
if (ispromise(ret)) { /* Needs check to prevent infinite loop in creating whens */
when(ret).then(resolve);
} else {
resolve(undefined, ret);
}
}
exports.pair = pair;
function pair() {
var resolved, err, val, callbacks;
var resolve = function resolve(e, v) {
if (resolved) {
throw Error("promise was attempted to resolve twice");
}
resolved = true;
err = e;
val = v;
if (callbacks) {
for (var i = 0; i < callbacks.length; i++) {
run_cb(callbacks[i][0], callbacks[i][1], err, val);
}
callbacks = undefined;
}
}
resolve.resolve = resolve;
var then = function then(callback) {
var p = pair();
if (resolved) {
run_cb(p.resolve, callback, err, val);
} else {
callbacks = callbacks || [];
callbacks.push([p.resolve, callback]);
}
return p.then;
};
then.then = then;
return { resolve: resolve,
then: then };
}
exports.when = when;
function when(v) {
var p = pair();
if (ispromise(v)) {
try {
v.then(p.resolve);
} catch (e) {
try {
p.resolve(e, undefined);
} catch(e) {} /* NOTE: eat away duplicate resolution in error handling */
}
} else {
p.resolve(undefined, v);
}
return p.then;
}
* Open issues
- What is the best callback calling convention? Are there better ideas
than "err, val"? Perhaps varargs style "err, ..."? I'd like to avoid
having multiple functions that would separate success callbacks from
error callbacks - and instead just build such distinctions on top of
the common callbacks that handle both.
- Is the requirement for not calling the callbacks in the same turn
really necessary? I find that in most case, I would *not* want
that. If I really need that functionality, the caller can make sure
their callback queues up the actual execution for the next turn. An
automatic wrapper is trivial to make for this. Obviously, infinite
recursion causes infinite stack usage, so a next-turn eval is needed
in those cases - but those should be really rare!
- Should calling "resolve" twice cause an exception? It's often a
programming error and it's easier to catch if an exception is thrown,
but almost as often it's just what you meant and avoiding (or
ignoring) the exception is just useless extra work.
- Helpers, helpers, helpers. This needs a bunch of helpers somewhat
akin to the "async" library.
- success: callback wrapper to call the callback only on success
values and re-raise errors
- failure: callback wrapper to pass success values onwards and only
call the callback on errors
- enqueue: callback wrapper to call the callback on the next turn
- resolved: return an already resolved promise for a value (almost
the same as "when" with a non-promise value)
- rejected: return an already errored promise from an exception
- nodewhen: call the given function with the given arguments,
appending a final "resolve" argument at the end and returning a
"then" - matching the node.js calling convention
* Final words
So, what do you think? Nothing new under the sky, or am I on to
something? Or just too obscure, better to do with explicit objects and
easily understandable state.
Thanks in advance,
-- Naked
There's a lot going on right now on the list, but just on cursorial
inspection, your requirements and the requirements for the Promises/B
proposal seem to be well aligned. In my opinion, Promises/B does a
better job of meeting those requirements, and permits resolutions and
rejections to be more cleanly forwarded. A Promises/B implementation
is similarly easy to implement.
exports.defer = defer;
function defer() {
var pending = [], value;
var promise = Object.create(Promise.prototype);
promise.emit = function () {
var args = Array.prototype.slice.call(arguments);
if (pending) {
pending.push(args);
} else {
forward.apply(undefined, [value].concat(args));
}
};
var resolve = function (resolvedValue) {
var i, ii, task;
if (!pending)
return;
value = ref(resolvedValue);
for (i = 0, ii = pending.length; i < ii; ++i) {
forward.apply(undefined, [value].concat(pending[i]));
}
pending = undefined;
};
return {
"promise": promise,
"resolve": resolve,
"reject": function (reason) {
resolve(reject(reason));
}
};
}
exports.Promise = Promise;
function Promise(descriptor, fallback) {
if (fallback === undefined) {
fallback = function (op) {
return reject("Promise does not support operation: " + op);
};
}
var promise = Object.create(Promise.prototype);
promise.emit = function (op, resolved /* ...args */) {
var args = Array.prototype.slice.call(arguments, 2);
var result;
if (descriptor[op])
result = descriptor[op].apply(descriptor, args);
else
result = fallback.apply(descriptor, arguments);
if (resolved)
return resolved(result);
return result;
};
return promise;
};
exports.isPromise = isPromise;
function isPromise(object) {
return object instanceof Promise;
};
exports.reject = reject;
function reject(reason) {
return Promise({
"when": function (rejected) {
return rejected ? rejected(reason) : reject(reason);
}
}, function fallback(op, resolved) {
var rejection = reject(reason);
return resolved ? resolved(rejection) : rejection;
});
}
exports.ref = ref;
function ref(object) {
if (isPromise(object))
return object;
return Promise({
"when": function (rejected) {
return object;
},
"get": function (name) {
return object[name];
},
"put": function (name, value) {
object[name] = value;
},
"delete": function (name) {
delete object[name];
},
"post": function (name, args) {
return object[name].apply(object, args);
}
});
}
exports.when = function (value, resolved, rejected) {
var deferred = defer();
var done = false;
forward(ref(value), "when", function (value) {
if (done)
return;
done = true;
deferred.resolve(ref(value).emit("when", resolved, rejected));
}, function (reason) {
if (done)
return;
done = true;
deferred.resolve(rejected ? rejected(reason) : reject(reason));
});
return deferred.promise;
};
function forward(promise /*, op, resolved, ... */) {
var args = Array.prototype.slice.call(arguments, 1);
setTimeout(function () {
promise.emit.apply(promise, args);
}, 0);
}
Kris Kowal
I read your proposal and these suggestions, and I certainly agree with
virtually all of these. Your ideas look great, well thought through. But
it sounds like you are in agreement with or arguing for Promises/A. All
of your proposals/suggestions either match Promises/A or have or
could/should be built on Promises/A. The only proposal I saw that
suggested something different was using a single callback to then()
instead of two like in Promises/A. I didn't look like this was a key
part of your proposal, and the rationale for two callbacks is that code
will inevitably branch anyway, and being able to omit the error callback
is important for easing error delegation without having to manually
rethrow (like a function without a try/catch block).
Anyway, Promises/A is intended to be a minimal interface such that
developers can create promise libraries and tools (like your
suggestions) on top of it. If you see anything in Promises/A that needs
clarification or improvement, lets get those in the there (feel free to
propose textual changes).
Your proposal looks like it would make an great library, I'd encourage
you to implement it!
--
Thanks,
Kris
On 10/24/2010 4:26 PM, Nuutti Kotivuori wrote:
> Kris Zyp <kri...@gmail.com> writes:
>> I read your proposal and these suggestions, and I certainly agree with
>> virtually all of these. Your ideas look great, well thought through. But
>> it sounds like you are in agreement with or arguing for Promises/A. All
>> of your proposals/suggestions either match Promises/A or have or
>> could/should be built on Promises/A. The only proposal I saw that
>> suggested something different was using a single callback to then()
>> instead of two like in Promises/A. I didn't look like this was a key
>> part of your proposal, and the rationale for two callbacks is that code
>> will inevitably branch anyway, and being able to omit the error callback
>> is important for easing error delegation without having to manually
>> rethrow (like a function without a try/catch block).
>>
>> Anyway, Promises/A is intended to be a minimal interface such that
>> developers can create promise libraries and tools (like your
>> suggestions) on top of it. If you see anything in Promises/A that needs
>> clarification or improvement, lets get those in the there (feel free to
>> propose textual changes).
>>
>> Your proposal looks like it would make an great library, I'd encourage
>> you to implement it!
> Thank you for the encouraging words - I think I'll complete the library
> :-)
>
> However, there was also a reason why I didn't just go with Promises/A.
>
> Like you say, Promises/A is intended to be a *minimal* interface, on top
> of which to create promise implementations. To rephrase this, Promises/A
> is just the minimal glue code between *different* promise
> implementations - the common contract that all the mutually suspicious
> promise implementations can adhere to.
>
> That's fine - I think it is good to define the lowest common denominator
> between all promise implementations and to gain complete
> interoperability between them.
>
> However, since the assumptions on the requirements page call for a
> promise implementation that is able to provide a consistent interface on
> top of something that might even be malicious, *no* consumer of the
> interface can actually use the callback as-is - instead they have to
> wrap it in their version of "when" which transforms the unreliable
> promise to a well-behaving promise of their choosing. So, since nobody
> needs to use the "raw" promise interface directly, it doesn't really
> matter if it is convenient to use for a user - only that it is the best
> interface possible for bridging between different promise
> implementations.
>
> In that light, Promises/A is pretty good - I'm not sure if it's the
> best, but if I get strong opinions about the matter, I'll write about
> it.
>
> But what is CommonJS Promises really supposed to be? Is it supposed to
> be *just* a framework in which various promise systems can exist - or is
> there supposed to be a standard promise API, on top of which CommonJS
> Promises users can code their own programs?
You don't need a spec to do this, you can already do this, just add a
dependency to your package and do this:
var promise = require("nuuttis-awesome-promises/promised");
promise.when(3).then(...
> I think the latter. Glue
> between promise implementations is good - but the user needs a
> *complete* promise API to work with, and it would be good if this API
> would be a part of CommonJS.
> Hence, Promises/A doesn't really touch much of the important stuff -
> since it doesn't define at all how promises are created, nor the other
> functions required for working with promises. The implementation written
> by you for Promises/A obviously has the full API, but on the whole I
> wasn't terribly thrilled by it, nor by the actual implementation.
>
> So, I think the Promise proposal should be split in half - one half
> dealing basically just with what is the best interface for consuming an
> input promise - and a part of this is defining what a promise is - is it
> duck-typed or something else. And the second half dealing with what
> would be the best standard API for a promise library. And this is mostly
> what I've concentrated on for now.
>
> While it is cool to have multiple interoperating (and mutually
> suspicious) promise implementations - I'm not convinced that this
> actually is better than a single, well-written an well-tested
> implementation that is used by everybody.
A single, well-written implementation that everyone uses is great, but
you earn that the old-fashioned way, by writing the best module/package
and getting adoption of it. I don't think CommonJS exists to bless
libraries. The less complicated specifications we produce, the more room
for people like you to innovate and create an awesome library. If we
attempted to standardize more promise APIs, it would just get in the way
of whoever has even better promise ideas in the future. Promises/A has
several implementations, but minimal enough that I don't think it would
get in your way if you wanted to use make your library work with it.
JavaScript/CommonJS is very open ecosystem. I certainly don't want to
discourage your from discussing your ideas (here or elsewhere) and
letting us know about your implementations. Design the best library out
there and go for it!
--
Thanks,
Kris
I also used this technique in my promise module. In IE, purportedly
named function expressions create uncollectable reference cycles.
Kris Kowal
Thank you for the encouraging words - I think I'll complete the library
:-)
Promises users can code their own programs? I think the latter. Glue
between promise implementations is good - but the user needs a
*complete* promise API to work with, and it would be good if this API
would be a part of CommonJS.
Hence, Promises/A doesn't really touch much of the important stuff -
since it doesn't define at all how promises are created, nor the other
functions required for working with promises. The implementation written
by you for Promises/A obviously has the full API, but on the whole I
wasn't terribly thrilled by it, nor by the actual implementation.
So, I think the Promise proposal should be split in half - one half
dealing basically just with what is the best interface for consuming an
input promise - and a part of this is defining what a promise is - is it
duck-typed or something else. And the second half dealing with what
would be the best standard API for a promise library. And this is mostly
what I've concentrated on for now.
While it is cool to have multiple interoperating (and mutually
suspicious) promise implementations - I'm not convinced that this
actually is better than a single, well-written an well-tested
implementation that is used by everybody.
-- Naked