C++ Promises

[Kenton Varda]

I just pushed the initial version of the KJ async API, which is based on E-style promises, and is also very similar to the Javascript Promises/A+.

https://github.com/kentonv/capnproto/blob/master/c%2B%2B/src/kj/async.h

I'm excited about this because I've found that C++11 makes this style work really well.  Asynchronous code looks almost like regular synchronous code.  Also, since you aren't calling the callbacks directly, you don't have to worry about a callback deleting you before it returns or crazy deadlock scenarios.  By default, callbacks occur on the same thread where they were registered, meaning by default you don't have to fret over thread-safety, but it's also pretty easy to schedule a callback on a different thread (via EventLoop::there()).

Currently there's not much you can do with this, because I haven't implemented any APIs using it yet.  But this is what Cap'n Proto interfaces will be based on.  Additionally, Cap'n Proto types will provide special subclasses of Promise which implement pipelining -- more on that later.

Here's the key parts of the interface:

template <typename T>

class Promise: public PromiseBase {

  // The basic primitive of asynchronous computation in KJ.  Similar to "futures", but more

  // powerful.  Similar to E promises and JavaScript Promises/A.


  //
  // A Promise represents a promise to produce a value of type T some time in the future.  Once

  // that value has been produced, the promise is "fulfilled".  Alternatively, a promise can be

  // "broken", with an Exception describing what went wrong.  You may implicitly convert a value of

  // type T to an already-fulfilled Promise<T>.  You may implicitly convert the constant

  // `kj::READY_NOW` to an already-fulfilled Promise<void>.


  //
  // Promises are linear types -- they are moveable but not copyable.  If a Promise is destroyed

  // or goes out of scope (without being moved elsewhere), any ongoing asynchronous operations

  // meant to fulfill the promise will be canceled if possible.


  //
  // To use the result of a Promise, you must call `then()` and supply a callback function to

  // call with the result.  `then()` returns another promise, for the result of the callback.

  // Any time that this would result in Promise<Promise<T>>, the promises are collapsed into a

  // simple Promise<T> that first waits for the outer promise, then the inner.  Example:

  //
  //     // Open a remote file, read the content, and then count the

  //     // number of lines of text.
  //     // Note that none of the calls here block.  `file`, `content`

  //     // and `lineCount` are all initialized immediately before any


  //     // asynchronous operations occur.  The lambda callbacks are
  //     // called later.

  //     Promise<Own<File>> file = openFtp("ftp://host/foo/bar");

  //     Promise<String> content = file.then(


  //         [](Own<File> file) -> Promise<String> {
  //           return file.readAll();

  //         });
  //     Promise<int> lineCount = content.then(

  //         [](String text) -> int {
  //           uint count = 0;

  //           for (char c: text) count += (c == '\n');


  //           return count;
  //         });

  //
  // For `then()` to work, the current thread must be looping in an `EventLoop`.  Each callback

  // is scheduled to execute in that loop.  Since `then()` schedules callbacks only on the current

  // thread's event loop, you do not need to worry about two callbacks running at the same time.

  // If you explicitly _want_ a callback to run on some other thread, you can schedule it there

  // using the `EventLoop` interface.  You will need to set up at least one `EventLoop` at the top

  // level of your program before you can use promises.


  //
  // To adapt a non-Promise-based asynchronous API to promises, use `newAdaptedPromise()`.

  //
  // Systems using promises should consider supporting the concept of "pipelining".  Pipelining

  // means allowing a caller to start issuing method calls against a promised object before the

  // promise has actually been fulfilled.  This is particularly useful if the promise is for a

  // remote object living across a network, as this can avoid round trips when chaining a series

  // of calls.  It is suggested that any class T which supports pipelining implement a subclass of

  // Promise<T> which adds "eventual send" methods -- methods which, when called, say "please

  // invoke the corresponding method on the promised value once it is available".  These methods

  // should in turn return promises for the eventual results of said invocations.


  //
  // KJ Promises are based on E promises:

  //   http://wiki.erights.org/wiki/Walnut/Distributed_Computing#Promises

  //
  // KJ Promises are also inspired in part by the evolving standards for JavaScript/ECMAScript

  // promises, which are themselves influenced by E promises:


  //   http://promisesaplus.com/
  //   https://github.com/domenic/promises-unwrapping


public:
  Promise(T value);

  // Construct an already-fulfilled Promise from a value of type T.  For non-void promises, the

  // parameter type is simply T.  So, e.g., in a function that returns `Promise<int>`, you can

  // say `return 123;` to return a promise that is already fulfilled to 123.


  //
  // For void promises, use `kj::READY_NOW` as the value, e.g. `return kj::READY_NOW`.


  inline Promise(decltype(nullptr)) {}


  template <typename Func, typename ErrorFunc = _::PropagateException>

  auto then(Func&& func, ErrorFunc&& errorHandler = _::PropagateException())

      -> PromiseForResult<Func, T>;

  // Mostly equivalent to `EventLoop::current().there(kj::mv(*this), func, errorHandler)`.


  //
  // Note that `then()` consumes the promise on which it is called, in the sense of move semantics.

  // After returning, the original promise is no longer valid, but `then()` returns a new promise.

  //
  // As an optimization, if the callback function `func` does _not_ return another promise, then

  // execution of `func` itself may be delayed until its result is known to be needed.  The


  // here expectation is that `func` is just doing some transformation on the results, not
  // scheduling any other actions, therefore the system doesn't need to be proactive about

  // evaluating it.  This way, a chain of trivial then() transformations can be executed all at

  // once without repeatedly re-scheduling through the event loop.


  //
  // On the other hand, if `func` _does_ return another promise, then the system evaluates `func`

  // as soon as possible, because the promise it returns might be for a newly-scheduled


  // long-running asynchronous task.
  //

  // On the gripping hand, `EventLoop::there()` is _always_ proactive about evaluating `func`.  This

  // is because `there()` is commonly used to schedule a long-running computation on another thread.

  // It is important that such a computation begin as soon as possible, even if no one is yet

  // waiting for the result.
  //

  // In most cases, none of the above makes a difference and you need not worry about it.



  T wait();


  // Equivalent to `EventLoop::current().wait(kj::mv(*this))`.  WARNING:  Although `wait()`
  // advances the event loop, calls to `wait()` obviously can only return in the reverse of the

  // order in which they were made.  `wait()` should therefore be considered a hack that should be

  // avoided.  Consider using it only in high-level and one-off code.  In deep library code, use

  // `then()` instead.
  //

  // Note that `wait()` consumes the promise on which it is called, in the sense of move semantics.

  // After returning, the promise is no longer valid, and cannot be `wait()`ed on or `then()`ed

  // again.
};





class EventLoop {

  // Represents a queue of events being executed in a loop.  Most code won't interact with

  // EventLoop directly, but instead use `Promise`s to interact with it indirectly.  See the

  // documentation for `Promise`.
  //

  // You will need to construct an `EventLoop` at the top level of your program.  You can then

  // use it to construct some promises and wait on the result.  Example:


  //
  //     int main() {

  //       SimpleEventLoop loop;
  //

  //       // Most code that does I/O needs to be run from within an


  //       // EventLoop, so it can use Promise::then().  So, we need to
  //       // use `evalLater()` to run `getHttp()` inside the event

  //       // loop.
  //       Promise<String> textPromise = loop.evalLater(

  //           []() { return getHttp("http://example.com"); });

  //
  //       // Now we can wait for the promise to complete.

  //       String text = loop.wait(kj::mv(textPromise));


  //       print(text);
  //       return 0;

  //     }



public:
  EventLoop();



  static EventLoop& current();


  // Get the event loop for the current thread.  Throws an exception if no event loop is active.



  template <typename T>

  T wait(Promise<T>&& promise);

  // Run the event loop until the promise is fulfilled, then return its result.  If the promise

  // is rejected, throw an exception.
  //

  // It is possible to call wait() multiple times on the same event loop simultaneously, but you

  // must be very careful about this.  Here's the deal:


  // - If wait() is called from thread A when it is already being executed in thread B, then

  //   thread A will block at least until thread B's call to wait() completes, _even if_ the
  //   promise is fulfilled before that.
  // - If wait() is called recursively from a thread in which wait() is already running, then

  //   the inner wait() will proceed, but the outer wait() obviously cannot return until the inner

  //   wait() completes.
  // - Keep in mind that while wait() is running the event loop, it may be firing events that have

  //   nothing to do with the thing you're actually waiting for.  Avoid holding any mutex locks

  //   when you call wait() as if some other event handler happens to try to take that lock, you

  //   will deadlock.
  //

  // In general, it is only a good idea to use `wait()` in high-level code that has a simple

  // goal, e.g. in the main() function of a program that does one or two specific things and then


  // exits.  On the other hand, `wait()` should be avoided in library code, unless you spawn a


  // private thread and event loop to use it on.  Put another way, `wait()` is useful for quick


  // prototyping but generally bad for "real code".
  //

  // If the promise is rejected, `wait()` throws an exception.  This exception is usually fatal,

  // so if compiled with -fno-exceptions, the process will abort.  You may work around this by

  // using `there()` with an error handler to handle this case.  If your error handler throws a

  // non-fatal exception and then recovers by returning a dummy value, wait() will also throw a

  // non-fatal exception and return the same dummy value.



  Promise<void> yield();

  // Returns a promise which is fulfilled when all work currently in the queue has completed.

  // Note that this doesn't necessarily mean the queue is empty at that point -- if you call

  // `yield()` twice, the promise from the first call will be fulfilled before the one returned

  // by the second call.
  //

  // Note that `yield()` is the only way to add events to the _end_ of the queue.  When a promise

  // is fulfilled and some other promise is waiting on it, the `then` callback for that promise

  // actually goes onto the _beginning_ of the queue, so that related callbacks occur together and

  // splitting a task into finer-grained callbacks does not cause the task to "lose priority"

  // compared to other tasks occurring concurrently.



  template <typename Func>

  auto evalLater(Func&& func) const -> PromiseForResult<Func, void>;

  // Schedule for the given zero-parameter function to be executed in the event loop at some

  // point in the near future.  Returns a Promise for its result -- or, if `func()` itself returns


  // a promise, `evalLater()` returns a Promise for the result of resolving that promise.
  //

  // Example usage:
  //     Promise<int> x = loop.evalLater([]() { return 123; });

  //
  // If the returned promise is destroyed before the callback runs, the callback will be canceled.

  // If the returned promise is destroyed while the callback is running in another thread, the

  // destructor will block until the callback completes.


  //
  // `evalLater()` is largely equivalent to `there()` called on an already-fulfilled

  // `Promise<Void>`.



  template <typename T, typename Func, typename ErrorFunc = _::PropagateException>

  auto there(Promise<T>&& promise, Func&& func,

             ErrorFunc&& errorHandler = _::PropagateException()) const

      -> PromiseForResult<Func, T>;

  // When the given promise is fulfilled, execute `func` on its result inside this `EventLoop`.

  // Returns a promise for the result of `func()` -- or, if `func()` itself returns a promise,

  // `there()` returns a Promise for the result of resolving that promise.


  //
  // If `promise` is broken/rejected (i.e. with an exception), then `errorHandler` is called rather

  // than `func`.  The default error handler just propagates the exception.


  //
  // If the returned promise is destroyed before the callback runs, the callback will be canceled.

  // If the returned promise is destroyed while the callback is running in another thread, the

  // destructor will block until the callback completes.  Additionally, canceling the returned

  // promise will transitively cancel the input `promise`.  Or, if `func()` already ran and


  // returned another promise, then canceling the returned promise transitively cancels that

  // promise.
};

[Stephen Rollyson]

Hi Kenton,

Woo boy, there's a lot of meat there. Looks good upon cursory inspection. The examples are helpful and look to be the same ones from our discussion in the other thread.

How did you guys document the C++ protobufs runtime? This header has a lot of private namespace stuff that could do with being excluded in the user documentation. I'd recommend using Doxygen if you're not already using a documentation generator. With Doxygen, you can provide example snippets of how to use your interface and have each function or class name auto-hyperlink to your detailed documentation. It makes complex interfaces like this much easier to grok, in my opinion.

Stephen.

[Kenton Varda]

Hi Stephen,

Thanks for the suggestion.  I definitely want to do auto-generated reference docs at some point, but it likely won't be with Doxygen.  While I like the concept of Doxygen in theory, and it's better than nothing, I've just never had a good experience with it in practice.  The document style it generates by default is horribly confusing and ugly, and its C++ parser seems full of bugs that cause it to randomly drop or misinterpret symbols.  For protobufs, I used Doxygen but had it output XML and then used scripts to convert that into a reasonably-organized web page, but I had to work around all kinds of bugs and design problems in the XML output.

Instead, I'm hoping to use cldoc:

  http://jessevdk.github.io/cldoc/

I haven't tried it yet, but it looks to be designed correctly -- i.e., based on Clang as the parser, and interpreting doc comments as Markdown -- and it even generates output by default in a style that doesn't make my eyes bleed.  :)

I suspect I will have to tweak it a bit to understand my commenting style where the comments appear after the declaration, but hopefully that won't take much effort.

Of course, I also intend to extend the current overview documentation on the web site to include an introduction to promises, with code samples.  The auto-generated docs would only be for the API reference.

-Kenton

--
You received this message because you are subscribed to the Google Groups "Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send an email to capnproto+...@googlegroups.com.
Visit this group at http://groups.google.com/group/capnproto.

[Stephen Rollyson]

I've definitely seen Doxygen's limitations in my own use of it. I just didn't know whether or not there was some in-house Google documentation generator used to document protobufs independent of Doxygen.

Thanks for the link on the clang-based documentation generator. It definitely seems like a much better approach than Doxygen's "one size fits all" design, especially with the C++98 -> C++11 changes. There's a saying in the Perl community that fits: "Only perl can parse Perl" ("perl" being the interpreter and "Perl" being the language).

Interestingly enough, I had a discussion with Paul Evans, the developer of Net::Async::WebSocket, about design advice for my upcoming WAMP module. He suggested I use futures to contain RPC function return values that haven't resolved yet, so it's looking like my own Perl WAMP module won't be terribly different from the Cap'n Proto RPC implementation. I suppose the biggest difference will be that the WAMP protocol won't let me push promises across the pipe to resolve dependent operations without a round trip.

Stephen.

[Stephen Rollyson]

Just for clarification, there() is essentially a way to provide two async callbacks (success/fail) for the resolution of a promise by way of lambdas, right? If that's the case, it might be clearer to name it something like handleResolution() unless you meant to have a short identifier for daisy-chaining several there() calls.

Stephen.

[Kenton Varda]

On Sat, Sep 14, 2013 at 4:44 AM, Stephen Rollyson <ste...@rollyson.org> wrote:

Just for clarification, there() is essentially a way to provide two async callbacks (success/fail) for the resolution of a promise by way of lambdas, right?

Right.  there() is equivalent to calling then() on its first argument, except that the callbacks run in some other event loop rather than the current thread's event loop.

 

If that's the case, it might be clearer to name it something like handleResolution() unless you meant to have a short identifier for daisy-chaining several there() calls.

Well, "there" and "then" both come from promises APIs that exist in other languages.  I want to match terminology...

 

[Geoffrey Romer]

It seems to me that you might be better off sticking closer to C++ idioms, where possible. In particular, you might consider calling Promise "Future", and PromiseFulfiller "Promise". You might also want to take a look at the proposal for promise pipelining in std::future (http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2013/n3721.pdf). It seems like the more similar your framework is to the eventual standard one, the better, and since the proposal is still pending (I think it's slated for C++17), the influence can potentially go in both directions.

Random side note about the code:

"Note that `wait()` consumes the promise on which it is called, in the sense of move semantics."

Do you maybe want to &&-qualify wait(), so it can only be called on rvalues? seems like

[Kenton Varda]

Hi Geoffrey,

On Sat, Sep 14, 2013 at 3:35 PM, Geoffrey Romer <gro...@google.com> wrote:

It seems to me that you might be better off sticking closer to C++ idioms, where possible. In particular, you might consider calling Promise "Future", and PromiseFulfiller "Promise". You might also want to take a look at the proposal for promise pipelining in std::future (http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2013/n3721.pdf). It seems like the more similar your framework is to the eventual standard one, the better, and since the proposal is still pending (I think it's slated for C++17), the influence can potentially go in both directions.

It's unfortunate that C++11 chose to use the word "promise" to mean something completely different from what the word means in other existing systems.  However, my promises are based on E promises, which are subtly-yet-critically different from C++ futures.  Therefore I'd like to stick to the terminology used in E, which is much older than C++11.

The big difference I see between E promises and C++ futures is that E promises start from the assumption that there is an event loop, and all continuations should occur as independent events on said event loop.  C++ futures start from the assumption that there is no event loop and continuations should be run when get() is called.  The latter is a fairly ad-hoc approach that breaks down when you start writing serious event-driven programs where multiple asynchronous operations must be chained.  It looks like the proposal you reference tries to address this by having a version of then() which takes an `executor` pointer, which could represent an event loop.  However, this seems to be an after-thought rather than the primary focus of the design (whereas in my framework, wait() is the after-thought).

In any case, since I'm taking my design from E, and since other languages like Javascript are now converging on E's terminology, I'd rather match that.  Since promises and promise pipelining are a key piece of Cap'n Proto's design, I'd rather have cross-language consistency in terminology than consistency with unrelated libraries in each language.

BTW, the proposal you reference does't appear to mention pipelining, and I wouldn't expect it to since pipelining is an RPC feature.

Random side note about the code:

"Note that `wait()` consumes the promise on which it is called, in the sense of move semantics."

Do you maybe want to &&-qualify wait(), so it can only be called on rvalues? seems like

Yes, but I'm currently targeting GCC 4.7 and Clang 3.2, neither of which support that.  At some point I will decide it's OK to demand that my users use GCC 4.8 or Clang 3.3, but we're not there yet.

-Kenton

[Jason Paryani]

Hey Kenton,

The Promise API looks really good. I'm excited to see what the RPC API ends up looking like.

I just started wrapping Promises/EventLoop for consumption in the Python library, and I ran into a weird bug with wait():

#include <kj/async.h>

#include <iostream>

using namespace kj;

int main() {

    SimpleEventLoop loop;

    Promise<int> myPromise = loop.evalLater([]() { return 1; });

    std::cout << loop.wait(kj::mv(myPromise)) << std::endl; // This works

    Promise<int> myPromise2 = loop.evalLater([]() { return 2; });

    std::cout << myPromise2.wait() << std::endl; // This throws: what():  src/kj/async.c++:96: requirement not met: expected result != nullptr; No event loop is running on this thread.

    return 0;

}

Is this not a valid way to use wait or is it a bug?

[Kenton Varda]

Hey Jason,

Promise::wait() and Promise::then() can only be called within an EventLoop -- i.e. from an event callback called by the event loop.  If you're not currently in an EventLoop, use EventLoop::wait() and EventLoop::there(), respectively, to indicate explicitly which EventLoop to use.

Actually, I think maybe I should remove Promise::wait() as waiting from inside another event is a really hacky thing to do, and if people really want to do it they can still use EventLoop::current().wait() easily enough.

-Kenton

[Jason Paryani]

Wait, I'm confused now. I'll definitely stop using `wait`, but `then` having to run inside of an event seems strange to me. How would the example you posted in the RPC/Websocket thread work?

void main() {

   EventLoop loop;

   RPCService svc(&loop);

   

   Promise<void> done =

       svc.connect("example.com:1337")

       .then([&]() { return svc.add(1, 2); })

       .then([](int addResult) {

           print("Yay! We've added 1 + 2. Here's the result: ");

           print(addResult);

       }, [](kj::Exception&& err) {

           print("Oh no! We've gotten an error: ");

           print(err);

           exitProgramWithError();

       });

    loop.wait(done);

}

Changing all those `then` calls to `there` seems inelegant to me, and breaks with how I would expect Promises to work (my experience is solely with Javascript Promises).

[Klaim - Joël Lamotte]

My opinion is that if your Promise "then()" implementation can't allow the caller to decide what "executor" will execute the continuation, 
it will be totally unusable in a lot of contexts.

"event loop" are not always unique, for example in my app I have as many "event loop" as I have concurrent systems,
passing work to each others. In a lot of cases, one task is pushed in system A to be executed, but it's continuation have to be executed through
System B. To achieve that I have a somewhat similar implementation to std::future but whith the "then()" implementation which have an overload 

[Klaim - Joël Lamotte]

(sorrry, sent too fast)

...with an overload taking an executor (in my case a callback to use to push the continuation in).

Your promise seem ok for a lot of cases but I don't think it'll be enough for some cases like mine. I'm not sure though, I'll take a deeper look later.​

[Kenton Varda]

On Sun, Sep 15, 2013 at 2:25 PM, Jason Paryani <jpar...@gmail.com> wrote:

Wait, I'm confused now. I'll definitely stop using `wait`, but `then` having to run inside of an event seems strange to me. How would the example you posted in the RPC/Websocket thread work?

My example was not quite correct.  To make sure it runs on the event loop, just wrap it in a runLater():

void main() {

   EventLoop loop;

   RPCService svc(&loop);

   

   Promise<void> done = loop.runLater([&]() {

       return svc.connect("example.com:1337")

         .then([&]() { return svc.add(1, 2); })

         .then([](int addResult) {

             print("Yay! We've added 1 + 2. Here's the result: ");

             print(addResult);

         }, [](kj::Exception&& err) {

             print("Oh no! We've gotten an error: ");

             print(err);

             exitProgramWithError();

         });

       });

    loop.wait(done);

}

 

Changing all those `then` calls to `there` seems inelegant to me, and breaks with how I would expect Promises to work (my experience is solely with Javascript Promises).

This is only an issue at the very top level -- in the main() function, or whatever function actually creates the event loop.  Everything below that point can just assume it's running in an event loop and use there() without a problem.

[Jason Paryani]

This is only an issue at the very top level -- in the main() function, or whatever function actually creates the event loop.  Everything below that point can just assume it's running in an event loop and use there() without a problem.

Thanks, I got it now. I was completely ignoring the fact that you would usually be running your program in an EventLoop. I was a bit too focused on the synchronous use case. As far as the Python API goes, I'll probably have a default EventLoop that Promise member methods will fall back to using if they're not already inside an event, so that you can use the library in a synchronous manner without a lot of effort.

[Andrew Lutomirski]

An issue with this type of system that I've had involves object
lifetime (or, alternatively cancellation). As a simple example (and
sorry for the inevitable syntax errors):

Suppose that a.readSomething returns a Promise of some sort. Then
some code could do:

a.readSomething().then([&](ArrayPtr<char> x) { b.WriteSomething(x); }

Later on, either a or b goes away. This causes a crash (if using
capture-by-reference like I did) or unexpected actions on zombie
objects (if the code captured a shared pointer, say).

I don't have a good general solution to this, but this is why I
usually prefer to have objects that explicitly send events to other
objects (where the set of events is small and the objects know to stop
sending events when they're dead).

Do you have best practices or, even better, some magic to solve this?

--Andy

[Kenton Varda]

On Sun, Sep 15, 2013 at 4:02 PM, Jason Paryani <jpar...@gmail.com> wrote:

Thanks, I got it now. I was completely ignoring the fact that you would usually be running your program in an EventLoop. I was a bit too focused on the synchronous use case. As far as the Python API goes, I'll probably have a default EventLoop that Promise member methods will fall back to using if they're not already inside an event, so that you can use the library in a synchronous manner without a lot of effort.

Yeah, making this work right with Python will be somewhat tricky.

If you think people will want to do things synchronously (noting that this means they can't use pipelining to avoid network round trips, which is a big deal), then the right thing to do would be to run an EventLoop on a background thread which will handle all of the C++ RPC machinery.  Then, each time someone makes a call from Python that would return a Promise in C++, initiate the operation on said background loop, then create a temporary SimpleEventLoop on the stack and use it to wait() on the promise.  This SimpleEventLoop won't actually run any events, it will just block waiting for the background loop.

On the other hand, promise pipelining will be very important for reducing latency, particularly with the kind of object-oriented protocols that one would be likely to design in Cap'n Proto.  So, you will probably want to provide a fully asynchronous mode as well, that actually exposes promises in Python.  In this mode, I think you just need to make people create an event loop and explicitly enter it just like they would in C++.  The event loop really needs to be running even when the program is not actively waiting for an operation, because there may be background tasks that need to occur in order to maintain an open connection, or the application may have exported a capability which needs to service incoming requests.

I guess a third, hybrid option would be to have the event loop on a background thread and allow Python code to choose to use promises on a per-call basis.  Then you rely on the GIL to synchronize continuations, which will allow the event loop to execute callbacks any time the main thread performs a blocking call, even if it isn't a Cap'n Proto call.  Designing around the GIL seems dirty, though.

-Kenton

[Kenton Varda]

On Sun, Sep 15, 2013 at 4:03 PM, Andrew Lutomirski <an...@luto.us> wrote:

An issue with this type of system that I've had involves object
lifetime (or, alternatively cancellation).  As a simple example (and
sorry for the inevitable syntax errors):

...

Do you have best practices or, even better, some magic to solve this?

If you allow a promise to go out-of-scope (i.e. run its destructor) without calling then() or wait() on it (or moving it elsewhere), the underlying operation is canceled, transitively.  This actually makes cancellation very natural in C++:  Just delete the object (and any promises you received from it), and whatever it might be doing in the background stops.  Your code will even tend to be exception-safe without thinking about it.

This also implies that if you ignore the result of then(), the continuation never occurs, and the thing the continuation is waiting on might even be canceled.  This could be a little confusing but my hope is that because you always need to pass something to wait() at the top level of your program in order to actually run the loop, it will be hard for people to accidentally forget to chain their promises correctly.  I think I can also get the compiler to issue a warning if the result is ignored...

-Kenton

[Jason Paryani]

On Sunday, September 15, 2013 4:20:44 PM UTC-7, Kenton Varda wrote:

If you think people will want to do things synchronously (noting that this means they can't use pipelining to avoid network round trips, which is a big deal), then the right thing to do would be to run an EventLoop on a background thread which will handle all of the C++ RPC machinery.  Then, each time someone makes a call from Python that would return a Promise in C++, initiate the operation on said background loop, then create a temporary SimpleEventLoop on the stack and use it to wait() on the promise.  This SimpleEventLoop won't actually run any events, it will just block waiting for the background loop.

Yep, this is what I was going to do for the synchronous case.

 

On the other hand, promise pipelining will be very important for reducing latency, particularly with the kind of object-oriented protocols that one would be likely to design in Cap'n Proto.  So, you will probably want to provide a fully asynchronous mode as well, that actually exposes promises in Python.  In this mode, I think you just need to make people create an event loop and explicitly enter it just like they would in C++.  The event loop really needs to be running even when the program is not actively waiting for an operation, because there may be background tasks that need to occur in order to maintain an open connection, or the application may have exported a capability which needs to service incoming requests.

I understand, for most serious use cases, people really need to be running an event loop. 

 

I guess a third, hybrid option would be to have the event loop on a background thread and allow Python code to choose to use promises on a per-call basis.  Then you rely on the GIL to synchronize continuations, which will allow the event loop to execute callbacks any time the main thread performs a blocking call, even if it isn't a Cap'n Proto call.  Designing around the GIL seems dirty, though.

So what I really want to do is plug into a Python async library (I'm eyeing gevent at the moment, but I do want to make it flexible), and to piggyback on their event loop. That way, IO as well as events from the RPC will all defer to each other when appropriate. 

[Kenton Varda]

On Sun, Sep 15, 2013 at 4:30 PM, Jason Paryani <jpar...@gmail.com> wrote:

So what I really want to do is plug into a Python async library (I'm eyeing gevent at the moment, but I do want to make it flexible), and to piggyback on their event loop. That way, IO as well as events from the RPC will all defer to each other when appropriate. 

It may conceivably be possible to design an EventLoop subclass which can integrate with an existing external EventLoop, e.g. by overriding the sleep() method to simply enter that other loop.  Let me know if any design changes in EventLoop would make this easier.

[Geoffrey Romer]

On Sat, Sep 14, 2013 at 4:25 PM, Kenton Varda <temp...@gmail.com> wrote:

Hi Geoffrey,

On Sat, Sep 14, 2013 at 3:35 PM, Geoffrey Romer <gro...@google.com> wrote:

It seems to me that you might be better off sticking closer to C++ idioms, where possible. In particular, you might consider calling Promise "Future", and PromiseFulfiller "Promise". You might also want to take a look at the proposal for promise pipelining in std::future (http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2013/n3721.pdf). It seems like the more similar your framework is to the eventual standard one, the better, and since the proposal is still pending (I think it's slated for C++17), the influence can potentially go in both directions.

It's unfortunate that C++11 chose to use the word "promise" to mean something completely different from what the word means in other existing systems.  However, my promises are based on E promises, which are subtly-yet-critically different from C++ futures.  Therefore I'd like to stick to the terminology used in E, which is much older than C++11. 

The big difference I see between E promises and C++ futures is that E promises start from the assumption that there is an event loop, and all continuations should occur as independent events on said event loop.  C++ futures start from the assumption that there is no event loop and continuations should be run when get() is called.  The latter is a fairly ad-hoc approach that breaks down when you start writing serious event-driven programs where multiple asynchronous operations must be chained.  It looks like the proposal you reference tries to address this by having a version of then() which takes an `executor` pointer, which could represent an event loop.  However, this seems to be an after-thought rather than the primary focus of the design (whereas in my framework, wait() is the after-thought).

I'm actually a little surprised that you adopted this model of having promises obtain their execution policy implicitly, from the callsite environment. It seems like an instance of the Environment Pattern, which you're on record describing as something that "Well-designed code should avoid" (http://www.object-oriented-security.org/lets-argue/singletons#TOC-This-is-a-compromise-). In fact, it seems even more Singleton-like than the basic Environment pattern, since I assume you can't construct a new EventLoop in a thread where one's already running, so you have to go to unusual lengths to override the parent environment: spin up a new thread, and make appropriate arrangements to communicate with it. The model proposed for std::future seems much more consistent with the philosophy of making parameters explicit (as well as being more general).

As a point of comparison, in the referenced proposal, future::when would by default execute the continuation using the same executor (or policy) as the antecedent future, so you could achieve the same behavior, so long as whoever produces the initial future configures it properly. However, from talking to someone who's been following C++ concurrency more closely than I, it sounds like that aspect of the design may still be up in the air, and option of inheriting the executor from the when() callsite may still be on the table. Practical experience with using this model in Cap'n Proto could conceivably 

 

In any case, since I'm taking my design from E, and since other languages like Javascript are now converging on E's terminology, I'd rather match that.  Since promises and promise pipelining are a key piece of Cap'n Proto's design, I'd rather have cross-language consistency in terminology than consistency with unrelated libraries in each language.

That makes sense, so far as terminology is concerned. Cross-language libraries always have this tension between consistency across languages and consistency with existing conventions in each language, and I think consistent naming across languages is an important goal. However, when it comes to the actual API design, I don't think you can treat std::future as a mere "unrelated library"- it is likely to become the standard way of representing and composing asynchronous computations in C++, so it's worth giving some thought to how the two will interoperate. 

The fact that kj::Promise only supports execution in an EventLoop is what gives me the most pause. How would clients who use other models of concurrency (e.g. cooperative threading and/or a pre-existing event loop framework) interoperate with it? What if they want to compose Cap'n Proto API operations with operations from std::future-based APIs?

The fact that std::future::then integrates error handling into the continuation, whereas kj::Promise::then takes a separate error-handler parameter, could also cause readability problems at callsites, if it's unclear whether a given expression denotes a Promise or a future.

However, I have no practical experience with this sort of concurrency model, so these concerns may be naive or misguided.

 

BTW, the proposal you reference does't appear to mention pipelining, and I wouldn't expect it to since pipelining is an RPC feature.

My mistake; I thought "pipelining" referred to then()-style continuation.

 

Random side note about the code:

"Note that `wait()` consumes the promise on which it is called, in the sense of move semantics."

Do you maybe want to &&-qualify wait(), so it can only be called on rvalues? seems like

Yes, but I'm currently targeting GCC 4.7 and Clang 3.2, neither of which support that.  At some point I will decide it's OK to demand that my users use GCC 4.8 or Clang 3.3, but we're not there yet.

Gotcha. It might make sense to note that in the API comments; it might make the meaning clearer to readers who are familiar with method ref-qualifiers, and it would encourage clients to code as if it were already the case, so they don't have to fix their code later.

[Kenton Varda]

On Mon, Sep 16, 2013 at 12:41 PM, Geoffrey Romer <gro...@google.com> wrote:

On Sat, Sep 14, 2013 at 4:25 PM, Kenton Varda <temp...@gmail.com> wrote:

Hi Geoffrey,

On Sat, Sep 14, 2013 at 3:35 PM, Geoffrey Romer <gro...@google.com> wrote:

It seems to me that you might be better off sticking closer to C++ idioms, where possible. In particular, you might consider calling Promise "Future", and PromiseFulfiller "Promise". You might also want to take a look at the proposal for promise pipelining in std::future (http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2013/n3721.pdf). It seems like the more similar your framework is to the eventual standard one, the better, and since the proposal is still pending (I think it's slated for C++17), the influence can potentially go in both directions.

It's unfortunate that C++11 chose to use the word "promise" to mean something completely different from what the word means in other existing systems.  However, my promises are based on E promises, which are subtly-yet-critically different from C++ futures.  Therefore I'd like to stick to the terminology used in E, which is much older than C++11. 

The big difference I see between E promises and C++ futures is that E promises start from the assumption that there is an event loop, and all continuations should occur as independent events on said event loop.  C++ futures start from the assumption that there is no event loop and continuations should be run when get() is called.  The latter is a fairly ad-hoc approach that breaks down when you start writing serious event-driven programs where multiple asynchronous operations must be chained.  It looks like the proposal you reference tries to address this by having a version of then() which takes an `executor` pointer, which could represent an event loop.  However, this seems to be an after-thought rather than the primary focus of the design (whereas in my framework, wait() is the after-thought).

I'm actually a little surprised that you adopted this model of having promises obtain their execution policy implicitly, from the callsite environment. It seems like an instance of the Environment Pattern, which you're on record describing as something that "Well-designed code should avoid" (http://www.object-oriented-security.org/lets-argue/singletons#TOC-This-is-a-compromise-).

Yes, I thought about that, and was wondering when someone would bring it up.  :)

My feelings on environments are a little bit more nuanced than "always avoid".  There are certain basic resources that are necessary for computation to occur which seem unreasonable to handle in any way other than via the environment pattern.  Here are some examples, in order from least-reasonable to most-reasonable:

- The basic ability to execute code.  Imagine if every time you wanted to call a function, you had to have an instance of the "Cpu" interface and call "cpu.call(function, args)".  (Obviously, this couldn't possibly work in practice, since you'd need to use Cpu::call() to call Cpu::call().)

- The ability to dereference pointers.  What if I needed an explicit reference to an implementation of the "Mmu" interface in order to dereference any pointers?  Certainly this would enable some interesting sandboxing techniques.  (This still probably couldn't work in practice, since how would you dereference the MMU object in order to call it?)

- The heap.  This is something that could actually be an explicit resource in practice.  An argument can be made that any object which wishes to allocate memory ought to ask for an allocator to do it with, and you can do various useful things if this is the case.  However, in practice this seems clearly too burdensome.

- The ability to import other modules.  Think Python here -- what if, in order to import anything, you first needed an "importer" object.  Where would you get it?

I like to think of these things as being features of "the virtual machine".  Every single piece of code that executes is implicitly given a reference to the VM, through which it can perform these basic operations.  All of these operations are naturally contained, such that they shouldn't have external side effects (other than resource usage), so a lot of the problems with singletons don't apply.

I think that in a highly-event-driven system, it makes sense for the event loop to be another such feature of the VM.

 

In fact, it seems even more Singleton-like than the basic Environment pattern, since I assume you can't construct a new EventLoop in a thread where one's already running, so you have to go to unusual lengths to override the parent environment: spin up a new thread, and make appropriate arrangements to communicate with it. The model proposed for std::future seems much more consistent with the philosophy of making parameters explicit (as well as being more general).

Well, actually, you can create another EventLoop in the same thread, and you can call wait() on it to make it run.  The thread's current event loop is then temporary changed to the new loop, and switches back when wait() returns.

But in practice this isn't a good idea anyway, because the thread's existing event loop stalls while the inner one is running.  It rarely makes sense to have multiple event loops in the same thread.

 

As a point of comparison, in the referenced proposal, future::when would by default execute the continuation using the same executor (or policy) as the antecedent future, so you could achieve the same behavior, so long as whoever produces the initial future configures it properly. However, from talking to someone who's been following C++ concurrency more closely than I, it sounds like that aspect of the design may still be up in the air, and option of inheriting the executor from the when() callsite may still be on the table. Practical experience with using this model in Cap'n Proto could conceivably 

I think that inheriting from the callsite is much more desirable.  The caller may not have any idea what thread originally produced the promise they are chaining on.  Meanwhile, it is very likely that the continuation manipulates state shared with its lexical scope -- i.e. shared with then()'s call site.  Therefore, it's probably important to run the continuation on the same thread to avoid concurrent access.  With my approach, things are safe-by-default, though you can still use an explicit EventLoop if you really want the continuation to run on a different thread (and you have arranged the proper synchronization).

Frankly, to me, inheriting from the original promise seems likely to lead to disaster.

In any case, since I'm taking my design from E, and since other languages like Javascript are now converging on E's terminology, I'd rather match that.  Since promises and promise pipelining are a key piece of Cap'n Proto's design, I'd rather have cross-language consistency in terminology than consistency with unrelated libraries in each language.

That makes sense, so far as terminology is concerned. Cross-language libraries always have this tension between consistency across languages and consistency with existing conventions in each language, and I think consistent naming across languages is an important goal. However, when it comes to the actual API design, I don't think you can treat std::future as a mere "unrelated library"- it is likely to become the standard way of representing and composing asynchronous computations in C++, so it's worth giving some thought to how the two will interoperate. 

The fact that kj::Promise only supports execution in an EventLoop is what gives me the most pause. How would clients who use other models of concurrency (e.g. cooperative threading and/or a pre-existing event loop framework) interoperate with it? What if they want to compose Cap'n Proto API operations with operations from std::future-based APIs?

It should be possible to use newAdaptedPromise() to adapt promises to any other asynchronous API.  Similarly it's easy to see how to adapt EventLoop to implement the executor interface.  A glue library should be easy to write.

The fact that std::future::then integrates error handling into the continuation, whereas kj::Promise::then takes a separate error-handler parameter, could also cause readability problems at callsites, if it's unclear whether a given expression denotes a Promise or a future.

Well, the continuation for a Promise takes the promised value as the parameter, whereas the continuation for a std::future takes the original std::future as a parameter.  So, it should be clear enough.

 

Gotcha. It might make sense to note that in the API comments; it might make the meaning clearer to readers who are familiar with method ref-qualifiers, and it would encourage clients to code as if it were already the case, so they don't have to fix their code later.

Agreed.  Will do.

-Kenton

[Dmitry S]

This is super-cool. I am a little hazy on exception handling, and would love to see an example.

E.g. suppose in the example with counting lines, both openFtp() and readAll() could throw an exception (have the returned promise broken). Is there a way to add a single error handler that could catch both exceptions? Where would that handler go?

Thanks!

Dmitry

On Thursday, September 12, 2013 10:15:59 AM UTC-4, Kenton Varda wrote:

[Kenton Varda]

Hi Dmitry,

Each time you call then(), you can optionally specify a second continuation to use on error, which is essentially like a "catch" clause.  If you don't specify one, the exception propagates from the old promise to the new.

So my line-counting example could be modified as follows (new part in bold):

     Promise<Own<File>> file = openFtp("ftp://host/foo/bar");

     Promise<String> content = file.then(

         [](Own<File> file) -> Promise<String> {

           return file.readAll();

         });

     Promise<int> lineCount = content.then(

         [](String text) -> int {

           uint count = 0;

           for (char c: text) count += (c == '\n');

           return count;

         }, [](kj::Exception&& exception) {

           return handleError(exception);

         });

Here, handleError() will be called if openFtp() or readAll() fails, and it can return a different value for the line count (or it can re-throw the exception).

-Kenton

[dsa...@gmail.com]

Very cool! Thank you.

One observation about the example is that promises file and content, which are immediately used with then(), get "consumed" (moved) and aren't usable. That seems a good reason to avoid giving them names. So the following seems a slightly better way to write the line-counting example:

     Promise<int> lineCount =
        openFtp("ftp://host/foo/bar")

        .then([](Own<File> file) -> Promise<String> {

           return file.readAll();

        })

        .then([](String text) -> int {

           uint count = 0;

           for (char c: text) count += (c == '\n');

           return count;

        },
        [](kj::Exception&& exception) {

           return handleError(exception);

        });

[Kenton Varda]

I agree, in practice I would always write it that way.  In the example, I wanted to show the type of each intermediate promise, so I gave them names.