Comments on resumable functions [N4402/N4403]

[Shahms King]

Thanks for publishing the update, it solves a number of the complications with the earlier versions and also clarifies some of my misunderstandings.  In fact, the specific mention of the potential problems with await and non-eventually returning resumable functions led me notice a similar problem with non-suspendable "immediately returning" promise types (such as a hypothetical one for expected<T, E>).

Fundamentally, I think there are 4 different promise types:

1. non-resumable / immediately returning (expected<T, E>)

1. set_result

2. lacks {initial,final}_suspend? (currently unspecified)

2. internally-resumable / eventually returning (future<T>)

1. set_result

3. externally-resumable (generator<T>)

1. yield_value

4. mutually-resumable / eventually returning (async_generator<T>)

1. yield_value

2. set_result

The key point is where it is *safe* to call resume(). You've resolved this my making it illegal to call `await` from within an externally-resumable function (because it is unsafe to call resume() from within such a function) and illegal to call `yield` from within an internally-resumable function (because the promise lacks yield_from(), but it is also unsafe to call resume() from *outside* such a function).

Unfortunately, given the current specification it is either impossible or excessively complicated to safely implement await for a non-resumable type such as `expected<T, E>` due to the possibility of (unexpectedly) suspending execution, e.g.

expected<T, E> foo() {

  T1 t = await UnwrapExpected();

  F f = await ReturnsAFuture();  // Ooops! Unexpected suspend & early return!

  return transform(t); 

}

I think the internal-vs-externally resumable dichotomy is sufficient to justify two separate types: resumable_handle<P>, which represents the entire resumable function; and suspension_handle<P>, which represents a single suspension-point within the function.  From here, we can specialize the handle based on the promise type to ensure resume is only present on the handle(s) which can safely be resumed.  This is slightly complicated by the import/export mechanism, but could be resolved by predicating conversion and export operations on the presence of resume() on the handle, e.g.

struct cancellable_handle {

 // 18.11.2.1 construct/reset

 cancellable_handle() noexcept;

 cancellable_handle(std::nullptr_t) noexcept;

 cancellable_handle& operator=(nullptr_t) noexcept;

 // 18.11.2.3 capacity

 explicit operator bool() const noexcept;

 // {cancellation}

 void destroy() const;

};

template <>

struct resumable_handle<void> : cancellable_handle {

 // 18.11.2.1 construct/reset

 using cancellable_handle::cancellable_handle;

 resumable_handle& operator=(nullptr_t) noexcept;

 // 18.11.2.2 export/import

 static resumable_handle from_address(void* addr) noexcept;

 void* to_address() const noexcept;

 // 18.11.2.4 resumption

 void operator()() const;

 void resume() const;

 // 18.11.2.5 completion check

 bool done() const noexcept;

};

template <>

struct resumable_handle<ExternallyResumable<P>> : resumable_handle<> {

 // 18.11.2.1 construct/reset

 using resumable_handle<>::resumable_handle;

 resumable_handle& operator=(nullptr_t) noexcept;

 // 18.11.2.6 export/import

 static resumable_handle from_promise(Promise*) noexcept;

 Promise& promise() noexcept;

 Promise const& promise() const noexcept;

};

template <>

struct resumable_handle<InternallyResumable<P>> : cancellable_handle {

 ...

};

You end up with a similar implementation for suspension_handle<>.  Additionally, it seems like you could greatly reduce the need for from_promise() by having get_return_object(resumable_handle<P>), rather than the void argument list.  Ultimately, supporting non-suspendable types (optional<T>, expected<T, E>, etc.) may be deemed out of scope, but I still feel that using different types for the resumable_handle and suspension_handle (or await_handle) would allow such a change to be made compatibly at a later date.

Thanks,

--Shahms

[Shahms King]

Hey, Gor, I'm not sure if you've had a chance to see my earlier comments yet or not.  My primary concern is with enabling zero-overhead unwrapping APIs.  Even if it's decided to be not worth the effort to do so with the current proposal, it would be nice to at least enable a future proposal to do so in a backwards-compatible fashion.  I think splitting the handle types accomplishes that, but may be missing some reason for keeping them the same (or at least leaving a path open for converting between the two).

Thanks,

--Shahms

[Gor Nishanov]

Hi Shahms:

Your resumable function classification and identifying the case where we would like to get a compile time error is spot on.

I think in addition to "no initial_suspend / final_suspend" that will mark the resumable function as "can never suspend", we need to also have traits that can explain that awaiting on certain types must never result in suspension.

Say:

is_suspendable_v<T> == true for arbitrary type T, but, for optional and expected you will specialize it to be:

is_suspendable_v<optional<T>> == false.

I am not sure how suggested changes to resumable_handle will help, as resumable_handle is what compiler gives to the library, not, the other way around.

Gor

P.S.

Here a post from vcblog on related matter:

~~~~~~~~~

Yes, it can work with optional and expected. Though to make the experience truly awesome, we would need to introduce some mechanism that can express whether a return object of a resumable function can express expansion of a particular monadic class.

optional<T> can represent either a value or the absence of value

expected<T> can represent either a value or the absence of value with a reason why not

future<T> can represent, please wait, a value or the absence of value with a reason why not

Thus in a resumable function returning a future<T>, it is fine to await on expression of any type that can be represented by the future, thus, you can await on optional, expected, future or some other async construct.

If a resumable function returns expected<T> or optional<T>, awaiting on an expression which type represents value not here yet, should not be possible.

At the moment, I don't have a mechanism in the proposal to deal with that.

I am looking for suggestions and ideas.

I have a half-baked idea that I am not sure I like that looks something like that:

if resumable promise does not contain initial_suspend and final_suspend members, that means that resumable function cannot be suspended.

There also should be a trait, let say, "suspendable<T>" that will default to "true_type", for arbitrary awaitable type, but, for optional and expected will say false.

When the compiler figures out that resumable promise does not support suspension (absence of initial_suspend / final_suspend members), it will not compile your code, if you await on an expression of any type for which suspendable says true.

[Shahms King]

Thanks, Gor!

On Sun, May 17, 2015 at 6:49 PM Gor Nishanov <gorni...@gmail.com> wrote:

Hi Shahms:

Your resumable function classification and identifying the case where we would like to get a compile time error is spot on.

I think in addition to "no initial_suspend / final_suspend" that will mark the resumable function as "can never suspend", we need to also have traits that can explain that awaiting on certain types must never result in suspension.

Say:

is_suspendable_v<T> == true for arbitrary type T, but, for optional and expected you will specialize it to be:

is_suspendable_v<optional<T>> == false.

It seems like this could be covered by the lack of a promise_type for T or the lack of initial/final_suspend on that promise type, but an explicit type trait wouldn't hurt.

 

I am not sure how suggested changes to resumable_handle will help, as resumable_handle is what compiler gives to the library, not, the other way around.

Sure, the compiler gives the handle to the library, but the modifications to the handle type allow the compiler to express valid operations on that handle using the existing type system.

Gor

P.S.

Here a post from vcblog on related matter:

~~~~~~~~~

Yes, it can work with optional and expected. Though to make the experience truly awesome, we would need to introduce some mechanism that can express whether a return object of a resumable function can express expansion of a particular monadic class.

optional<T> can represent either a value or the absence of value

expected<T> can represent either a value or the absence of value with a reason why not

future<T> can represent, please wait, a value or the absence of value with a reason why not

Thus in a resumable function returning a future<T>, it is fine to await on expression of any type that can be represented by the future, thus, you can await on optional, expected, future or some other async construct.

If a resumable function returns expected<T> or optional<T>, awaiting on an expression which type represents value not here yet, should not be possible.

At the moment, I don't have a mechanism in the proposal to deal with that.

I am looking for suggestions and ideas.

This is the problem splitting the handle types aims to solve.  Specifically, calling resume() or to_address() on a non-suspendable promise becomes a compiler error because the handle lacks those members.  In addition to resovling the problem presented by:

optional<T> foo() {

  auto a = await some_future();

  ... 

}

 

It also addresses the (similar) yield problem as the internal handle type will lack those members, while the external handle has them.  I'm envisioning something like:

struct generator_promise_type {

    generator<T> get_return_object(resumable_handle<generator_promise_type> handle) {

  // precondition: &handle.promise() == this

  return generator<T>{handle};

}

  ...

};

Such that rather than having to go through hole of resumable_handle<P>::from_promise(*this), the compiler directly supplies the resumable_handle to the promise's get_return_object().

From what I can tell, this *should* obviate the need for the from_promise() function entirely.  Similarly, it does away with the arbitrary restriction on yield/await, which could go back to the earlier specification as syntactic sugar around await, e.g.

struct _Yielder {

  ...

  void await_suspend(await_handle<P> handle) {

    handle.promise().yield_value(m_value);

  }

  T m_value;

};

template<typename T>

__yield(T&& value) {

  return _Yielder<T>(std::foward<T>(value));

}

yield foo; // becomes effectively await __yield(foo);

That means that calling await on a non-suspendable awaitable is safe from with a non-eventually returning function, e.g.

generator<T> foo() {

  yield await expected_value();  // Either yield if present or exit.

}

I have a half-baked idea that I am not sure I like that looks something like that:

if resumable promise does not contain initial_suspend and final_suspend members, that means that resumable function cannot be suspended.

There also should be a trait, let say, "suspendable<T>" that will default to "true_type", for arbitrary awaitable type, but, for optional and expected will say false.

When the compiler figures out that resumable promise does not support suspension (absence of initial_suspend / final_suspend members), it will not compile your code, if you await on an expression of any type for which suspendable says true.

 I actually like the explicit type trait, I'm just not sure it's necessary if we can give the handle different members such that only valid operations are allowed on it.

Thanks again,

--Shahms

--

---
You received this message because you are subscribed to the Google Groups "ISO C++ Standard - Future Proposals" group.
To unsubscribe from this group and stop receiving emails from it, send an email to std-proposal...@isocpp.org.
To post to this group, send email to std-pr...@isocpp.org.
Visit this group at http://groups.google.com/a/isocpp.org/group/std-proposals/.