std::future::fetch() to provide lock free accesses in callbacks

[rmn...@gmail.com]

I recently stumbled upon a post that talked about how futures are slow because get() blocks (https://isocpp.org/blog/2016/07/cppcon-2015-completion-t-improving-the-future-t-with-monads-travis-gockel) even when you don't need to block.  For example in a callback attached with .then()

auto future = async(...);

future.then([](std::future<int> future) {

    future.get(); // unnecessary locking

});

A lot of future libraries out there have a different versions of then to avoid this problem (for example Facebook's futures and in the aforementioned post).  Why not have a version of get() that does no locking or atomic loads?  And assume that the user will only call this within a callback attached with .then()?

auto future = async(...);

future.then([](std::future<int> future) {

    future.fetch(); // no unnecessary locking

});

[Sergey Vidyuk]

вторник, 16 мая 2017 г., 4:08:51 UTC+7 пользователь rmn...@gmail.com написал:

I don't think that introducing unsafe function fo access to a potentially uninitialized value in a shared state is a good design for safe high level synchronization primitive. Right now I'm playing with my own Concurrency TS future::then implementation and trying to solve the same performance issue you are talking in a quite a different way.

The idea is the following:
 * future keeps shared_ptr to shared_state
 * shared_state is an interface with two implementations: ready (no thread-synchronization performed in any function) and async (with synchronization inside)
 * async shared state has optional<read_shared_state> as a member so when shared state is ready then shared_ptr<ready_shared_state> can be constructed from shared_ptr<async_shared_state> with aliasing constructor.
 * future passed to continuation holds pointer to ready_shared_state and all of the operations on it are guarantied to perform no threads-synchronization operations.

Sergey Vidyuk

[Giovanni Piero Deretta]

On Monday, May 15, 2017 at 10:08:51 PM UTC+1, rmn...@gmail.com wrote:

I recently stumbled upon a post that talked about how futures are slow because get() blocks (https://isocpp.org/blog/2016/07/cppcon-2015-completion-t-improving-the-future-t-with-monads-travis-gockel) even when you don't need to block.  For example in a callback attached with .then()

auto future = async(...);

future.then([](std::future<int> future) {

    future.get(); // unnecessary locking

});

A lot of future libraries out there have a different versions of then to avoid this problem (for example Facebook's futures and in the aforementioned post).  Why not have a version of get() that does no locking or atomic loads?  And assume that the user will only call this within a callback attached with .then()?

There is no reason for get to issue anything more expensive than a load acquire, (or even consume) + cmp + (very predictable) branch  which can be very cheap. In fact there is no reason for locking to be used at all (including in setting up and dispatching 'then' callback) in an std future implementation outside of shared_future. It can be completely lock free as long as only 'then' is used to wait for values.

[Arthur O'Dwyer]

I believe the type-safe way to design this API has already been designed and used in practice, e.g. in Folly.

    auto future = async(...);

    future.then([](int result) {

        use(result);

    }).on_error([](std::exception_ptr exc) {

        handle(exc);

    });

That is, .then() allows "unwrapped" callbacks, and if you need to handle the exceptional case, then you add an .on_error() handler.  The above code is 100% equivalent in semantics to

    auto future = async(...);

    future.then([](std::future<int> maybe_result) {

        try {

            int result = maybe_result.get();

            use(result);

        } catch (...) {

            std::exception_ptr exc = std::current_exception();

            handle(exc);

        }

    });

except that the former version is much much faster, because it doesn't have to do any atomic operations and it doesn't have to throw any exceptions on the exceptional path.  (I'm fairly sure that throwing that exception is going to be much much slower than the couple of atomic exchanges necessary to lock and unlock the shared state of the future.)

I've also seen "unwrapped .then()" referred to as ".on_value()", which has nice symmetry and also makes the metaprogramming much easier.

The .on_value()/.then() and .on_error() functions don't seem to be part of std::experimental::future, but of course they should be. No actual codebase is going to implement .then() without a fast .on_value()/.on_error() equivalent.

–Arthur