ExecutionContext preparation

524 views
Skip to first unread message

Rich Dougherty

unread,
Jun 6, 2013, 6:37:04 AM6/6/13
to scala...@googlegroups.com
I've recently been refactoring Play's iteratees to use implicit ExecutionContexts and I've found the need to prepare ExecutionContexts before they are used has been quite awkward when writing asynchronous code. I know ExecutionContext preparation is simple in concept—just call the "prepare" method before use—but I've actually found it to be quite error-prone in practice!

I'd like to suggest a change to the semantics of ExecutionContext preparation, but first I'll talk a bit more about the problems I've found.

-

As I understand it, this is the way ExecutionContext preparation works at the moment:
- every method that accepts an implicit ExecutionContext must call the prepare method on the ExecutionContext it receives as an argument
- the prepare method must be called in the caller's thread, so that any thread local state can be gathered by the ExecutionContext if necessary
- only the prepared ExecutionContext should be used for execution
- the (possibly unprepared) ExecutionContext originally supplied to the method should not be used for execution

(Please correct me if any of my assumptions are incorrect.)

While refactoring Play's iteratees, despite being careful, I've made mistakes with all of those requirements:
- I've forgotten to prepare the ExecutionContext.
- I've prepared the ExecutionContext in the wrong thread, e.g. by accidentally preparing inside a call-by-name parameter that runs on a different thread.
- I've prepared the ExecutionContext properly, but then used the unprepared one accidentally (easy to do since the unprepared ExecutionContext is implicit in the method's scope).

Errors are difficult to spot, because prepared and unprepared ExecutionContexts both have the same type and so are indistinguishable to both compiler and programmer. The unprepared ExecutionContext, which should not be used, is also implicit within the method's scope, so it's incredibly easy to use accidentally.

Also, when errors in preparation do occur, they often go undected. This is because most ExecutionContexts don't need preparation at all and function fine without preparation. For this type of ExecutionContext, errors with preparation won't lead to any visible errors in program execution. Only more "exotic" ExecutionContexts, such as those that propagate thread local information, will actually have any chance of showing errors when they're used incorrectly. Unfortunately these ExecutionContext may only occur once the async code in question has shipped and is being used in production!

Finally, when errors do occur, they will likely be subtle and it possibly quite difficult to debug. We may be able to deduce that a thread local variable is not being propagated. But where exactly in a long chain of asynchronous calls did we forget to prepare the ExecutionContext? Did we prepare it on the wrong thread? Or perhaps we prepared it but then accidentally used the unprepared ExecutionContext instead?

In the Play iteratee library I've tried to write tests to ensure that preparation happens properly. I test that preparation occurs, that preparation happens in the right thread, and that only the correctly prepared ExecutionContexts are actually used for executing code. Every asynchronous method needs tests for ExecutionContext preparation. Despite this effort, I suspect there are probably still some bugs related to ExecutionContext prepration in the library…

-

So, I'd like to suggest a simpler approach to ExecutionContext preparation.

I propose that ExecutionContexts should be prepared when they are constructed. Once constructed, no further preparation is ever needed. The prepare() method can be deprecated.

To me this seems to me like quite a nice idea. It makes working with ExecutionContexts much, much easier, since if you have an ExecutionContext then you can work with it without needing to worry about whether it is prepared or unprepared. Since the prepare method doesn't need to be called, it reduces the amount of code needed in asynchronous code, and it reduces the amount of code that can have bugs.

Most ExecutionContexts, i.e. those that don't don't need any preparation at all, would obviously be unaffected by this change. Like now, must users would be able to import an implicit value and use that as their ExecutionContext when writing asynchronous code.

But what about ExecutionContexts that do need preparation? If the ExecutionContext is provided explicitly then it can be obtained, like any other object, by constructing it or by calling a method. Most users will probably prefer to use an implicit import to provide their ExecutionContext. Preparation can still occur in this case, by importing an implicit method rather than an implicit value. Methods that require an implicit ExecutionContext will be provided one by the compiler. The compiler will call the implicit method to provide the argument as needed. The method will have the opportunity to do any preparation that is needed. Another benefit is that the method will automatically be called in the caller thread, which is the correct thread for preparation—another common source of errors avoided.

So I can see that there are some big benefits to changing the preparation semantics, and no major downsides. Or maybe I've missed something? :)

Benefits:
- user code looks the same as before—import an implicit value, now either a val or a def
- async library code is simplified and has have fewer bugs
- less code to write and execute

Downsides:
- ?

-

I'd like to hear other people's thoughts on this proposal. I feel like my suggestion is so simple that it must have been considered at some time in the past and discarded for some reason. If there's something obvious I'm missing, I'm ready to be educated!

I think it would be nice for Play if we could stop preparing ExecutionContexts within every asynchronous method. We'd be able to remove a lot of code from the iteratee library that is solely devoted to managing and tracking ExecutionContext preparation. It would probably also fix a few bugs. I think some simplification around ExecutionContext preparation would be of benefit to other people writing asynchronous code too.

Cheers

James Roper

unread,
Jun 6, 2013, 7:13:33 PM6/6/13
to scala...@googlegroups.com
This is slightly relevant.  Java 8's CompletableFuture (equivalent to Scala Promise/Future rolled into one) takes a juc.Executor, which has no prepare method.  On Play Framework, our approach to handling this (this is in the Java API where thread locals and context classloaders are important) willl be to write an interface, probably called ExecutionContext, with a method called current(), that will return a prepared Executor.

If Java and Scala both use the same approach, of having something else that prepares the executor which is invoked by the caller, then this will make interoperability between Java and Scala Future APIs much simpler (they will be able to very easily just wrap each other).

√iktor Ҡlang

unread,
Jun 28, 2013, 6:56:48 AM6/28/13
to <scala-sips@googlegroups.com>
prepare() was introduced to be able to capture specific "context locals" that could then be reinstated on execute of the submitted runnables.

I don't see how your proposal could work, could you give an example with Futures for instance?

Cheers,


--
You received this message because you are subscribed to the Google Groups "scala-sips" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scala-sips+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 



--
Viktor Klang
Director of Engineering

Twitter: @viktorklang

Rich Dougherty

unread,
Jun 28, 2013, 7:39:23 AM6/28/13
to scala...@googlegroups.com
With this proposal, context locals could still be captured.

User code would remain unchanged. Library code would change, but would be simpler and—I think—less error-prone.

Here's some illustrative code (not tested).

User code - stays the same

import com.myproject.ExecutionContexts.capturingContext

myAsync { ... code using context captured by ExecutionContext ... } // takes implicit ExecutionContext

Supplying an ExecutionContext - current situation

Provides an implicit val that is threaded to Future.apply. The actual ExecutionContext provided by the implicit should never be used to execute anything. It's actually an unusable object that you must call prepare on (in the correct thread) to get a usable object. Unfortunately the usable/unusable objects have the same type which can confuse both the compiler and the programmer.

object ExecutionContexts {
  implicit val capturingContext = new ExecutionContext {
    def run(r: Runnable) = throw new IllegalStateException("Must prepare first")
    override def prepare() = new ExecutionContext {
      ... logic to capture context goes here
      def run(r: Runnable) = ... run runnable in captured context ...
      override def prepare() = this
    }
  }
}

Using an ExecutionContext - current situation

Every method receiving an implicit ExecutionContext needs to prepare it, then use the prepared one. Better not forget to prepare! Even if you remember to prepare properly, it's really easy to accidentally use the unprepared one, since it's in scope as an implicit and has the same type as a prepared one.

def myAsync(f: => Unit)(implicit ec: ExecutionContext) {
  implicit val prepared = ec.prepare() // make implicit to conflict with unprepared implicit arg
  Future { f }(prepared) // need to explicitly supply due to implicit conflict
}

Supplying an ExecutionContext - proposed change

Instead of an implicit val, provides an implicit def that is threaded to Future.apply. The def, when called, gets the local context and creates an ExecutionContext that captures that info. The ExecutionContext is immediately usable and doesn't need further preparation. The def will be called just before calling a method that needs an ExecutionContext, which is automatically the correct place for capturing locals.

Notice the code is simpler.

object ExecutionContexts {
  implicit def capturingContext = new ExecutionContext {
    ... logic to capture context goes here
    def run(r: Runnable) = ... run runnable in captured context ...
  }
}

Using an ExecutionContext - proposed change

Since the ExecutionContext is "prepared" when it is constructed, no further preparation is ever needed. Any ExecutionContext can be used without further work.

Notice the code is simpler here too, and cannot be coded incorrectly. Implicits work properly within the method too so the code looks cleaner.

def myAsync(f: => Unit)(implicit ec: ExecutionContext) {
  Future { f }
}

√iktor Ҡlang

unread,
Jun 28, 2013, 7:57:19 AM6/28/13
to <scala-sips@googlegroups.com>
so: Future(1).map(_.toString).filter(_.length > 0).foreach println // creates 4 new ExecutionContexts?

Roland Kuhn

unread,
Jun 28, 2013, 8:16:39 AM6/28/13
to scala...@googlegroups.com
Hi Rich,

what you are proposing is first of all a change in semantics—the implementation-driven motivation should come second. The precise change is that we currently prepare the ExecutionContext where it crosses the boundary from user-code to library-code, whereas after the change it would be prepared when coming into existence (or: crossing the boundary from library-code to user-code). This would make a visible difference if user-code would for example transport an ExecutionContext across threads by some means, and the consequences of the change could be non-obvious to the user.

In order to answer your question of whether this is a good idea or not we should first determine what the users’ gain is and then assess the technical aspects, because if the users do not gain anything then the reduction in library complexity would have to be rather large to justify the change—put differently we need to be able to sell the change to our users.

My gut feeling is that the user expects ThreadLocals to be captured at the place where a Future is created or transformed, and the invocation of the implicit ExecutionContext factory is not visible in the code, so I’d think that intuitively the magic should happen in—say—Future.flatMap. Your initial motivation was to remove the pitfalls for implementors of flatMap and friends, and that could well be addressed by only adding a subtype PreparedExecutionContext which offers the execute() method, forcing the implementation of flatMap to first call prepare() and then execute(). The subtype relationship allows reusing and re-preparing an ExecutionContext, which from my point of view is a desirable property. The default implementation will still be as efficient as today (i.e. returning `this`). Migration to this scheme would be possible via deprecation of ExecutionContext.execute().

What do you think?

Regards,

Roland


Dr. Roland Kuhn
Akka Tech Lead
Typesafe – Reactive apps on the JVM.
twitter: @rolandkuhn


Rich Dougherty

unread,
Jun 28, 2013, 8:56:49 AM6/28/13
to scala...@googlegroups.com
On Fri, Jun 28, 2013 at 11:57 PM, √iktor Ҡlang <viktor...@gmail.com> wrote:
so: Future(1).map(_.toString).filter(_.length > 0).foreach println // creates 4 new ExecutionContexts?

The current and proposed code would both create 4 ExecutionContexts with the above code. The current code would create these contexts when prepare() is called inside each method. The proposed code would create it just before invocation of each method.

If an ExecutionContext doesn't need any preparation then the current and proposed code can both use a single object across all method invocations.

Supplying an ExecutionContext that doesn't need preparation - current situation

object ExecutionContexts {
  implicit val staticContext = new ExecutionContext {
    def run(r: Runnable) = ... run runnable ...
    override def prepare() = this
  }
}

Supplying an ExecutionContext that doesn't need preparation - proposed situation

object ExecutionContexts {
  implicit val staticContext = new ExecutionContext {
    def run(r: Runnable) = ... run runnable ...
  }
}

√iktor Ҡlang

unread,
Jun 28, 2013, 9:00:02 AM6/28/13
to <scala-sips@googlegroups.com>
So what happens when someone writes this:

implicit val ec = getSomeEC()

Future(1).map(_.toString).filter(_.length > 0).foreach println


--
You received this message because you are subscribed to the Google Groups "scala-sips" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scala-sips+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

Rich Dougherty

unread,
Jun 28, 2013, 10:11:14 AM6/28/13
to scala...@googlegroups.com
On Sat, Jun 29, 2013 at 12:16 AM, Roland Kuhn <goo...@rkuhn.info> wrote:
what you are proposing is first of all a change in semantics—the implementation-driven motivation should come second. The precise change is that we currently prepare the ExecutionContext where it crosses the boundary from user-code to library-code, whereas after the change it would be prepared when coming into existence (or: crossing the boundary from library-code to user-code). This would make a visible difference if user-code would for example transport an ExecutionContext across threads by some means, and the consequences of the change could be non-obvious to the user.

In order to answer your question of whether this is a good idea or not we should first determine what the users’ gain is and then assess the technical aspects, because if the users do not gain anything then the reduction in library complexity would have to be rather large to justify the change—put differently we need to be able to sell the change to our users.

Yes, absolutely. Possibly there isn't a good way to get from where we are now to somewhere (arguably) better without inconveniencing users too much. I don't have a concrete migration plan yet, but I'm interested in exploring the idea that prepare() might not be ideal, and maybe planting the seed of an alternative. Or maybe I'll just learn how prepare() is awesome and does stuff in the best possible way! :)

The current semantics is IMO pretty bad for library implementors (like myself, recently, in Play). The main problem is that the semantics makes it hard to avoid bugs in libraries. In Play I've forgotten to prepare contexts, prepared contexts in the wrong thread accidentally and often prepared a context but then used the unprepared context by mistake. Each of these issues was discovered only through a lot of testing. As I wrote these tests and fixed these bugs I couldn't help but think about how a few changes to semantics might be able to eliminate all of these bugs trivially.

As an example, until Scala 2.10.2, the widely used Future.apply method didn't prepare its ExecutionContext. This is one of the most commonly used concurrency methods in Scala, but it had a preparation bug. I don't want to single out anyone for blame for this bug; I just want to illustrate how easy it is for ExecutionContext preparation to go wrong. It can happen in even the most widely used and thoroughly reviewed code.

These kind of bugs will surface as hard-to-debug concurrency issues, perhaps when ThreadLocals disappear partway through a chain of async operations. Even if users are rightly oblivious to the pain of library implementors, then I think they will still care about bugs in libraries.

My gut feeling is that the user expects ThreadLocals to be captured at the place where a Future is created or transformed, and the invocation of the implicit ExecutionContext factory is not visible in the code, so I’d think that intuitively the magic should happen in—say—Future.flatMap.

There's a trade-off between implicit/automatic and explicit/manual. From a user point of view I'd argue they both happen at the same place—either just before invocation or just inside the method immediately after invocation. The manual option risks preparation being forgotten or being done incorrectly. The automatic option is maybe more confusing. Or maybe it's less confusing? I don't know if there's a clear answer on the confusion/intuition question since we're all coming from the point of view of how things happen now. (Of course how things happen now is perfectly relevant to the question of how we might do migration.)
 
Your initial motivation was to remove the pitfalls for implementors of flatMap and friends, and that could well be addressed by only adding a subtype PreparedExecutionContext which offers the execute() method, forcing the implementation of flatMap to first call prepare() and then execute(). The subtype relationship allows reusing and re-preparing an ExecutionContext, which from my point of view is a desirable property. The default implementation will still be as efficient as today (i.e. returning `this`). Migration to this scheme would be possible via deprecation of ExecutionContext.execute().

What do you think?

I think distinguishing prepared/unprepared contexts by type is definitely worth considering. It would be great if we can get some compiler support to prevent some of these bugs. Do you think your proposal would help out in a method like the example one I posted earlier (see below)? The risks are: forgetting to prepare, preparing in the wrong thread (luckily can't happen below) and accidentally using the unprepared context.

def myAsync(f: => Unit)(implicit ec: ExecutionContext) {
  implicit val prepared = ec.prepare() // make implicit to conflict with unprepared implicit arg
  Future { f }(prepared) // need to explicitly supply due to implicit conflict
}

Thanks a lot for your excellent comments and ideas.

Cheers
Rich

Rich Dougherty

unread,
Jun 28, 2013, 10:29:26 AM6/28/13
to scala...@googlegroups.com
On Sat, Jun 29, 2013 at 1:00 AM, √iktor Ҡlang <viktor...@gmail.com> wrote:
So what happens when someone writes this:

implicit val ec = getSomeEC()

Future(1).map(_.toString).filter(_.length > 0).foreach println

The "preparation", if any, would happen at the point getSomeEC() is called. It is different than the current behaviour, where preparation happens at every method call.

I quite like this behaviour. It is simple and easy to understand. If you want a fixed value, create an object and store it in a val. If you want to delay construction for some reason, use a function or method. These are just the usual rules for creating objects. :)

By the way, I realise that it would be very confusing if we changed ExecutionContext semantics in a way that wasn't perfectly compatible for our users. I'm interpreting your question as asking about what my ideal API would be, rather than as a detailed question about how we might migrate to that different API in practice.
Reply all
Reply to author
Forward
0 new messages