Best strategy to pass the ExecutionContext for a library

[Christian Kaps]

Hi,

currently in the Silhouette authentication library, we have hard-coded the global Play Framework ExecutionContext all over the code. Now we would like to remove this hard-coded imports and allow the user to specify its own ExecutionContext. But we are struggling with the best approach. Currently we have a pull request which adds an implicit execution context to every method that returns a Future. But we are not sure if this is the right approach. Can someone give us an advice how to choose the best strategy? Any help would be highly appreciated!

Best regards,

Christian

[Viktor Klang]

Hi Christian,

Who is responsible for the execution of the logic: the caller or the creator if the context? (i.e. pass into constructor or pass into each method)

--
You received this message because you are subscribed to the Google Groups "scala-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scala-user+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Cheers,

√

[Christian Kaps]

Hi Victor,

Thanks for your answer. I'm not sure If I understand you correct?

In the lib there is code which is implemented by us and which gets called by the user and there is code which must be implemented by the user and which gets called by the lib.

Best regards,

Christian

[Viktor Klang]

Hi Christian,

On Mon, May 18, 2015 at 5:30 PM, Christian Kaps <kaps.ch...@gmail.com> wrote:

Hi Victor,

Thanks for your answer. I'm not sure If I understand you correct?

In the lib there is code which is implemented by us and which gets called by the user and there is code which must be implemented by the user and which gets called by the lib.

Yes, but is it the caller that should control the execution, or the thing that creates the thing that the caller calls?

ex.

val c = new C(executionContext) // <-- the "person" creating C control the execution

val future = c.doFoo() // <-- look no ExecutionContext provided by the caller, since it was provided in the constructor

or

val c = new C() // <--- the "person" creating C does not specify the execution

val future = c.doFoo(executionContext) // <-- the caller must to specify the execution for each invocation

Does that make more sense?

It's about division of responsibility. I.e. who should control the execution at which point?

 

Best regards,

Christian

Am Montag, 18. Mai 2015 16:55:26 UTC+2 schrieb √iktor Klang:

Hi Christian,

Who is responsible for the execution of the logic: the caller or the creator if the context? (i.e. pass into constructor or pass into each method)

On Mon, May 18, 2015 at 2:58 PM, Christian Kaps <kaps.ch...@gmail.com> wrote:

Hi,

currently in the Silhouette authentication library, we have hard-coded the global Play Framework ExecutionContext all over the code. Now we would like to remove this hard-coded imports and allow the user to specify its own ExecutionContext. But we are struggling with the best approach. Currently we have a pull request which adds an implicit execution context to every method that returns a Future. But we are not sure if this is the right approach. Can someone give us an advice how to choose the best strategy? Any help would be highly appreciated!

Best regards,

Christian

--
You received this message because you are subscribed to the Google Groups "scala-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scala-user+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Cheers,

√

--
You received this message because you are subscribed to the Google Groups "scala-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scala-user+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Cheers,

√

[Christian Kaps]

Hi Victor,

> It's about division of responsibility. I.e. who should control the execution at which point?

I think that's the point we struggle with! Is there a general rule who should control the execution at which point?

I hope that doesn't sound dumb (o;

Thanks for your patience,

Christian

[Viktor Klang]

On Mon, May 18, 2015 at 7:30 PM, Christian Kaps <kaps.ch...@gmail.com> wrote:

Hi Victor,

> It's about division of responsibility. I.e. who should control the execution at which point?

I think that's the point we struggle with! Is there a general rule who should control the execution at which point?

I hope that doesn't sound dumb (o;

Not dumb at all!

All I'm saying is that the provider of the ExecutionContext is the one which will control the execution, I think a good exercise is: Who should control the execution, the person who creates the thing with the methods or the one calling the methods?

In other words, what is using the EC and what for? Who should be in control of that?
 

--

Cheers,

√

[Paul Keeble]

I am not convinced this is really the right question. Generally the need to provide an implicit EC in the API is kind of an anti pattern because its a leak of implementation detail, its about how the algorithm works not what its meant to do. A library might very well never have concurrency primitives in its API definition and yet be exposing the need for an execution context either via constructor or individual methods. But from the application using the library perspective the few cases that need to pass in custom ECs really need to and the library might be worthless if that isn't possible.

There is a third option, which is that you import the Default or your own EC and never expose it to the outside API at all. That to me is the more important question and its a very real option as well. Its really hard to choose between the ways in which EC is injected because it requires you to predict how the library will be used and how application logic will call construction and library methods and what EC is available at these points. If you put it in the constructor and your object lives for the life of the application then the application has to have that ready before your library is called. However putting it into the call sites means that dependency is required in all the classes that use the library, but it can be binded to later and dynamically. The flexibility hurts on the usability of the API and the amount of extrinsic pain your cause the users of your library.

I think the topic is a bit more complicated and nuanced and there is a real issue of API design as I definitely see a third option (hide it completely) and why you would want to do that. The constant pollution of Scala library APIs with ExecutionContext's I find irritating, at method call site more than constructor based. It forces a lot of external dependency outside of the library and forces you to program in a very particular way. Its not an easy choice what to do and the way a lot of library's do this (spray can!) I don't think is all that usable.

PK

[Yann Simon]

My 2 cents:

- When the library is completely asynchronous and can work with any EC, I'd expose it via constructor.

- When some method are kinda blocking, I'd expose the EC at method definition.

But I agree with Paul, it is really polluting the API.
Especially when I know that the library itself is completely asynchronous, I'd like to have a way to say "choose whatever EC you want, I do not care".

[Viktor Klang]

Hey Paul,

apologies for the late reply,

On Mon, May 18, 2015 at 11:53 PM, Paul Keeble <paulr...@gmail.com> wrote:

I am not convinced this is really the right question. 

Generally the need to provide an implicit EC in the API is kind of an anti pattern because its a leak of implementation detail, its about how the algorithm works not what its meant to do.

I disagree. Think of it more as a "capability".

 

A library might very well never have concurrency primitives in its API definition and yet be exposing the need for an execution context either via constructor or individual methods.

If a library needs a means of concurrent execution, then this is a dependency and having that passed in instead of being global or non-customizable seems like a good thing?

 

But from the application using the library perspective the few cases that need to pass in custom ECs really need to and the library might be worthless if that isn't possible.

I'm not sure I follow what you mean, could you elaborate?

 

There is a third option, which is that you import the Default or your own EC and never expose it to the outside API at all. That to me is the more important question and its a very real option as well.

TBH I am more and more convinced that the "global" EC is the antipattern and have been considering suggesting to deprecate it.

Because it means that it becomes impossible for someone from the outside to bulkhead the execution of one library from the other.

 

Its really hard to choose between the ways in which EC is injected because it requires you to predict how the library will be used and how application logic will call construction and library methods and what EC is available at these points.

IMO that is not how it is supposed to be used, but the questions I outlined before. It's about responsibilities and capabilities.

 

If you put it in the constructor and your object lives for the life of the application then the application has to have that ready before your library is called. However putting it into the call sites means that dependency is required in all the classes that use the library, but it can be binded to later and dynamically. The flexibility hurts on the usability of the API and the amount of extrinsic pain your cause the users of your library.

I think the topic is a bit more complicated and nuanced and there is a real issue of API design as I definitely see a third option (hide it completely) and why you would want to do that. The constant pollution of Scala library APIs with ExecutionContext's I find irritating, at method call site more than constructor based. It forces a lot of external dependency outside of the library and forces you to program in a very particular way. Its not an easy choice what to do and the way a lot of library's do this (spray can!) I don't think is all that usable.

We're considering adding a Task-style API to complement s.c.Future in an upcoming Scala release, but keep in mind that they are complementary—a Task is about what to do, a Future is a placeholder for a value.

--

Cheers,

√

[Viktor Klang]

On Tue, May 19, 2015 at 7:51 AM, Yann Simon <yann.s...@gmail.com> wrote:

My 2 cents:

- When the library is completely asynchronous and can work with any EC, I'd expose it via constructor.

- When some method are kinda blocking, I'd expose the EC at method definition.

But I agree with Paul, it is really polluting the API.
Especially when I know that the library itself is completely asynchronous, I'd like to have a way to say "choose whatever EC you want, I do not care".

If you are a consumer of a library and you're not a library yourself (as in, you shouldn't be making the decision for the end user) then by all means just import EC.Implicits.global and be done with it.

The reason for EC.Implicits.global not being the default value of all `implicit ec: ExecutionContext` parameters is because it causes very hard to find issue where things aren't executing where you expect it to. AFAIR Scalaz has had that issue in the past.

--

Cheers,

√

[Naftoli Gugenheim]

In a way it boils down to a problem with implicits in general. The same issue could occur with other types as well.
Actually I think Haoyi had something that will fill in the implicit parameters for you.

But what exactly do you mean by "polluting the API"? Is it conceptually wrong to say that the code is parameterized by it? Is it the repetitive typing? Line noise reading the code? The API docs? The autocomplete choices?
(I guess the answer well be multiple, but which ones?)

As an aside, as far as asynchronous code, IIUC scalaz Tasks don't have this problem, since they don't run until you start them explicitly, and so that is the point at which you need to specify the execution characteristics, not when you're creating or composing them.

[Lance Gatlin]

So this issue comes up just about every day at work. Honestly, I hear what folks are saying about implicits, but I just don't care much for how things "should be" in regards to implicits. Much more interested in creating things that are friendly for users (i.e. other coders) of my code (which includes future-me). 

I have three main ways I deal with implicits:

Method #1: Inject something fixed

1a) I never do this in production code (for tests though, I do this all the time!)

1b) I always want my code to give the ultimate user the greatest flexibility in how to handle implicits. Impossible to predict DI method/framework in lower level code.

1c) This can also make injecting mocks for testing impossible (e.g. import ExecutionContext.Implicits._ can't be mocked since its statically bound by import)

Method #2: Propagate on method signature

2a) This provides the most flexibility to callers

2b) But the tradeoff is that the method is ugly, cludgy and burdens callers with handling this same decision (they now have to answer: how do I handle this implicit?)

Method #3: Inject as a constructor implicit parameter so that all declared methods in the class can use it in the implementation

3a) Less flexible for callers than method #2 since the implicit is bound when constructing the object

3b) Requires a class

3c) My preferred method since it reduces caller's level of mental effort

Method #4: Bind as an implicit def in a base trait and let derived classes figure out how to implement

4a) Alternative for method #3 for when I am writing mix-ins that must be traits but require an implicit I would like to handle using  method #3

 

My rules for deciding which method to use:

R1) If I'm writing library code that my expected user will consume per-method, I use M#2 (implicits as a parameter on the function):

implicit class PimpMyAtomicReference[A](val self: AtomicReference[A]) extends AnyVal {
  import self._

  def asyncCasSet[B](
    f: A => Future[A]
  )(implicit
    ec:ExecutionContext
  ) : Future[CasSet[A]] = {
    def loop() : Future[CasSet[A]] = {
      val current = get
      for {
        next <- f(current)
        result <- {
          if (compareAndSet(current, next) == false) {
            loop()
          } else {
            CasSet(current, next).future
          }
        }
      } yield result
    }
    loop()
  }
}

R2) If I'm writing a component that I expect a user to construct to interact with (e.g. a service), I use M#3 (constructor injection):

trait PersonService extends Contract {
  import PersonService._

  // No constructor code or initialized vals

  /**
   * Create a new person. Fails if id exists.
   * @param id id of person
   * @param name name of person
   * @param age age of person
   * @return if successful unit
   */
  def create(id: Int, name: String, age: Int) : Future[PassOrFail]

class PersonServiceImpl()(implicit ec: ExecutionContext) extends PersonService {
  
  override def create(id: Int, name: String, age: Int) = ???

R2a) Since the base trait doesn't reveal the implicit, I'm free to mock the base trait without worrying about the implicit

R2b) I can substitute a mock implicit instance when I'm testing the implementation

R3) If I would like to use M#3 but I'm writing mix-ins that must be traits, I use M#4:

/**
 * A mix-in trait for a component that has an execution context
 */
trait HasExecutionContext {
  implicit def executionContext: ExecutionContext
}

trait AuthServiceRpcActions extends
  AuthService with
  HasExecutionContext with
  Logging {

4a) Since this mix-in inherits from HasExecutionContext, any method called in the impl that requires one will bind to the not-yet-implemented implicit def executionContext

case class Session()(implicit
  val executionContext: ExecutionContext,
  val loggingContext: LoggingContext,
  users: userDataDao.Session,
  groups: groupDataDao.Session
) extends super.Session with HasExecutionContext with HasLoggingContext {

4b) Example of how to implement the HasExecutionContext using M#3

-Lance

[Som Snytt]

The other common pattern is to squirrel the implicit away in implicit scope.

scala> case class X[A](x: Int)
defined class X

scala> def f[A](i: Int)(implicit x: X[A]) = i + x.x
f: [A](i: Int)(implicit x: X[A])Int

scala> class C ; object C { implicit val myx: X[C] = X[C](42) }
defined class C
defined object C

scala> f[C](3)
res0: Int = 45

scala> class D[A] { def f(i: Int)(implicit x: X[A]) = i + x.x }
defined class D

scala> val d = new D[C]
d: D[C] = D@1f554b06

scala> d.f(3)
res1: Int = 45

or similarly

scala> trait T { def t: Int }
defined trait T

scala> case class X[A](x: Int)
defined class X

scala> case class C(t: Int) extends T ; object C { implicit val myx: X[C] = X[C](42) }
defined class C
defined object C

scala> def f[A <: T : X](a: A) = a.t + implicitly[X[A]].x
f: [A <: T](a: A)(implicit evidence$1: X[A])Int

scala> f(C(3))
res2: Int = 45

[Viktor Klang]

That's a fantastic reply, Lance

[Christian Kaps]

Hi,

I can only agree with Victor. That's the answer I've expected. Anyway, thanks to all for the inspiring input.

Best regards,

Christian