Scalaz support

16 views
Skip to first unread message

Derek Williams

unread,
Jan 28, 2011, 6:33:26 PM1/28/11
to akka...@googlegroups.com
My akka-scalaz project that I have been experimenting with has been
working out quite well, better then I thought it would have. If there
is any interest, and no objections, I'd like to include it in
akka-modules (master, not for 1.0 release unless there was some need).
With Lift experimenting with Scalaz right now as well I felt it might
be a good time to do this. Very little modification would be needed,
just trimming down or removal of a couple tests that I use for
measuring performance, and reverting to an earlier scalatest.

At the moment it mostly deals with Future, but I have also started
experimenting with Actors as well.

Some simple examples of it's usefulness that don't require much
functional programming knowledge:

- Turning a List[Future[A]] into a Future[List[A]]

val list: List[Future[A]] = ...
list.sequence // returns a Future[List[A]], which will wait for all
original Futures to complete

- Turning a function (A => B) into a (A => Future[B])

val f = ((_: Int) * 2).future
f(10).await.result.get == 20

- Chaining functions that return a Future (like the one above)

val f = ((_: String).toInt).future
val g = ((_: Int) * 2).future
val h = ((_: Int) * 10).future
val fgh = f >=> g >=> h
fgh("10").await.result.get == 200

- Create a Future returning function from an ActorRef

val a = actorOf[MyActor].start
val f = a.future
// or, if you want to specifiy the parameter and return type
val f = a.future[String, Int]

f("10").await.result.get == ... // returns Any, or an Int if
previously specified

This makes it easy to compose actors without the actors needing to
know where to send their result. This of course could be done directly
by using Future.onComplete() but this is a more elegent way. I have
implementations of many Scalaz type classes: Functor, Bind, Pure,
Apply, Each, Semigroup, and Zero, which also gives free
implementations of Monad, Applicative, and Monoid, perhaps more. There
are conversions to and from Validation and Promise, and convience
methods for getting the result of a Future, like Future.getOrThrow
which awaits completion of the Future, then either returns the result
(without being contained in an Option) or throws the exception, and
Future.fold, which works like Either.fold, applying 1 of 2 functions
depending on if the future has a result or exception.

There is some overlap with already provided functionality, but it does
work differently. I've tried to minimize any kind of blocking I can,
all functions that are applied to results of futures are spun off to
run in a separate thread, making it safe to apply an intensive
function to the result of a future without worrying about delaying the
actor that compeleted the future (which is what happens normally when
using Future.onComplete). This does increase the overhead, but in some
quick comparisons I did between Viktor's Future.fold and using scalaz
folds I only saw a performance drop of about 5-10%.

If this seems like a good addition to akka-modules, I can create a
ticket and start moving it into master. It depends on the current
scalaz 5.1 snapshot, but since Lift is starting to use it as well
there is a good chance that more up to date releases will be made so
we don't have to depend on our own compilation.

Code is currently located at:
https://github.com/derekjw/akka-scalaz

more examples of use can be found in the tests:
https://github.com/derekjw/akka-scalaz/blob/master/src/test/scala/FuturesSpec.scala

--
Derek

Jonas Bonér

unread,
Jan 29, 2011, 11:39:02 AM1/29/11
to akka...@googlegroups.com, akka...@googlegroups.com
Hi Derek.

I would love to see that in Akka Modules.
You have commit rights right? Add it.

But along with that I'd like to see documentation and some practical examples on how to use it, to make it accessible for non-scalaz-fans. Can you add that as well?

Thanks a lot for the hard work, I'm looking forward to even ,ore scalazification of Akka.

--
Jonas Bonér
CEO | Specialist at Large
work: http://scalablesolutions.se
code: http://akka.io
twtr: @jboner

√iktor Klang

unread,
Jan 29, 2011, 7:08:19 PM1/29/11
to akka...@googlegroups.com
On Sat, Jan 29, 2011 at 12:33 AM, Derek Williams <de...@nebvin.ca> wrote:
My akka-scalaz project that I have been experimenting with has been
working out quite well, better then I thought it would have. If there
is any interest, and no objections, I'd like to include it in
akka-modules (master, not for 1.0 release unless there was some need).
With Lift experimenting with Scalaz right now as well I felt it might
be a good time to do this. Very little modification would be needed,
just trimming down or removal of a couple tests that I use for
measuring performance, and reverting to an earlier scalatest.

At the moment it mostly deals with Future, but I have also started
experimenting with Actors as well.

Some simple examples of it's usefulness that don't require much
functional programming knowledge:

- Turning a List[Future[A]] into a Future[List[A]]


Slick!
 

What thread is executing the onComplete?

(You can do something like this if you want to avoid the completing thread to run the onComplete code:

future.onComplete(f => spawn { doSomethingInAnotherThreadAsProvidedByImplicitDispatcher(f) } )

https://github.com/jboner/akka/blob/master/akka-actor/src/main/scala/akka/actor/Actor.scala#L214

 
If this seems like a good addition to akka-modules, I can create a
ticket and start moving it into master. It depends on the current
scalaz 5.1 snapshot, but since Lift is starting to use it as well
there is a good chance that more up to date releases will be made so
we don't have to depend on our own compilation.


I definitely think that having an akka-scalaz module in akka-modules is a _very_ good thing,
thanks for your work Derek, I totally appreciate it!
 
Code is currently located at:
https://github.com/derekjw/akka-scalaz

more examples of use can be found in the tests:
https://github.com/derekjw/akka-scalaz/blob/master/src/test/scala/FuturesSpec.scala

--
Derek



--
Viktor Klang,
Code Connoisseur
Work:   Scalable Solutions
Code:   github.com/viktorklang
Follow: twitter.com/viktorklang
Read:   klangism.tumblr.com

Derek Williams

unread,
Jan 29, 2011, 10:39:56 PM1/29/11
to akka...@googlegroups.com
On Sat, Jan 29, 2011 at 9:39 AM, Jonas Bonér <jo...@jonasboner.com> wrote:
> Hi Derek.
>
> I would love to see that in Akka Modules.
> You have commit rights right? Add it.

will do!

> But along with that I'd like to see documentation and some practical examples on how to use it, to make it accessible for non-scalaz-fans. Can you add that as well?

Definitely. I'll add relevant documentation to the source, and also
start work on a page for akka.io.

--
Derek

Derek Williams

unread,
Jan 29, 2011, 10:53:57 PM1/29/11
to akka...@googlegroups.com
On Sat, Jan 29, 2011 at 5:08 PM, √iktor Klang <viktor...@gmail.com> wrote:
> What thread is executing the onComplete?

For example, this is my implementation of Functor[Future]:

implicit object FutureFunctor extends Functor[Future] {
def fmap[A, B](r: Future[A], f: A => B): Future[B] = {
val fb = new DefaultCompletableFuture[B](nanosToMillis(r.timeoutInNanos))
r onComplete (fa => fa.result.cata(a => exec(try
{fb.completeWithResult(f(a))} catch {case e =>
fb.completeWithException(e)}),
fa.exception.foreach(fb.completeWithException)))
fb
}
}

I went with a "fail fast on exception" implementation. The exec method
is configurable to either use Actor.spawn, or the HawtDispatch global
queue (to avoid overhead of an actor while still using the same thread
pool). I am also working on a third option for using a pool of actors
with work stealing dispatcher. Right now you choose which one to use
in akka.conf, but I think I have figured out a way of passing the
config around implicitly.

--
Derek

Debasish Ghosh

unread,
Jan 30, 2011, 3:17:57 AM1/30/11
to akka...@googlegroups.com
sounds great. scalaz has two abstractions Strategy and Promise. Strategy separates the concerns of parallelism and algorithm. I guess this is somewhat similar to the Dispatchers in Akka. Promise is like Future, but non-blocking. It implements flatMap and map, but doesn't call get. A Promise takes an implicit Strategy for execution.

Can we bring the abstraction of Promise in Akka ?

Also interesting to see that u r thinking of experimenting with Actors. Now, actors generally don't compose - they are all about side-effects. Though we can compose an actor with a function. scalaz does it with comap, which has the following contract:

comap :: (B=>A) => Actor[A] => Actor[B]

Thanks for your contributions to Akka ..

√iktor Klang

unread,
Jan 30, 2011, 7:45:19 AM1/30/11
to akka...@googlegroups.com
On Sun, Jan 30, 2011 at 9:17 AM, Debasish Ghosh <ghosh.d...@gmail.com> wrote:
sounds great. scalaz has two abstractions Strategy and Promise. Strategy separates the concerns of parallelism and algorithm. I guess this is somewhat similar to the Dispatchers in Akka. Promise is like Future, but non-blocking. It implements flatMap and map, but doesn't call get. A Promise takes an implicit Strategy for execution.

Akka's Future is non-blocking as well, it currently doesn't implement flatMap tho.
https://github.com/jboner/akka/blob/master/akka-actor/src/main/scala/akka/dispatch/Future.scala#L164
 

Can we bring the abstraction of Promise in Akka ?

That would be nice, but I think it gets a bit confusing to have both Promise and Future, so what do we do?
 

Also interesting to see that u r thinking of experimenting with Actors. Now, actors generally don't compose - they are all about side-effects. Though we can compose an actor with a function. scalaz does it with comap, which has the following contract:

comap :: (B=>A) => Actor[A] => Actor[B]


Wouldn't really be usable in Akka since ActorRef and Actor are Any => Unit (not generified)
 

√iktor Klang

unread,
Jan 30, 2011, 7:47:18 AM1/30/11
to akka...@googlegroups.com

Cool, I also would like to have the option to have it as it is right now, to execute the completion listener within the current thread (to avoid Future-starvation), sounds sensible?
 
--
Derek

Derek Williams

unread,
Jan 30, 2011, 8:20:08 AM1/30/11
to akka...@googlegroups.com
On Sun, Jan 30, 2011 at 1:17 AM, Debasish Ghosh
<ghosh.d...@gmail.com> wrote:
> sounds great. scalaz has two abstractions Strategy and Promise. Strategy
> separates the concerns of parallelism and algorithm. I guess this is
> somewhat similar to the Dispatchers in Akka. Promise is like Future, but
> non-blocking. It implements flatMap and map, but doesn't call get. A Promise
> takes an implicit Strategy for execution.
> Can we bring the abstraction of Promise in Akka ?

Sure, by taking advantage of Scalaz type classes :)
It was having an onComplete method that allows us to do non blocking
operations with Future just like with Promise. I don't think there is
any point in having explicit map and flatMap methods for Future since
we get them implicitly with Functor and Bind.

> Also interesting to see that u r thinking of experimenting with Actors. Now,
> actors generally don't compose - they are all about side-effects. Though we
> can compose an actor with a function. scalaz does it with comap, which has
> the following contract:
> comap :: (B=>A) => Actor[A] => Actor[B]
> Thanks for your contributions to Akka ..

At the moment I'm composing Actors through Futures. I turn an ActorRef
into a Kleisli[Future, Any, Any] by simply doing kleisli(actorref !!!
_). I have added the option of including input and output parameter
types, but of course this isn't type safe. I haven't tried TypedActors
yet, but it should be trivial to do the same with it's Future
returning methods.

--
Derek

Derek Williams

unread,
Jan 30, 2011, 8:28:04 AM1/30/11
to akka...@googlegroups.com
On Sun, Jan 30, 2011 at 5:45 AM, √iktor Klang <viktor...@gmail.com> wrote:
> Akka's Future is non-blocking as well, it currently doesn't implement
> flatMap tho.
> https://github.com/jboner/akka/blob/master/akka-actor/src/main/scala/akka/dispatch/Future.scala#L164
>

I'd actually prefer if we removed the map method as well. That was a
method I had suggested before we had onComplete. But due to it's
existence, I have to use the unicode representation of map for it to
use Scalaz. It is much less confusing for most people to be able to
just use map.

PS:
I have worked 36 hours in the last 2 days outside in the cold and
snow... will probably take me another day or so to find time to start
incorporating this unless work slows down.

--
Derek

Derek Williams

unread,
Jan 30, 2011, 8:40:57 AM1/30/11
to akka...@googlegroups.com
On Sun, Jan 30, 2011 at 5:47 AM, √iktor Klang <viktor...@gmail.com> wrote:
> Cool, I also would like to have the option to have it as it is right now, to
> execute the completion listener within the current thread (to avoid
> Future-starvation), sounds sensible?

If my idea using implicits works it would be no problem. I'm pretty
sure when I was looking at the code for Promise that there was a few
instances where you could not supply a Strategy so the default (which
is not specifiable) was used, and I'd like to do better.

--
Derek

Peter Vlugter

unread,
Jan 30, 2011, 3:11:09 PM1/30/11
to akka...@googlegroups.com
Hi Derek,

Very cool! Will be great to have your akka-scalaz in akka-modules.

I've been interested in exploring this area as well. Especially Scalaz's Promise.

Cheers,
Peter

Derek Williams

unread,
Jan 30, 2011, 3:58:55 PM1/30/11
to akka...@googlegroups.com
On Sun, Jan 30, 2011 at 1:11 PM, Peter Vlugter <pvlu...@gmail.com> wrote:
> I've been interested in exploring this area as well. Especially Scalaz's Promise.

That is the point of having type class instances for Akka's Future,
there is no reason to use Promise if you are using Akka since they do
the same thing (I haven't tested how Promise handles exceptions
though). It is also easy to convert a Promise into a Future, but it is
a blocking call to convert a Future to a Promise because we can't
instantiate a Promise without the function that returns the value
(meaning, I have to use promise(future.await.get) which blocks a
thread), although it would be possible to lift a Future[A] into a
Future[Promise[A]].

Oh wait, I think I misunderstood what you said now that I had to
reread it. I'll keep all that up there though since someone might find
it interesting.

Yeah, if you were interested in Scalaz's Promise before, well good
news! Akka's Future now works just like Promise (at least the
important parts do). Although I've avoided implementing anything that
requires a call to Future.await (like implementing Equals since it
returns a Boolean, not a M[Boolean])

--
Derek

√iktor Klang

unread,
Jan 30, 2011, 4:04:26 PM1/30/11
to akka...@googlegroups.com

Yeah, I really do must say that Future has shaped up quite a lot recently, much thanks to you Derek.
 

--
Derek

Derek Williams

unread,
Jan 31, 2011, 5:57:23 PM1/31/11
to akka...@googlegroups.com
I have commited akka-scalaz into akka-modules. Just need to work on
some documentation now.

I think I followed the correct conventions for layout and project
config, but someone who is familiar with the way the sbt project is
set up might want to make sure I didn't do something wrong.

--
Derek

Jonas Bonér

unread,
Feb 1, 2011, 2:49:32 AM2/1/11
to akka...@googlegroups.com
Awesome.
--
Jonas Bonér

Specialist at Large
work: http://scalablesolutions.se
code: http://akka.io
twtr: @jboner

"Derek Williams" <de...@nebvin.ca> skrev:

Jason Zaugg

unread,
Feb 2, 2011, 4:56:47 AM2/2/11
to akka...@googlegroups.com
On Sat, Jan 29, 2011 at 12:33 AM, Derek Williams <de...@nebvin.ca> wrote:
> If this seems like a good addition to akka-modules, I can create a
> ticket and start moving it into master. It depends on the current
> scalaz 5.1 snapshot, but since Lift is starting to use it as well
> there is a good chance that more up to date releases will be made so
> we don't have to depend on our own compilation.

I've just configured a CI build [1] on the Scala Tools Hudson (Thanks
Dave!). We build against 2.8.0, 2.8.1, and 2.9.0-SNAPSHOT. This
automatically deploys snapshots to the scala-tools repository. The
group ID and version have changed.

lazy val scalazDep = "org.scalaz" %% "scalaz-core" % "6.0-SNAPSHOT"

We're working towards a 6.0 release, currently targeting early March.

Let us know what else we can do to support this integration.

-jason

[1] http://hudson.scala-tools.org/view/scalaz/

Derek Williams

unread,
Feb 2, 2011, 2:10:08 PM2/2/11
to akka...@googlegroups.com
On Wed, Feb 2, 2011 at 2:56 AM, Jason Zaugg <jza...@gmail.com> wrote:
> Let us know what else we can do to support this integration.
>
> -jason

The 2 things I was hoping for was updated snapshots (done) and
hopefully a new release soon (almost done). So everything is looking
great. Thanks Jason.

--
Derek

Reply all
Reply to author
Forward
0 new messages