Could spray have been a wrapper over netty?

[Rajiv]

Hi:
First of all Spray is an excellent project and integration with Akka applications is a joy.

I was wondering why the authors decided to go all the way down to Java NIO instead of providing an actor based interface on top of Netty which is a well tested and highly performant project? It seems like if this alternative was chosen a lot of Netty's other goodies like frame decoder implementations, web socket implementation, spdy implementation would come for almost free.

Netty is also used by a lot of other frameworks like Finagle and Vertx giving it a lot of indirect testers.

I hope this did not come of as arrogant. It seems like using Netty would free the Spray team to focus on the API and contribute to Netty performance through patches. Was this option evaluated?

Thanks,
Rajiv

[David Pratt]

Actually, part of the genius of the Spray stack is that you're *not* locked into using the Spray HTTP backend. We've found that spray-can is a perfectly fine implementation of an HTTP server, but there are some features available in Netty/Finagle that we require for our business problems.

I don't claim to speak for Mathias, but it's been my impression that the genesis of spray-can was that the existing nonblocking I/O solutions weren't idiomatic - they rely on mutable state, and are really more geared towards an imperative rather than a functional style. There is, of course, nothing wrong with that, but part of the elegance and beauty of the spray codebase is that it's built nearly as pure functional code (I don't recall seeing a single var in the whole codebase, and I've read pretty much all of it over the past couple years).

Shameless self-promotion - we built a Spray connector for Finagle (available here - https://github.com/dpratt/spray-finagle-connector) that has worked really well for us in production for over a year now.

On the server side, I actually do slightly prefer Finagle - not due to any weakness on spray-can's part, it's just a more broad implementation with a lot of monitoring/management features that we require. We do use spray-can for a few of our projects, but the 'big' ones run on Finagle.

With that said - spray-client is absolutely amazing. I've standardized on using it for all of our HTTP/REST clients - it's a joy to work with, is incredibly performant, and produces very clean, elegant and easy to understand code. I played around with the client side of Finagle and found it cumbersome, overly complex, and too low level for our needs.

--
You received this message because you are subscribed to the Google Groups "spray-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to spray-user+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Rajiv]

Yeah, I guess the reason to not use Netty was to stay away from its mutable state hocus pocus. Wonder though if it would have been possible to provide an immutable HTTP model on top of Netty so that users of Spray did not have to deal with it. 

Anyway I was curious, because it seems like the Spray team or the community will slowly have to build a lot of the functionality already in Netty (more transports, more enhancements spray-io to support custom protocols etc).

Thanks,
Rajiv

[Mathias]

Rajiv,

the question why we decided to not build upon Netty is a very good (and therefore somewhat frequent) question, which deserves a proper answer.
Therefore I'd like to point you to [this episode][1] of the Scalatypes podcast, in which this topic is discussed in quite some detail.
Let us know if it leaves you with follow-up questions...

Cheers,
Mathias

[1]: http://scalatypes.com/episode-27-mathias-doenitz-on-spray-part-1

---
mat...@spray.io
http://spray.io

>>> email to spray-user+...@googlegroups.com <javascript:>.

[Rajiv Kurian]

Just read the bit about Netty. Interesting - I can see why staying within the Akka ecosystem is so important. I am a bit confused as to why the lowest level could not be Netty instead of Java NIO. Maybe because of efficiency - one would still have to provide Netty with its boss pool etc and there would be fewer opportunities to share execution context with Akka.

It's an interesting trade off, since Netty does bring a lot of protocol implementations and other nifty features with it, but probably doesn't matter long term.

[Rajiv Kurian]

Just another question. Do you see a lot of people making the choice of Netty vs Spray? For me if I had to only deal with HTTP I would gladly only use Spray, but am torn between the two (favoring Netty) when it comes to implementing custom protocols. I am really rooting for Spray though. The API is a great match for actor based systems.

Thanks,

Rajiv

[Mathias]

Rajiv,

as I say in the podcast there is absolutely nothing wrong with Netty.
It's a great project with an excellent track record so far and quite a few very sharp minds behind it that definitely know what they are doing.

But: Netty is still a Java project with a Java-style API. And as such it frequently relies on mutable data structures and doesn't use case-classes, functions, Scala/Akka Futures, actors, scala collections, type-classes, etc. You will need an adapter layer if you'd like to keep the Java-side structures out of your codebase.
Also it doesn't leverage a lot of the things that you might have come to love with regard to your Scala + Akka stack, like: configuration via the type safe config, akka event-bus logging, unified and nicely configurable thread-pool mgmt., full-stack problem analysis / optimization, the typesafe console, etc.

A (Scala + Akka)-only solution like spray on the other hand is Scala "all the way through": no adapter layers, instead: actors and Futures as API primitives, integrated configuration, logging, etc. It offers the same principles, concepts and coding style in all layers of the stack, which for example makes problem/bottleneck analysis much easier.

We decided to go down the Scala-only route because we think there is value in doing so.

> For me if I had to only deal with HTTP I would gladly only
> use Spray, but am torn between the two (favoring Netty) when it comes to
> implementing custom protocols. I am really rooting for Spray though.

Yes, we are aware of the missing things.
Netty is quite a bit older than spray and as such has a feature set that is more rounded off.
We are working on improving support for building custom protocols, so expect more soon.

Cheers,
Mathias

---
mat...@spray.io
http://spray.io

[Rajiv Kurian]

Thank you for the detailed response Mathias. I can definitely see why you guys chose the route you did.

>>>> has worked really well for us in production for over a year n> --