Migrating from wandoulabs websockets to stock akka-io

173 views
Skip to first unread message

Sam Halliday

unread,
May 23, 2015, 6:38:22 AM5/23/15
to akka...@googlegroups.com
Hi all,

I'm very excited that akka-io now has WebSocket support.

In ENSIME, we're planning on using this wrapper over wandoulab's websockets


to easily create a REST/WebSockets endpoint with JSON marshalling for a sealed family, with backpressure.

Smootoo's wrapper works really well, and I have had the pleasure of using it in a corporate environment so I trust it to be stable.


For future proofing, it would seem sensible to move to stock akka-io for WebSockets, so I'm considering sending a PR to retrofit the wrapper. I have a couple of questions about that:

1. does akka-io's HTTP singleton actor support WebSockets now? That was the big caveat about using wandoulabs. It means all kinds of workarounds if you want to just use HTTP in the same actor system.

2. is there a migration guide for wandoulabs to akka-io? Or would it be best just to rewrite the wrapper from scratch on top of akka-io?

3. where is the documentation? This just has a big TODO on it


I can't even find any examples. I guess the key thing is the handshaking, which would mean rewriting this bit (and the corresponding client side handshake)


Best regards,
Sam

Arnaud Gourlay

unread,
May 23, 2015, 1:49:26 PM5/23/15
to akka...@googlegroups.com
Hi Sam,

I am also really interested in migrating from Wandoulabs to the new Akka WS implementation.
So far this is the best example I could find https://github.com/jrudolph/akka-http-scala-js-websocket-chat

Hope this helps,
Arnaud

Sam Halliday

unread,
May 26, 2015, 7:37:38 AM5/26/15
to akka...@googlegroups.com
Thanks Arnaud!

The key is in this line

  https://github.com/jrudolph/akka-http-scala-js-websocket-chat/blob/239af857da2f174ea1624a84b0861c42cf4d1f2d/backend/src/main/scala/example/akkawschat/Webservice.scala#L33

so on the server side, the REST endpoint upgrades to a WebSocket `Flow` (i.e. akka streams API) via a directive.

My legacy code is just an Actor, so I'll have to reimplement my marshalling layer and so on around this (or maybe marshalling is already handled).

I've had a look at the Akka Streams documentation but it is not clear to me how to manually create a Flow from an Actor. I may have to read this a few more times, but I don't think it contains the information that I need... there doesn't appear to be a `Flow.actorRef`

Roland Kuhn

unread,
May 27, 2015, 2:35:29 AM5/27/15
to akka-user
Hi Sam,

it might be better to take a step back before potentially running in the wrong direction. First off, Akka HTTP offers a complete solution for everything HTTP (including websockets) within an ActorSystem. Before deciding to combine this with another tool I recommend that you explore first how Akka HTTP works, because it introduces several fundamentally new concepts. In particular, when talking about it as “Spray 2.0” it is important to note that everything ActorRef-related in Spray has been replaced by Streams—a completely different abstraction that is not an Actor. The whole underpinnings are completely rewritten in a radically different fashion, so don’t expect any Spray modules that live “beneath the surface” to seamlessly fit onto Akka HTTP.

We could go into the details Wandoulabs’ websocket add-on, but I don’t see much value in discussing that before the basics are clear. The other piece of information that I’m lacking is why you would want to “retrofit” something in this context, it might be better to explain the ends and not the means in order to get help.

Regards,

Roland

--
>>>>>>>>>> Read the docs: http://akka.io/docs/
>>>>>>>>>> Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups "Akka User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+...@googlegroups.com.
To post to this group, send email to akka...@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.



Dr. Roland Kuhn
Akka Tech Lead
Typesafe – Reactive apps on the JVM.
twitter: @rolandkuhn

Sam Halliday

unread,
May 27, 2015, 2:51:25 AM5/27/15
to akka...@googlegroups.com

Hi Roland,

I've read the documentation, several times, I've even given you feedback on the documentation in an earlier milestone phase. Also, the documentation for WebSockets in Akka is TODO and TODO. The documentation on the routing directives are extremely sparse. In particular, there are no promises around the implementation of back pressure from the new websockets.

What I'm missing is the ability to hook an existing actor system into something that expects a Flow, with back pressure preserved. I understand Flow, but I don't understand the implementation in terms of Actors (which incidentally, is exactly my primary feedback on the earlier documentation). You're now confusing me further by saying that Streams are not actors, because I was told at the time that streams are implemented in terms of actors.

In case you didn't pick up on it, I'm planning on moving away from wandoulabs, not integrate it. This is the key piece, distilled into a standalone problem.

Best regards, Sam

You received this message because you are subscribed to a topic in the Google Groups "Akka User List" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/akka-user/39HItLST7lw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to akka-user+...@googlegroups.com.

Roland Kuhn

unread,
May 27, 2015, 3:18:09 AM5/27/15
to akka-user
27 maj 2015 kl. 08:51 skrev Sam Halliday <sam.ha...@gmail.com>:

Hi Roland,

I've read the documentation, several times, I've even given you feedback on the documentation in an earlier milestone phase. Also, the documentation for WebSockets in Akka is TODO and TODO. The documentation on the routing directives are extremely sparse. In particular, there are no promises around the implementation of back pressure from the new websockets.


For this venture you need no HTTP documentation, and the documentation for Streams is complete (only lacking a translation to Java in one place). The statement “there are no promises around back-pressure” indicates that you did not, in fact, understand the full extent of what Akka Streams are. We’d love to improve the documentation in this regard, but we’d need to first figure out where their deficiency is, hence I’m talking with you.

What I'm missing is the ability to hook an existing actor system into something that expects a Flow, with back pressure preserved.


As I hopefully explained in my other mail about hoses: your Actors would need to implement the full spec, they’d need to be watertight. This is why “simple” Sink.actorRef integration will not work if you don’t intend on spilling messages.

I understand Flow, but I don't understand the implementation in terms of Actors (which incidentally, is exactly my primary feedback on the earlier documentation). You're now confusing me further by saying that Streams are not actors, because I was told at the time that streams are implemented in terms of actors.


How we implement Flows internally should be of no consequence—suffice it to say that the project took 1.5 years because that task is genuinely hard.

In case you didn't pick up on it, I'm planning on moving away from wandoulabs, not integrate it. This is the key piece, distilled into a standalone problem.


In order to solve this particular problem you’ll need to carefully describe the back-pressure protocol spoken by your Actor. As Viktor suggested, an easy way to integrate is via the ask pattern, and that works because of the 1:1 correspondence between ins and outs that allow automatic back-pressure propagation. If that is not applicable then there is no generic solution.

Regards,

Roland

Sam Halliday

unread,
May 27, 2015, 6:01:46 AM5/27/15
to akka...@googlegroups.com
Hi Roland,

On Wednesday, 27 May 2015 08:18:09 UTC+1, rkuhn wrote:
> For this venture you need no HTTP documentation

Actually I'd like to go futher than akka-http and see documentation that
allows me to use WebSockets with akka-io directly, bypassing the
Streaming API.

The historic limitation has always been that the HTTP
ConnectionManager of akka-io was unable to handle the upgrade
request. Wandoulabs use their own actor to handle HTTP and UHTTP,
but it has annoying side effects. I'm interested to know how
akka-http has managed to implement WebSockets with that
constraint in place (or if that constraint has been lifted).


> The statement “there are no promises around back-pressure”
> indicates that you did not, in fact, understand the full extent
> of what Akka Streams are.

Just because something is a Flow does not make any promises about
how back pressure is actually implemented in that flow: you have
even pointed out how to create "open hoses", or just blow up
internal buffers when downstream doesn't consume fast enough.

I'd like to know if the underlying Source of incoming client
messages on a websocket endpoint will respect the backpressure
from the `handleWebsocketMessages: Flow` that is dealing with
it (i.e. not read from the network unless
`handleWebsocketMessages` is pulling), and conversely if the
underlying Sink back to the client is going to backpressure using
akka-io's Ack mechanism so that `handleWebsocketMessages` will
only be pulled when akka-io gives the green lights.


> We’d love to improve the documentation in this regard, but we’d
> need to first figure out where their deficiency is, hence I’m
> talking with you.

I think clarity on the above two points would be a good start. In
addition, and more generally, integration between "hoses"
and "hammers" is extremely important --- unless you intentionally
want to limit akka-streams uptake to green field projects only.
The project I work on has 1.2 million lines of Scala code with
legacy components dating from Scala 2.6. There isn't a snowball's
chance in hell of rewriting it to use akka-streams.


>> What I'm missing is the ability to hook an existing actor
>> system into something that expects a Flow, with back pressure
>> preserved.
>
> As I hopefully explained in my other mail about hoses: your
> Actors would need to implement the full spec, they’d need to be
> watertight.

This is a start. Is there a test framework that can be used to
stress test the implementation of a Stream / Actor bridge?


> How we implement Flows internally should be of no consequence

On the contrary, I feel the implementation is of huge
significance. Firstly, it helps to understand the expected
performance, and second it is critical when writing
integration code. It is extremely bizarre that both projects
should be released under the akka banner, yet be so siloed.


> In order to solve this particular problem you’ll need to
> carefully describe the back-pressure protocol spoken by your
> Actor.

The Actor is expecting to send messages directly to akka-io and
speaks the akka-io Ack protocol using a simple object as the Ack
message. It expects an Ack before it will send a message
upstream (and it greedily consumes data, but that could easily be
changed).

Best regards,
Sam

Roland Kuhn

unread,
May 27, 2015, 9:01:46 AM5/27/15
to akka-user
Hi Sam,

27 maj 2015 kl. 12:01 skrev Sam Halliday <sam.ha...@gmail.com>:

Hi Roland,

On Wednesday, 27 May 2015 08:18:09 UTC+1, rkuhn wrote:
> For this venture you need no HTTP documentation

Actually I'd like to go futher than akka-http and see documentation that
allows me to use WebSockets with akka-io directly, bypassing the
Streaming API.

The historic limitation has always been that the HTTP
ConnectionManager of akka-io was unable to handle the upgrade
request. Wandoulabs use their own actor to handle HTTP and UHTTP,
but it has annoying side effects. I'm interested to know how
akka-http has managed to implement WebSockets with that
constraint in place (or if that constraint has been lifted).

Maybe I misunderstand you still, but ConnectionManager is not something that we have in Akka HTTP, and with akka-io you seem to mean the akka.io package in akka-actor, which is not related to HTTP as far as user API is concerned—Streams have TCP support that abstracts over the low-level mechanism and provides higher-level semantics including back-pressure. Bypassing these mechanisms is neither desirable nor possible.

The whole architecture of the Wandoulabs solution is completely unrelated to how Akka HTTP works, so the constraint you mention (which I don’t know more about, never having used that library) is unlikely to exist; but we also did not “lift” it since we completely replaced the mechanism wholesale. When thinking about Streams it is best to first forget Actors, learn how to work with Streams alone, and then as an advanced topic interface with plain Actors.



> The statement “there are no promises around back-pressure”
> indicates that you did not, in fact, understand the full extent
> of what Akka Streams are.

Just because something is a Flow does not make any promises about
how back pressure is actually implemented in that flow: you have
even pointed out how to create "open hoses", or just blow up
internal buffers when downstream doesn't consume fast enough.

This is not true: a Flow has one open input and one open output port and all data elements that flow through these ports will do so governed by the Reactive Streams back-pressure semantics. This means that the Flow has the ability to slow down the Source that is connected to it and it also reacts to slow-down requests from the Sink that it will be connected to.

It is the Sink.actorRef implementation that presents the proper “hose” interface but then just lets the water fall into the bucket. There is also a Sink.ignore which replaces the bucket with a black hole …


I'd like to know if the underlying Source of incoming client
messages on a websocket endpoint will respect the backpressure
from the `handleWebsocketMessages: Flow` that is dealing with
it (i.e. not read from the network unless
`handleWebsocketMessages` is pulling), and conversely if the
underlying Sink back to the client is going to backpressure using
akka-io's Ack mechanism so that `handleWebsocketMessages` will
only be pulled when akka-io gives the green lights.

Yes, of course, this is what the Flow interface designates as explained above. And you are also right that a few layers down akka.io’s Ack and ResumeReading mechanisms are used to propagate the back-pressure to and from the TCP socket.



> We’d love to improve the documentation in this regard, but we’d
> need to first figure out where their deficiency is, hence I’m
> talking with you.

I think clarity on the above two points would be a good start. In
addition, and more generally, integration between "hoses"
and "hammers" is extremely important --- unless you intentionally
want to limit akka-streams uptake to green field projects only.
The project I work on has 1.2 million lines of Scala code with
legacy components dating from Scala 2.6. There isn't a snowball's
chance in hell of rewriting it to use akka-streams.

Right. My responses to you have been deliberately cautious in order to make it clear that for a successful integration between hammers and hoses you will need a solid understanding of each of those in isolation. I think that point got across :-)

Now, looking at Integrating With Actors you’ll see ActorPublisher and ActorSubscriber. These two helpers handle some of the Reactive Streams protocol primitives but leave the exact use of back-pressure and data generation up to you, I think this is what you are looking for. Be aware, though, that combining both sides into one Actor—a so-called Processor—is more difficult than it seems due to failure and termination handling: if you forget to handle a certain ordering of events then the stream will not properly be torn down when “finished” and your websocket connections turn into Actor leaks. I admit to being a bit dramatic here, but I don’t want to leave the dangers unmentioned.

There is another way of integrating arbitrary asynchronous elements into a Stream, but it is not yet documented since we want to first gain more experience with it internally: AsyncStage. This limited abstraction provides all the stream safety properties automatically and leaves it to you when to request or generate elements. You can see it in action in the implementation of mapAsync. My goal is to refine this into the best all-round tool for Stream–Actor integration, your feedback would of course be welcome! (but here you’ll have to take code for documentation for now)



>> What I'm missing is the ability to hook an existing actor
>> system into something that expects a Flow, with back pressure
>> preserved.
>
> As I hopefully explained in my other mail about hoses: your
> Actors would need to implement the full spec, they’d need to be
> watertight.

This is a start. Is there a test framework that can be used to
stress test the implementation of a Stream / Actor bridge?

There is the Reactive Streams TCK, see http://www.reactive-streams.org/ .



> How we implement Flows internally should be of no consequence

On the contrary, I feel the implementation is of huge
significance. Firstly, it helps to understand the expected
performance, and second it is critical when writing
integration code. It is extremely bizarre that both projects
should be released under the akka banner, yet be so siloed.

Hopefully I was able to provide enough background to show that we have indeed thought of the relation between Streams and Actors, but we still need to find an appropriate way of deflecting or avoiding the initial/intuitive reaction of wanting to have Stream-Actors prematurely. Streams are really a much nicer abstraction as soon as you forget about Actors for a second.



> In order to solve this particular problem you’ll need to
> carefully describe the back-pressure protocol spoken by your
> Actor.

The Actor is expecting to send messages directly to akka-io and
speaks the akka-io Ack protocol using a simple object as the Ack
message. It expects an Ack before it will send a message
upstream (and it greedily consumes data, but that could easily be
changed).

As explained above directly interacting with akka.io will not happen—there are multiple protocol layers between the ByteStrings and the websocket data format messages—but you might find inspiration on how to use AsyncStage in the following (feel free to re-read after having digested the Stage abstraction):
  • when the downstream ball comes in (i.e. onPull—demand from downstream [which you seem to call upstream]) dispatch an Ack to your Actor and holdDownstream
  • when message comes back from your Actor (via onAsyncInput()), push() it downstream towards the websocket
  • when the upstream ball comes in with a message from the client (i.e. onPush) dispatch to your Actor and pull() if the Actor has previously signaled demand, or holdUpstream
  • when demand is signaled by your Actor, record that and/or release the ball upstream (if previously held) to read more data from the websocket

Regards,

Roland


Best regards,
Sam

--
>>>>>>>>>> Read the docs: http://akka.io/docs/
>>>>>>>>>> Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups "Akka User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+...@googlegroups.com.
To post to this group, send email to akka...@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

Sam Halliday

unread,
May 27, 2015, 9:58:57 AM5/27/15
to akka...@googlegroups.com
On Wednesday, 27 May 2015 14:01:46 UTC+1, rkuhn wrote:
Maybe I misunderstand you still, but ConnectionManager is not something that we have in Akka HTTP

I am referring to the `manager` field in an implementation of `akka.io.IO.Extension`. The limitation that I'm referring to lies with spray-can, in that it cannot handle the WebSockets Upgrade request... presumably akka-http bypasses this because your HTTP manager is more advanced and you are not using spray-can in the first instance.

I should very much like to have access to the new HTTP manager so that I can bypass the Streams API for legacy integration with Actors. However, it would appear that you're saying there is no way to use akka-io for HTTP communication in this way, and that is a great shame. The spray-can/http integration with the actor frameworks is fantastic, and the loss of this integration will be sorely missed when it is deprecated.
 
Just because something is a Flow does not make any promises ...

This is not true: a Flow has one open input and one open output port and all data elements that flow through these ports will do so governed by the Reactive Streams back-pressure semantics. This means that the Flow has the ability to slow down the Source that is connected to it and it also reacts to slow-down requests from the Sink that it will be connected to.

This means nothing if the Source doesn't backpressure properly or the Sink just acks everything. I don't really care about what happens in one component, I care about the entire system. In your new websockets API, the user provides the implementation of the Flow... what I care about is that the framework behaves correctly in their Source and Sink.

In particular, I'd like to have confirmation that network packets are only read from the network when the Source is told to progress and that the backpressure callback to the user-provided Flow is only invoked when akka-io has confirmed that it has written the last message to the the network (i.e. the akka-io Ack). Details of these points are *absolutely essential* to the management of the heap and I do not want to simply assume that it is true.


With regards to the rest of your comments and suggestions, thanks for that. I shall study it further if I have time to undertake writing a wrapper layer. In the short-term, it looks like the barrier to entry is far too high without a convenient Stream/Actor bridge in place, so I will be sticking with wandoulabs' and smootoo's convenience classes.


Best regards,
Sam

Reply all
Reply to author
Forward
0 new messages