Router Like Behavior with Streams

[danny.b...@protectwise.com]

Are there plans to add routers to streams? FlexiRoute accomplishes some of that behavior, but I'm not seeing how to safely dynamically add/remove OutputHandles. Also not sure of how one would accomplish consistent hashing, where we're looking to split up a stream to specific processors for the purpose of state maintenance. Wasn't sure how to create a state where you wait till an element is ready, then wait for demand for the output that the element hashes to. Alternatively, buffer elements/outputs until an element hashes to an output with demand.

[Konrad Malawski]

Hi Danny,

What you describe sounds more like an "stream of streams" scenario (like our groupBy for example) - using it you can create substreams based on some key and then produce to that substream.

In FlexiRoute / FlexiMerge it's not supported to change the number of endpoints - it's a static junction currently.

If you could come up on a few use case descriptions and maybe formulate them as an issue we can take a look at it during our planing and decide if it's something fundamental enough to include as building block or if it's something that should be described as a

cookbook recipe as "how to build these kinds of junctions if you need one". We definitely do appreciate your feedback, thanks a lot in advance!

On Thu, Nov 13, 2014 at 4:04 PM, <danny.b...@protectwise.com> wrote:

Are there plans to add routers to streams? FlexiRoute accomplishes some of that behavior, but I'm not seeing how to safely dynamically add/remove OutputHandles. Also not sure of how one would accomplish consistent hashing, where we're looking to split up a stream to specific processors for the purpose of state maintenance. Wasn't sure how to create a state where you wait till an element is ready, then wait for demand for the output that the element hashes to. Alternatively, buffer elements/outputs until an element hashes to an output with demand.

--
>>>>>>>>>> Read the docs: http://akka.io/docs/
>>>>>>>>>> Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups "Akka User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+...@googlegroups.com.
To post to this group, send email to akka...@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

--

Cheers,

Konrad 'ktoso' Malawski

hAkker @ Typesafe

[Danny Browning]

The main use case would be a flexiroute or groupBy with an elastic cluster. Since remoting/clustering is down the road, was just looking for a possible way to use streams with remoting/clustering, other than ActorPublisher/ActorSubscriber.

You received this message because you are subscribed to a topic in the Google Groups "Akka User List" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/akka-user/HyvIUMaIBbQ/unsubscribe.
To unsubscribe from this group and all its topics, send an email to akka-user+...@googlegroups.com.

[Konrad Malawski]

For now you'll have to push it to an Actor{Subscriber / Publisher} boundary and take it from there into remoting.

We're also very much looking forward to streams in clusters, but for now the local ones are loads of work and fun already :-)

[Danny Browning]

That's what we're doing in the interim, just wanted to make sure we weren't missing something. Thanks for the work on streams!