Is there thought to making ActorProcessor a public API?

[dpratt]

The ActorPublisher and ActorSubscriber are natural (and easy) ways to deal with integrating with external or non-streaming APIs. Adding the obvious ActorProcessor implementation would also be a pretty big help. Currently the Http and StreamTcp modules actually do have an implementation of this, but it's marked as an internal only API.

As you can imagine, it would make life a lot easier if we could somehow get a Flow[Input, Output] from either a Props for an ActorProcessor or even create one from a reactive streams Processor instance. For example, I'm currently working on some things that would be drastically simplified if there was an easy way to model a Flow[Ack, MessageDelivery] using this proposed API. I know it seems a little off, since from a semantic point of view, the output IncomingMessage instances aren't created from the incoming Ack messages, but the goal here is to use this Flow as the left hand side of a stack of BidiFlows. Of course, I could do this today by just implementing all of the AMQP protocol from the network framing level up on top of a StreamTcp Flow[ByteString, ByteString], but as you can imagine, AMQP is complex, I'd really rather just play with my dog, and my employer would frown on a monthlong diversion reinventing the wheel when there are already decent AMQP JVM implementations out there.

Thoughts?

[zeroni...@gmail.com]

Have you seen this thread? The authors have stated that custom ActorProcessors aren't going to be implemented anytime soon, howewer I would be very happy too if they'd changed their minds =)

[David Pratt]

This makes me sad because wrapping an ask inside of mapAsync seems to fundamentally discard/break two very useful abstractions at the same time. I'd much rather have the ability to have a component of a stream graph that actually knows it's participating in such. This is of course not to mention the fact that having to use an Ask (with all its associated overhead and need for an implicit Timeout) seems to kind of violate every reason to actually be using Akka in the first place.

I'm well aware that this could be implemented with a custom Stage, but that API seems a bit clunky for something that is by nature reactive and asynchronous.

Caveat to all of this - I'm well aware that the Typesafe/Akka guys grok this domain on a more fundamental level than I do, so I could be missing something. On the other hand, I am a consumer/user of the platform, and I'd like to think that I'm quite smart and capable. If I can't do it, then maybe the abstraction or API needs some work.

--
>>>>>>>>>> Read the docs: http://akka.io/docs/
>>>>>>>>>> Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to a topic in the Google Groups "Akka User List" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/akka-user/__jiYeub9Ew/unsubscribe.
To unsubscribe from this group and all its topics, send an email to akka-user+...@googlegroups.com.
To post to this group, send email to akka...@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

[Endre Varga]

Hi David,

The reason why we don't expose a raw actor based ActorProcessor class because it is much harder to write a correct Processor than you think. Much much much harder. Knowing how to write a Publisher and Pubscriber does not translate to knowing how to write a Processor. We already know many of the pitfalls of creating various stages and we are reluctant to put our users through of them. 

I'm well aware that this could be implemented with a custom Stage, but that API seems a bit clunky for something that is by nature reactive and asynchronous.

What makes you say that the Stage API is clunky? It is actually a much better abstraction than the actor API for building processors, that is the reason why we came up with them. It is actually easier for *us*, Akka devs, to code Stages instead of Processor actors, they are much simpler conceptually. In fact, if everything goes well, eventually almost all of the built-in stages will be implemented as Stages (most of them already are). Also, the goal with Stages was to expose a low level API so that users can build abstractions, libraries DSLs on top of them, tailored to certain domains. Unfortunately that did not happen yet. For example the reason for having the context as a parameter is just to be able to make things allocation free but, you can put a functional abstraction that uses allocations on top of it.

I have a feeling that people just look at custom stages and because they seem alien, they bail out and want to go actors, since they already know actors. The problem is that actors are not tailored to this backpressured processing so you are on your own, even APIs cannot help much. Stages on the other hand can already cut some of the illegal actions by requiring exectly one reaction to an event as a return value and enforcing the type of those. 

Plus stages have an execution model that is not tied to actors, can be fused, suspended, and easily debugged (well, we don't have it implemented yet).

What is currently missing form stages is the support for asynchronous side channels. This is now implemented internally and called AsyncStage, the only reason we do not expose it as a user API yet is because we need a little bit more experience how to express the API in the best way.

-Endre

 

Caveat to all of this - I'm well aware that the Typesafe/Akka guys grok this domain on a more fundamental level than I do, so I could be missing something. On the other hand, I am a consumer/user of the platform, and I'd like to think that I'm quite smart and capable. If I can't do it, then maybe the abstraction or API needs some work.

On Apr 22, 2015, at 8:49 PM, zeroni...@gmail.com wrote:

Have you seen this thread? The authors have stated that custom ActorProcessors aren't going to be implemented anytime soon, howewer I would be very happy too if they'd changed their minds =)

On Wednesday, April 22, 2015 at 8:23:54 AM UTC+3, dpratt wrote:

The ActorPublisher and ActorSubscriber are natural (and easy) ways to deal with integrating with external or non-streaming APIs. Adding the obvious ActorProcessor implementation would also be a pretty big help. Currently the Http and StreamTcp modules actually do have an implementation of this, but it's marked as an internal only API.

As you can imagine, it would make life a lot easier if we could somehow get a Flow[Input, Output] from either a Props for an ActorProcessor or even create one from a reactive streams Processor instance. For example, I'm currently working on some things that would be drastically simplified if there was an easy way to model a Flow[Ack, MessageDelivery] using this proposed API. I know it seems a little off, since from a semantic point of view, the output IncomingMessage instances aren't created from the incoming Ack messages, but the goal here is to use this Flow as the left hand side of a stack of BidiFlows. Of course, I could do this today by just implementing all of the AMQP protocol from the network framing level up on top of a StreamTcp Flow[ByteString, ByteString], but as you can imagine, AMQP is complex, I'd really rather just play with my dog, and my employer would frown on a monthlong diversion reinventing the wheel when there are already decent AMQP JVM implementations out there.

Thoughts?

--
>>>>>>>>>> Read the docs: http://akka.io/docs/
>>>>>>>>>> Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to a topic in the Google Groups "Akka User List" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/akka-user/__jiYeub9Ew/unsubscribe.
To unsubscribe from this group and all its topics, send an email to akka-user+...@googlegroups.com.
To post to this group, send email to akka...@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

--
>>>>>>>>>> Read the docs: http://akka.io/docs/
>>>>>>>>>> Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
---

You received this message because you are subscribed to the Google Groups "Akka User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+...@googlegroups.com.

[Roland Kuhn]

Very good reply, Endre. I’d like to second this in case there is remaining doubt: we have been working with implementing bounded back-pressure stream processing for 1.5 years now and we still make mistakes when open-coding a Processor. These mistakes are subtle and lead to stream deadlocks or resource leaks and therefore we are not comfortable with offering a user API on this level of abstraction. This is also the reason why we consider Reactive Streams as an SPI for library interop only, it models the complexity of its domain which is going to be too much for normal users.

By offering a restricted API we enable users to express valid stream topologies. If you want the full power then you can obviously implement the Reactive Streams interfaces (using ActorPublisher and ActorSubscriber), but please take a look at AsyncStage and watch its development as I'm certain that that will be a more suitable API.

Regards,

Roland 

Sent from my iPhone