Using Identify & ActorIndentity as messages for flow control

[Yaroslav Klymko]

Hi guys,

I'm wondering are there any drawback on using Identify and ActorIndentity reply for flow control mechanism.

Why I want to use Identify & ActorIndentity - because those are implicitly implemented in all actor.

So let's say producer is sending messages with high rate to consumer, and after each N messages sent producer will send Identify and more N messages, thus ActorIndentity reply means that actor finished with all previous messages, and producer can send more N messages.

Idea that slow consumer will have up to 2N messages + Identify

It will give me ability to decrease pressure on consumer and better throughput then Pull or Ack based approaches

Consumer actor state example with N=2

PROCESSING

msg1

MAILBOX

msg2

Identify

msg3

msg4

after a while:

PROCESSING

msg3

MAILBOX

msg4

msg5

Identify

msg6

msg7

Thanks in advance.

[Oleg Zhurakousky]

So, what you are saying is that you want to batch up and process N amount of messages before the next batch. Is that correct?

Cheers

Oleg

--
>>>>>>>>>> Read the docs: http://akka.io/docs/
>>>>>>>>>> Check the FAQ: http://akka.io/faq/
>>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups "Akka User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+...@googlegroups.com.
To post to this group, send email to akka...@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/groups/opt_out.

[Yaroslav Klymko]

I would not say "to batch", just to have ability to not overfill consumer's mailbox if it's slow, to avoid OutOfMemory

[Oleg Zhurakousky]

So, 'throttling' would be a better term for that. 

Have you look at this: http://letitcrash.com/post/28901663062/throttling-messages-in-akka-2

Cheers

Oleg

[Yaroslav Klymko]

Took a look on http://letitcrash.com/post/28901663062/throttling-messages-in-akka-2

So I want to have a ActorIdentity reply or absence of reply from Consumer as a trigger for throttling.

Thus I'm interested whether it's safe to use it.

You received this message because you are subscribed to a topic in the Google Groups "Akka User List" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/akka-user/4VI4My8tECo/unsubscribe.
To unsubscribe from this group and all its topics, send an email to akka-user+...@googlegroups.com.

To post to this group, send email to akka...@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/groups/opt_out.

--

Yaroslav Klymko  |  Software Engineer 

[Patrik Nordwall]

On Sun, Feb 2, 2014 at 4:12 PM, Yaroslav Klymko <t3h...@gmail.com> wrote:

Took a look on http://letitcrash.com/post/28901663062/throttling-messages-in-akka-2

So I want to have a ActorIdentity reply or absence of reply from Consumer as a trigger for throttling.

Thus I'm interested whether it's safe to use it.

It is possible to use it as you describe.

/Patrik

--

Patrik Nordwall
Typesafe -  Reactive apps on the JVM
Twitter: @patriknw

[Yaroslav Klymko]

thx Patrik

[Oleg Zhurakousky]

I have to admit, this is a pretty cool way of throttling in the completely async architecture.

However, while sending Identify works I wonder if its the right type of Message to use here (from the "correctness" standpoint). As I understand, Identify is mainly used with actorSelections to help determine if an actual actorRef can be obtained for a given paths. I may be wrong, but thats what I am picking up from the docs.  

So, when you do have an actual ActorRef, constantly sending Identify Message to it for the purpose of controlling the flow doesn't seem to make a lot of sense. On top of that, the response you get is ActorIdentity rather then something more specific/informative to the throttling use case (e.g., StatusMessage). Here is what I mean; I may want to know the elapsed time between last StatusMessage and the current one to track the performance of my system exposing possibilities of dynamically adjusting the amount of messages sent between each StatusMessage. I may also want to gather statistics to compute averages etc. Having a dedicated message type for this specific use case seem to make more sense then falling back to Identify pattern. 

Obviously its all achievable today, but since we are talking about features already embedded into Akka, I was wondering as to what you guys (akka eng) think about adding something like that. 

Cheers

Oleg

[Patrik Nordwall]

On Sun, Feb 2, 2014 at 10:33 PM, Oleg Zhurakousky <oleg.zhu...@gmail.com> wrote:

I have to admit, this is a pretty cool way of throttling in the completely async architecture.

Well, I think the applicability of flow control by using Identify/ActorIdentity in this way is limited to a few specific use cases, where you know that the bottleneck is in the consumer actor, and thereby its mailbox grows when it is slower than producer. It will not work if the consumer delegates the actual work to other actors, or external async service (e.g. async http request).

 

However, while sending Identify works I wonder if its the right type of Message to use here (from the "correctness" standpoint). As I understand, Identify is mainly used with actorSelections to help determine if an actual actorRef can be obtained for a given paths. I may be wrong, but thats what I am picking up from the docs.  

That is right, but you can send it with ActorRef as well. It is only a message that is handled automatically by the target actor, i.e. it is placed in the mailbox in the same way as all application messages.

 

So, when you do have an actual ActorRef, constantly sending Identify Message to it for the purpose of controlling the flow doesn't seem to make a lot of sense. On top of that, the response you get is ActorIdentity rather then something more specific/informative to the throttling use case (e.g., StatusMessage). Here is what I mean; I may want to know the elapsed time between last StatusMessage and the current one to track the performance of my system exposing possibilities of dynamically adjusting the amount of messages sent between each StatusMessage. I may also want to gather statistics to compute averages etc. Having a dedicated message type for this specific use case seem to make more sense then falling back to Identify pattern. 

I agree that it is more clear and powerful with a more specific message. There are also other ways, e.g. let the consumer send a request for more messages to the producer, when it is ready to consume more.

Cheers,

Patrik

[Oleg Zhurakousky]

Thanks Patrick

> . . .limited to a few specific use cases, where you know that the bottleneck is in the consumer actor

True, unless you look at it from a slightly different angle. Producers typically don't know or care about implementation details of the Consumers (internally long running tasks - blocking or delegation to other consumers). This means that all such mechanism helps to control is the state of the mailbox essentially making sure its empty (almost empty) before sending more messages. And if such consumer is simply a delegate to another consumer then continue using the same mechanism downstream will now give the same effect to a delegating consumer.

In other frameworks, this problem is easily solved via bounded queue channels where send(..) will actually block if the channel is full. In Akka send never blocks, but it also doesn't mean that the message was enqueued, just a promise, so the mechanism in discussion IMHO provides a fairly robust way of monitoring the state of mailboxes without introducing any blocking and/or promises that could not be kept ;). 

> . . .can send it with ActorRef as well

Right, and that's what wasn't making much sense to me - sending Identify requests to something that's been identified already, hence the Status message comment. 

> . . .let the consumer send a request for more messages to the producer

True, and while a common pattern I never liked it myself since it splits throttling and flow control responsibilities between two components (producer and consumer) instead of one. Producer needs to send arbitrary number of messages, then lay dormant hoping the consumer sends a requests asking for more. What if Consumer is not the asking type? Does the Producer have to know the type of Consumer it deals with? This is why I liked this pre-defined message type strategy. As a producer I don't have to know anything about the consumer (send RequestForStatus type message and receive Status), just as Consumer needs to know nothing about the producer (don't ask since producer already knows). 

Cheers

Oleg