Doing a broadcast to multiple actors

[Ian Holsman]

Hi.

I was searching around, and couldn't find a easy way.. So I thought I'd ask.

I want to be able to send a message to a group of actors (10-60k of them) so I can do a simulation, like a discrete time based thing. Besides from a 'for' loop is there a better way of doing this? I was thinking of using a latch or gate-type mechanism but that seems a bit heavy to me.

---
Ian Holsman - 703 879-3128

I saw the angel in the marble and carved until I set him free -- Michelangelo

[√iktor Ҡlang]

What's wrong with the loop?

> --
> You received this message because you are subscribed to the Google Groups "Akka User List" group.
> To post to this group, send email to akka...@googlegroups.com.
> To unsubscribe from this group, send email to akka-user+...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/akka-user?hl=en.
>

[Ian Holsman]

there is nothing specifically wrong with it per se, I was just trying to get the actors to all receive the message closer to the same time,  and thought there may be a api to do a 'bulk send' or something that I was missing.

I'm trying to write some simulation software where each actor acts independently (similar to santa-fe swarm project if you've heard of that), and didn't want the 'first' actor to get an advantage over the 'last' one.. which may be a factor if this thing scales by an order of magnitude..

--
Ian Holsman
I...@Holsman.net

PH: +1-703 879-3128 AOLIM: ianholsman Skype:iholsman

People of accomplishment rarely sat back & let things happen to them. They went out & happened to things. Leonardo Da Vinci

[√iktor Ҡlang]

Unless you have equal amounts of cores as you have actors you'll never achieve what you want. Putting a message into the mailbox and when that message is processed is completely different things.

If you need synchronizity you shouldn't use actors. Perhaps try Futures instead?

V

[Peter Veentjer]

I think actors are more expensive than latches.

You need to deal with the overhead message-invocation creation, all kinds of locks used for dispatcher, actor itself etc.

Akka is not designed or meant to compete on that level. 

The for loop for sending of course could be parallelized. We have 'broadcast' functionality in place when the Routing is used, so you can broadcast to a bunch of actors. But currently this is not designed to deal with such a large number of recipients. So if it really is important, what you could do, is to create your own ActorRef that can be tailored made for this purpose. Perhaps even using the FJ framework of Java 7 to parallelize the broadcast.

If you want I can show you some pointers how to do it.

[√iktor Ҡlang]

On Sat, Aug 6, 2011 at 10:40 AM, Peter Veentjer <alarm...@gmail.com> wrote:

I think actors are more expensive than latches.

You need to deal with the overhead message-invocation creation, all kinds of locks used for dispatcher, actor itself etc.

Akka is not designed or meant to compete on that level. 

The for loop for sending of course could be parallelized. We have 'broadcast' functionality in place when the Routing is used, so you can broadcast to a bunch of actors. But currently this is not designed to deal with such a large number of recipients. So if it really is important, what you could do, is to create your own ActorRef that can be tailored made for this purpose. Perhaps even using the FJ framework of Java 7 to parallelize the broadcast.

As I said, that would be completely pointless.
 

If you want I can show you some pointers how to do it.

On Sat, Aug 6, 2011 at 6:52 AM, Ian Holsman <i...@holsman.net> wrote:

Hi.

I was searching around, and couldn't find a easy way.. So I thought I'd ask.

I want to be able to send a message to a group of actors (10-60k of them) so I can do a simulation, like a discrete time based thing. Besides from a 'for' loop is there a better way of doing this? I was thinking of using a latch or gate-type mechanism but that seems a bit heavy to me.

---
Ian Holsman - 703 879-3128

I saw the angel in the marble and carved until I set him free -- Michelangelo

--
You received this message because you are subscribed to the Google Groups "Akka User List" group.
To post to this group, send email to akka...@googlegroups.com.
To unsubscribe from this group, send email to akka-user+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/akka-user?hl=en.

--
You received this message because you are subscribed to the Google Groups "Akka User List" group.
To post to this group, send email to akka...@googlegroups.com.
To unsubscribe from this group, send email to akka-user+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/akka-user?hl=en.

--
Viktor Klang

Akka Tech Lead

Typesafe - Enterprise-Grade Scala from the Experts

Twitter: @viktorklang

[Peter Veentjer]

2011/8/6 √iktor Ҡlang <viktor...@gmail.com>

On Sat, Aug 6, 2011 at 10:40 AM, Peter Veentjer <alarm...@gmail.com> wrote:

I think actors are more expensive than latches.

You need to deal with the overhead message-invocation creation, all kinds of locks used for dispatcher, actor itself etc.

Akka is not designed or meant to compete on that level. 

The for loop for sending of course could be parallelized. We have 'broadcast' functionality in place when the Routing is used, so you can broadcast to a bunch of actors. But currently this is not designed to deal with such a large number of recipients. So if it really is important, what you could do, is to create your own ActorRef that can be tailored made for this purpose. Perhaps even using the FJ framework of Java 7 to parallelize the broadcast.

As I said, that would be completely pointless.

Thank you!

It all depends on what the goal is. If the sending thread should not be busy too long for sending messages to all the recipient, parallelizing the send of all messages is going to deliver a scalability improvement.

And in case of sending 50k messages, I certainly is something worth considering.

[Ian Holsman]

Thanks for the advice guys... I'll check out the FJ stuff in Java 7, and go back to latches.

regards

Ian

--
Ian Holsman
I...@Holsman.net
PH: +1-703 879-3128 AOLIM: ianholsman Skype:iholsman

To know recursion, you must first know recursion.

[Peter Veentjer]

He Ian,

I think with some care, you even could create a latch that supports some parallelisation to releases everyone in the waitset. Especially for 50k listeners, this could be something worth to experiment with. Can also be based on the FJ framework I think.

[√iktor Ҡlang]

On Sat, Aug 6, 2011 at 11:45 AM, Peter Veentjer <alarm...@gmail.com> wrote:

2011/8/6 √iktor Ҡlang <viktor...@gmail.com>

On Sat, Aug 6, 2011 at 10:40 AM, Peter Veentjer <alarm...@gmail.com> wrote:

I think actors are more expensive than latches.

You need to deal with the overhead message-invocation creation, all kinds of locks used for dispatcher, actor itself etc.

Akka is not designed or meant to compete on that level. 

The for loop for sending of course could be parallelized. We have 'broadcast' functionality in place when the Routing is used, so you can broadcast to a bunch of actors. But currently this is not designed to deal with such a large number of recipients. So if it really is important, what you could do, is to create your own ActorRef that can be tailored made for this purpose. Perhaps even using the FJ framework of Java 7 to parallelize the broadcast.

As I said, that would be completely pointless.

Thank you!

It all depends on what the goal is. If the sending thread should not be busy too long for sending messages to all the recipient, parallelizing the send of all messages is going to deliver a scalability improvement.

And in case of sending 50k messages, I certainly is something worth considering.

Adding an entry into a ConcurrentLinkedQueue is in the nanosecond space.

I did some quick bench on my machine for 1 MILLION sends, it took between 150ms and 215ms (1 million queues (CLQ), adding the same, preallocated message to them)

For 50000 it took between 1 and 5 milliseconds (on my machine, a laptop, without cable plugged in)

Also, the point is quite moot since you cannot schedule them all to run at the same time _anyway_. Message sent != Message started to get processed

[√iktor Ҡlang]

On Sat, Aug 6, 2011 at 12:54 PM, Ian Holsman <i...@holsman.net> wrote:

Thanks for the advice guys... I'll check out the FJ stuff in Java 7, and go back to latches.

I still don't understand why it's important for all actors to start processing the message at the same time, they need a thread to process the message anyway, and you don't have 50000 cores...

If you actually have a viable reason to make sure that no one starts before everyone has got the message you can suspend the actors, send the messages and then resume all actors.

for(actor <- actors)
actor.dispatcher.suspend(actor)

for(actor <- actors)
actor ! message

for(actor <- actors)
actor.dispatcher.resume(actor)
 

But that means you'll make it O(N3)

--

[Peter Veentjer]

2011/8/6 √iktor Ҡlang <viktor...@gmail.com>

On Sat, Aug 6, 2011 at 11:45 AM, Peter Veentjer <alarm...@gmail.com> wrote:

2011/8/6 √iktor Ҡlang <viktor...@gmail.com>

On Sat, Aug 6, 2011 at 10:40 AM, Peter Veentjer <alarm...@gmail.com> wrote:

I think actors are more expensive than latches.

You need to deal with the overhead message-invocation creation, all kinds of locks used for dispatcher, actor itself etc.

Akka is not designed or meant to compete on that level. 

The for loop for sending of course could be parallelized. We have 'broadcast' functionality in place when the Routing is used, so you can broadcast to a bunch of actors. But currently this is not designed to deal with such a large number of recipients. So if it really is important, what you could do, is to create your own ActorRef that can be tailored made for this purpose. Perhaps even using the FJ framework of Java 7 to parallelize the broadcast.

As I said, that would be completely pointless.

Thank you!

It all depends on what the goal is. If the sending thread should not be busy too long for sending messages to all the recipient, parallelizing the send of all messages is going to deliver a scalability improvement.

And in case of sending 50k messages, I certainly is something worth considering.

Adding an entry into a ConcurrentLinkedQueue is in the nanosecond space.

Try to do it concurrently and performance will drop singe the gc will kill scalability and performance.

Afaik you need to create at least 2 objects, the linked node in the queue and the message invocation.

http://multiverse.codehaus.org/benchmark-misc.html

As the benchmark shows, cleaning millions of objects per second kills scalability..  And this is only use a true 8 core machine.. I guess when you go to the 32/64 core machines, performance drops even more.

[Ian Holsman]

On Aug 6, 2011, at 9:37 PM, √iktor Ҡlang wrote:

On Sat, Aug 6, 2011 at 12:54 PM, Ian Holsman <i...@holsman.net> wrote:

Thanks for the advice guys... I'll check out the FJ stuff in Java 7, and go back to latches.

I still don't understand why it's important for all actors to start processing the message at the same time, they need a thread to process the message anyway, and you don't have 50000 cores...

looking at the simulation i'm trying to solve, I don't need them to be exactly the same time. I'm more concerned that the first one in the loop always gets preference. 

Imagine a game where you have thousands of people looking for a couple of treasures. and every 'tick' they are allowed to move.

you don't want it so that the first one in the loop will always get the treasure as it gets there first all the time.

--

Ian Holsman

I...@Holsman.net

sometimes we watch the TV. sometimes the TV watches us

[√iktor Ҡlang]

On Sat, Aug 6, 2011 at 1:45 PM, Peter Veentjer <alarm...@gmail.com> wrote:

2011/8/6 √iktor Ҡlang <viktor...@gmail.com>

On Sat, Aug 6, 2011 at 11:45 AM, Peter Veentjer <alarm...@gmail.com> wrote:

2011/8/6 √iktor Ҡlang <viktor...@gmail.com>

On Sat, Aug 6, 2011 at 10:40 AM, Peter Veentjer <alarm...@gmail.com> wrote:

I think actors are more expensive than latches.

You need to deal with the overhead message-invocation creation, all kinds of locks used for dispatcher, actor itself etc.

Akka is not designed or meant to compete on that level. 

The for loop for sending of course could be parallelized. We have 'broadcast' functionality in place when the Routing is used, so you can broadcast to a bunch of actors. But currently this is not designed to deal with such a large number of recipients. So if it really is important, what you could do, is to create your own ActorRef that can be tailored made for this purpose. Perhaps even using the FJ framework of Java 7 to parallelize the broadcast.

As I said, that would be completely pointless.

Thank you!

It all depends on what the goal is. If the sending thread should not be busy too long for sending messages to all the recipient, parallelizing the send of all messages is going to deliver a scalability improvement.

And in case of sending 50k messages, I certainly is something worth considering.

Adding an entry into a ConcurrentLinkedQueue is in the nanosecond space.

Try to do it concurrently and performance will drop singe the gc will kill scalability and performance.

Depends on contention and choice of queue, 1000 threads writing to different queues (remember there was one million queues) is quite different from 1000 threads writing to 1 queue.
 

Afaik you need to create at least 2 objects, the linked node in the queue and the message invocation.

http://multiverse.codehaus.org/benchmark-misc.html

As the benchmark shows, cleaning millions of objects per second kills scalability..  And this is only use a true 8 core machine.. I guess when you go to the 32/64 core machines, performance drops even more.

Yup, but then you could always switch to ArrayBlockingQueue so you won't need any allocations per send.
 

 

I did some quick bench on my machine for 1 MILLION sends, it took between 150ms and 215ms (1 million queues (CLQ), adding the same, preallocated message to them)

For 50000 it took between 1 and 5 milliseconds (on my machine, a laptop, without cable plugged in)

Also, the point is quite moot since you cannot schedule them all to run at the same time _anyway_. Message sent != Message started to get processed

This still holds true.

 

[√iktor Ҡlang]

On Sat, Aug 6, 2011 at 1:50 PM, Ian Holsman <i...@holsman.net> wrote:

On Aug 6, 2011, at 9:37 PM, √iktor Ҡlang wrote:

On Sat, Aug 6, 2011 at 12:54 PM, Ian Holsman <i...@holsman.net> wrote:

Thanks for the advice guys... I'll check out the FJ stuff in Java 7, and go back to latches.

I still don't understand why it's important for all actors to start processing the message at the same time, they need a thread to process the message anyway, and you don't have 50000 cores...

looking at the simulation i'm trying to solve, I don't need them to be exactly the same time. I'm more concerned that the first one in the loop always gets preference. 

Then randomize which actor you'll start sending to.
 

Imagine a game where you have thousands of people looking for a couple of treasures. and every 'tick' they are allowed to move.

you don't want it so that the first one in the loop will always get the treasure as it gets there first all the time.

If you have 1 source of input, you can always randomize, and if you have several, you already have non-determinism (multiple senders to the same target)
 

[Peter Veentjer]

Depends on contention and choice of queue, 1000 threads writing to different queues (remember there was one million queues) is quite different from 1000 threads writing to 1 queue.

Even with 1 thread per queue without any form of contention, this is not going to scale since gc won't scale. That is the point.

 

 

Afaik you need to create at least 2 objects, the linked node in the queue and the message invocation.

http://multiverse.codehaus.org/benchmark-misc.html

As the benchmark shows, cleaning millions of objects per second kills scalability..  And this is only use a true 8 core machine.. I guess when you go to the 32/64 core machines, performance drops even more.

Yup, but then you could always switch to ArrayBlockingQueue so you won't need any allocations per send.

Not with the queue itself, but in case of Akka, you are still stuck with the MessageInvocation object. 

[√iktor Ҡlang]

On Sat, Aug 6, 2011 at 2:07 PM, Peter Veentjer <alarm...@gmail.com> wrote:

Depends on contention and choice of queue, 1000 threads writing to different queues (remember there was one million queues) is quite different from 1000 threads writing to 1 queue.

Even with 1 thread per queue without any form of contention, this is not going to scale since gc won't scale. That is the point.

There is nothing to GC until the message is processed.
 

 

 

Afaik you need to create at least 2 objects, the linked node in the queue and the message invocation.

http://multiverse.codehaus.org/benchmark-misc.html

As the benchmark shows, cleaning millions of objects per second kills scalability..  And this is only use a true 8 core machine.. I guess when you go to the 32/64 core machines, performance drops even more.

Yup, but then you could always switch to ArrayBlockingQueue so you won't need any allocations per send.

Not with the queue itself, but in case of Akka, you are still stuck with the MessageInvocation object. 

Indeed. In the future we might optimize this by omitting it when there's no reply channel.
 

[Peter Veentjer]

2011/8/6 √iktor Ҡlang <viktor...@gmail.com>

On Sat, Aug 6, 2011 at 2:07 PM, Peter Veentjer <alarm...@gmail.com> wrote:

Depends on contention and choice of queue, 1000 threads writing to different queues (remember there was one million queues) is quite different from 1000 threads writing to 1 queue.

Even with 1 thread per queue without any form of contention, this is not going to scale since gc won't scale. That is the point.

There is nothing to GC until the message is processed.

I give up.

[Patrick Logan]

On Saturday, August 6, 2011 4:50:09 AM UTC-7, Ian Holsman wrote:

 

looking at the simulation i'm trying to solve, I don't need them to be exactly the same time. I'm more concerned that the first one in the loop always gets preference. 

Imagine a game where you have thousands of people looking for a couple of treasures. and every 'tick' they are allowed to move.

you don't want it so that the first one in the loop will always get the treasure as it gets there first all the time.

Implementations of discrete event simulations should almost always distinguish between the "system clock(s)" of the computer(s) running the simulation and the "simulation clock" of the world being simulations. Unless you have a very good reason to conflate the two, you probably want to explicitly program the concept of a clock in the simulated world itself.

The same is almost always true of concepts like "fairness" - you probably do not want to conflate the fairness of the concurrency mechanisms used to implement a concurrent simulator with the "fairness" concepts of the world itself. To follow on to the example of a treasure hunt in a fantasy game, suppose one of the characters gains and applies some special capability to grab the treasure - you probably will be better off if you do not have to change how you've used your programming language's concurrency mechanisms just to implement some new kind of magic in your game world.

Just saying... there be dragons.

-Patrick