Akka 1.3.1, create frequently a lot of actors, best practices
Alex
as I've writeen before I work on a project wich uses OpenStreetMap
data.
I create a map with 10k-20k nodes. Each node is an actor and serves as
a possible source/target. I have to create frequently (10 times per
second) actors for each possible source target combination. The
created actors communicate with the node actors and are destroyed
after few seconds.
If I have 10k nodes, I have to create 100 millions actors each time
(600 millions actors per second).
1. Is something like that possible on a single machine?
2. How should I do this?
Cheers,
Alex
√iktor Ҡlang
what were the other possible solutions that were discarded in favor of this?
Cheers,
√
> --
> You received this message because you are subscribed to the Google Groups "Akka User List" group.
> To post to this group, send email to akka...@googlegroups.com.
> To unsubscribe from this group, send email to akka-user+...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/akka-user?hl=en.
>
--
Viktor Klang
Akka Tech Lead
Typesafe - The software stack for applications that scale
Twitter: @viktorklang
Jonas Bonér
I have a hard time believing you could usefully serve 600 million
actors on a single machine.
Try it :-)
> 2. How should I do this?
I'd put a load balancer in front and fan out to N physical nodes. N is
defined by your SLA. You need a real-world like bench.
>
> Cheers,
> Alex
>
> --
> You received this message because you are subscribed to the Google Groups "Akka User List" group.
> To post to this group, send email to akka...@googlegroups.com.
> To unsubscribe from this group, send email to akka-user+...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/akka-user?hl=en.
>
--
Jonas Bonér
CTO
Typesafe - The software stack for applications that scale
Phone: +46 733 777 123
Twitter: @jboner
Alex
unfortunately there are/were no other solutions.
I have a lot of source/destination nodes and I have to create actors
for each possible source/destination combination. I could use smaller
maps with less nodes. But how many nodes are ok for this approach?
Cheers,
Alex
On Feb 10, 10:51 am, √iktor Ҡlang <viktor.kl...@gmail.com> wrote:
> Hi Alex,
>
> what were the other possible solutions that were discarded in favor of this?
>
> Cheers,
> √
>
>
>
>
>
>
>
>
>
> On Fri, Feb 10, 2012 at 10:47 AM, Alex <bertram.a...@gmail.com> wrote:
> > Hi,
>
> > as I've writeen before I work on a project wich uses OpenStreetMap
> > data.
>
> > I create a map with 10k-20k nodes. Each node is an actor and serves as
> > a possible source/target. I have to create frequently (10 times per
> > second) actors for each possible source target combination. The
> > created actors communicate with the node actors and are destroyed
> > after few seconds.
>
> > If I have 10k nodes, I have to create 100 millions actors each time
> > (600 millions actors per second).
>
> > 1. Is something like that possible on a single machine?
> > 2. How should I do this?
>
> > Cheers,
> > Alex
>
> > --
> > You received this message because you are subscribed to the Google Groups "Akka User List" group.
> > To post to this group, send email to akka...@googlegroups.com.
> > To unsubscribe from this group, send email to akka-user+...@googlegroups.com.
Alex
On Feb 10, 10:52 am, Jonas Bonér <jo...@jonasboner.com> wrote:
> > Hi,
>
> > as I've writeen before I work on a project wich uses OpenStreetMap
> > data.
>
> > I create a map with 10k-20k nodes. Each node is an actor and serves as
> > a possible source/target. I have to create frequently (10 times per
> > second) actors for each possible source target combination. The
> > created actors communicate with the node actors and are destroyed
> > after few seconds.
>
> > If I have 10k nodes, I have to create 100 millions actors each time
> > (600 millions actors per second).
>
> > 1. Is something like that possible on a single machine?
>
> I have a hard time believing you could usefully serve 600 million
> actors on a single machine.
> Try it :-)
>
> > 2. How should I do this?
>
> I'd put a load balancer in front and fan out to N physical nodes. N is
> defined by your SLA. You need a real-world like bench.
>
>
>
> > Cheers,
> > Alex
>
> > --
> > You received this message because you are subscribed to the Google Groups "Akka User List" group.
> > To post to this group, send email to akka...@googlegroups.com.
> > To unsubscribe from this group, send email to akka-user+...@googlegroups.com.
√iktor Ҡlang
On Fri, Feb 10, 2012 at 11:27 AM, Alex <bertra...@gmail.com> wrote:
> Hi Viktor,
>
> unfortunately there are/were no other solutions.
> I have a lot of source/destination nodes and I have to create actors
> for each possible source/destination combination. I could use smaller
> maps with less nodes. But how many nodes are ok for this approach?
Why do they need to be recreated 6 times per second? Why not just send
a rebind-message to each? Or something like that?
With the current memory overhead I think you'd have to have in excess
of 36GB just for the actors. But more likely 48GB RAM. And I don't
think you'd be able to initialize all of them and do any meaningful
work within 167ms.
Cheers,
√
> For more options, visit this group at http://groups.google.com/group/akka-user?hl=en.
Alex
> > Hi Viktor,
>
> > unfortunately there are/were no other solutions.
> > I have a lot of source/destination nodes and I have to create actors
> > for each possible source/destination combination. I could use smaller
> > maps with less nodes. But how many nodes are ok for this approach?
>
> Why do they need to be recreated 6 times per second? Why not just send
> a rebind-message to each? Or something like that?
pool an reuse idle actors.
I try to implement an ant colony optimization algorithm (http://
en.wikipedia.org/wiki/Ant_colony_optimization_algorithms) for vehicle
routing problem with real world data.
> With the current memory overhead I think you'd have to have in excess
> of 36GB just for the actors. But more likely 48GB RAM. And I don't
> think you'd be able to initialize all of them and do any meaningful
> work within 167ms.
Chhers,
Alex
Alex
let's assume, I would reduce the amount of source/destination nodes to
100. Then I have to create (or in case of an actor pool reuse) 10000
actors per creating act and 60000 actors per second. How should I
realize this?
Someone mentioned in an older post, it makes sense to create an actor
tree. Should I follow this approach or should I use Actor.actorOf to
create all actors?
Cheers,
Alex
√iktor Ҡlang
I think you'll need to experiment and see what works best for your use-case.
Cheers,
√
> For more options, visit this group at http://groups.google.com/group/akka-user?hl=en.
Patrik Nordwall
/Patrik