Managing Millions of Actors (Lookup).

[Bernd Hofner]

Hello,

I have about 25 million objects, that can be uniquely identified by a
10 digit number.
I need to collect and process a bunch of information for each of these
objects.
After everything is processed, I need to retrieve the final state for
each of my 25 Mio objects (sorting them into "good" and "bad").

Is it a "good idea" to create an Actor for each of these objects?

I need to look up each Actor several times (let's say 10 times).
Or, to put in other words: I have about 10 independend data streams,
potentially sorted by my 10 digit object id and need to feed the
relevant portions of these streams into the corresponding data object
- which will change its state accordingly.

Will the lookup for each Actor (via its ActorPath with my unique ID as
a name) be efficient?
Does it make sense to place the ActorReferences into a separate
datastructure (index-like) to find them quicker?

Or would it be a better idea to keep my objects passive and just
create a few workers that lookup the "right" data object (let's say
from a Hazelcast Map) when a message comes in, modify it with the
incoming message and place it back in the Map?


Best regards,

Bernd

[√iktor Ҡlang]

I'd probably use a Router with a consistent-hashing Rouer and let the routees each handle a slice of Ids and keep the book-keeping internal inside the routees. Then you can use Broadcast to tell them all to report in their status.

Cheers,

√

 

Best regards,

Bernd

--
You received this message because you are subscribed to the Google Groups "Akka User List" group.
To post to this group, send email to akka...@googlegroups.com.
To unsubscribe from this group, send email to akka-user+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/akka-user?hl=en.

--
Viktor Klang

Akka Tech Lead

Typesafe - The software stack for applications that scale

Twitter: @viktorklang

[Jonas Bonér]

2012/4/13 √iktor Ҡlang <viktor...@gmail.com>

On Fri, Apr 13, 2012 at 4:57 PM, Bernd Hofner <bernd....@gmx.net> wrote:

Hello,

I have about 25 million objects, that can be uniquely identified by a
10 digit number.
I need to collect and process a bunch of information for each of these
objects.
After everything is processed, I need to retrieve the final state for
each of my 25 Mio objects (sorting them into "good" and "bad").

Is it a "good idea" to create an Actor for each of these objects?

I need to look up each Actor several times (let's say 10 times).
Or, to put in other words: I have about 10 independend data streams,
potentially sorted by my 10 digit object id and need to feed the
relevant portions of these streams into the corresponding data object
- which will change its state accordingly.

Will the lookup for each Actor (via its ActorPath with my unique ID as
a name) be efficient?
Does it make sense to place the ActorReferences into a separate
datastructure (index-like) to find them quicker?

Or would it be a better idea to keep my objects passive and just
create a few workers that lookup the "right" data object (let's say
from a Hazelcast Map) when a message comes in, modify it with the
incoming message and place it back in the Map?

I'd probably use a Router with a consistent-hashing Rouer and let the routees each handle a slice of Ids and keep the book-keeping internal inside the routees. Then you can use Broadcast to tell them all to report in their status.

Might be helpful: https://github.com/akka/akka/blob/149fbea5a7bef924834d7f21d06c88a75eb0f00b/akka-actor/src/main/scala/akka/routing/ConsistentHash.scala

 

Cheers,

√

 

Best regards,

Bernd

--
You received this message because you are subscribed to the Google Groups "Akka User List" group.
To post to this group, send email to akka...@googlegroups.com.
To unsubscribe from this group, send email to akka-user+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/akka-user?hl=en.

--
Viktor Klang

Akka Tech Lead

Typesafe - The software stack for applications that scale

Twitter: @viktorklang

--
You received this message because you are subscribed to the Google Groups "Akka User List" group.
To post to this group, send email to akka...@googlegroups.com.
To unsubscribe from this group, send email to akka-user+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/akka-user?hl=en.

--
Jonas Bonér
CTO

Typesafe - The software stack for applications that scale

Phone: +46 733 777 123
Twitter: @jboner

[Bernd Hofner]

Thanks for the tips!

Bernd

On 16 Apr., 12:24, Jonas Bonér <jo...@jonasboner.com> wrote:
> 2012/4/13 √iktor Ҡlang <viktor.kl...@gmail.com>

> Might be helpful:https://github.com/akka/akka/blob/149fbea5a7bef924834d7f21d06c88a75eb...

>
>
>
>
>
>
>
>
>
> > Cheers,
> > √
>
> >> Best regards,
>
> >> Bernd
>
> >> --
> >> You received this message because you are subscribed to the Google Groups
> >> "Akka User List" group.
> >> To post to this group, send email to akka...@googlegroups.com.
> >> To unsubscribe from this group, send email to
> >> akka-user+...@googlegroups.com.
> >> For more options, visit this group at
> >>http://groups.google.com/group/akka-user?hl=en.
>
> > --
> > Viktor Klang
>
> > Akka Tech Lead

> > Typesafe <http://www.typesafe.com/> - The software stack for applications