an other use case for akka custer
94 views
Skip to first unread message
bruno darrigues
Jun 4, 2012, 5:11:00 PM6/4/12
to Akka User List
Hi all
At business level, locking distributed (structured) data is a key
point for performance. based on akka, i am trying to convert
distributed lock in object actors distributed on cluster nodes and use
the queued mailbox to avoid lock.
so each nodes have to manage a range of objectid and the clients have
just to locate the node where the object belong. To make this magic
glue, i use consistent hash map to put on the same circle the node
and the objectid (based on this article
http://weblogs.java.net/blog/tomwhite/archive/2007/11/consistent_hash.html).
back to akka cluster, when a node join a cluster, the leader put the
incomer node in the circle and recalcuate a range plan to each node
and send the new range plan to each nodes. when a range policy change
for one node, it has to migrate all objects wich are not in the new
range to the new incoming node.
what do ou think of this kind of use case ?
Regards
Bruno
At business level, locking distributed (structured) data is a key
point for performance. based on akka, i am trying to convert
distributed lock in object actors distributed on cluster nodes and use
the queued mailbox to avoid lock.
so each nodes have to manage a range of objectid and the clients have
just to locate the node where the object belong. To make this magic
glue, i use consistent hash map to put on the same circle the node
and the objectid (based on this article
http://weblogs.java.net/blog/tomwhite/archive/2007/11/consistent_hash.html).
back to akka cluster, when a node join a cluster, the leader put the
incomer node in the circle and recalcuate a range plan to each node
and send the new range plan to each nodes. when a range policy change
for one node, it has to migrate all objects wich are not in the new
range to the new incoming node.
what do ou think of this kind of use case ?
Regards
Bruno
Patrik Nordwall
Jun 6, 2012, 1:55:16 AM6/6/12
to akka...@googlegroups.com
cluster for doing this?
Have you seen the consistent hash in akka?
https://github.com/akka/akka/blob/master/akka-actor/src/main/scala/akka/routing/ConsistentHash.scala
>
> Regards
> Bruno
>
> --
> You received this message because you are subscribed to the Google Groups "Akka User List" group.
> To post to this group, send email to akka...@googlegroups.com.
> To unsubscribe from this group, send email to akka-user+...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/akka-user?hl=en.
>
--
Patrik Nordwall
Typesafe - The software stack for applications that scale
Twitter: patriknw
bruno darrigues
Jun 6, 2012, 7:46:39 AM6/6/12
to Akka User List
Hello Patrick,
thk you for the link but i think it doesn t fit my problems.
to simply the use case, you 'll have to mange all numbers over the
cluster. If you want to ensure unicity, you have to use a router wich
create all uniques numbers over the cluster, but the router is still a
spof (even with supervision) the node wich hold the router could be
overcloked loll), so you decide to use 2 routers, one for odd numbers
and an other for evens.Based on an easy policy, process at business
level could easly select the right router. But the previous router
exist and when the second is added, the first one have to know wich
number stayed and wich have to migrate to the new even number router.
in the same way, if the new even router stopped, the odd router could
manage all numbers. And so on you can addd more and more routers (even
mod n, odd mod n) to let clients (external business event) have more
sharding over routers entries.
In resume, what do you think something like routerPolicy for shardings
between routers entries based on consistentHashMap per example.
clients could subscribe when the policy changed, node could say "i am
full at 80" and automaticaly the over traffic could be migrate
transparently to his neighbours.
is it a stupid feature ?
Regards
Bruno
On 6 juin, 07:55, Patrik Nordwall <patrik.nordw...@gmail.com> wrote:
> On Mon, Jun 4, 2012 at 11:11 PM, bruno darrigues
>
>
>
>
>
>
>
>
>
> <bruno.darrig...@gmail.com> wrote:
> > Hi all
>
> > At business level, locking distributed (structured) data is a key
> > point for performance. based on akka, i am trying to convert
> > distributed lock in object actors distributed on cluster nodes and use
> > the queued mailbox to avoid lock.
>
> > so each nodes have to manage a range of objectid and the clients have
> > just to locate the node where the object belong. To make this magic
> > glue, i use consistent hash map to put on the same circle the node
> > and the objectid (based on this article
> >http://weblogs.java.net/blog/tomwhite/archive/2007/11/consistent_hash...).
thk you for the link but i think it doesn t fit my problems.
to simply the use case, you 'll have to mange all numbers over the
cluster. If you want to ensure unicity, you have to use a router wich
create all uniques numbers over the cluster, but the router is still a
spof (even with supervision) the node wich hold the router could be
overcloked loll), so you decide to use 2 routers, one for odd numbers
and an other for evens.Based on an easy policy, process at business
level could easly select the right router. But the previous router
exist and when the second is added, the first one have to know wich
number stayed and wich have to migrate to the new even number router.
in the same way, if the new even router stopped, the odd router could
manage all numbers. And so on you can addd more and more routers (even
mod n, odd mod n) to let clients (external business event) have more
sharding over routers entries.
In resume, what do you think something like routerPolicy for shardings
between routers entries based on consistentHashMap per example.
clients could subscribe when the policy changed, node could say "i am
full at 80" and automaticaly the over traffic could be migrate
transparently to his neighbours.
is it a stupid feature ?
Regards
Bruno
On 6 juin, 07:55, Patrik Nordwall <patrik.nordw...@gmail.com> wrote:
> On Mon, Jun 4, 2012 at 11:11 PM, bruno darrigues
>
>
>
>
>
>
>
>
>
> <bruno.darrig...@gmail.com> wrote:
> > Hi all
>
> > At business level, locking distributed (structured) data is a key
> > point for performance. based on akka, i am trying to convert
> > distributed lock in object actors distributed on cluster nodes and use
> > the queued mailbox to avoid lock.
>
> > so each nodes have to manage a range of objectid and the clients have
> > just to locate the node where the object belong. To make this magic
> > glue, i use consistent hash map to put on the same circle the node
> > and the objectid (based on this article
>
> > back to akka cluster, when a node join a cluster, the leader put the
> > incomer node in the circle and recalcuate a range plan to each node
> > and send the new range plan to each nodes. when a range policy change
> > for one node, it has to migrate all objects wich are not in the new
> > range to the new incoming node.
>
> > what do ou think of this kind of use case ?
>
> Sounds good. Do you have anything in mind that is missing in akka
> cluster for doing this?
>
> Have you seen the consistent hash in akka?https://github.com/akka/akka/blob/master/akka-actor/src/main/scala/ak...
> > back to akka cluster, when a node join a cluster, the leader put the
> > incomer node in the circle and recalcuate a range plan to each node
> > and send the new range plan to each nodes. when a range policy change
> > for one node, it has to migrate all objects wich are not in the new
> > range to the new incoming node.
>
> > what do ou think of this kind of use case ?
>
> Sounds good. Do you have anything in mind that is missing in akka
> cluster for doing this?
>
>
>
>
> > Regards
> > Bruno
>
> > --
> > You received this message because you are subscribed to the Google Groups "Akka User List" group.
> > To post to this group, send email to akka...@googlegroups.com.
> > To unsubscribe from this group, send email to akka-user+...@googlegroups.com.
> > For more options, visit this group athttp://groups.google.com/group/akka-user?hl=en.
>
>
> > Regards
> > Bruno
>
> > --
> > You received this message because you are subscribed to the Google Groups "Akka User List" group.
> > To post to this group, send email to akka...@googlegroups.com.
> > To unsubscribe from this group, send email to akka-user+...@googlegroups.com.
Patrik Nordwall
Jun 6, 2012, 9:16:25 AM6/6/12
to akka...@googlegroups.com
Cluster aware routers is one of the first thing we will work on when we have the basic cluster membership in place. We have one ticket for consistent hashing router. Your ideas might fit into that.
http://www.assembla.com/spaces/akka/tickets/944
/Patrik
http://www.assembla.com/spaces/akka/tickets/944
/Patrik
Reply all
Reply to author
Forward
0 new messages