Is there a recommended max number of shard regions in a cluster?

[Richard Ney]

I have a application in development where in grouping collections of entities by the customer in a SaaS environment. Instead of creating a single sharding region for each entity type and mixing my customers actors I want to have each customer created get their own collection of shard regions for simplicity of control. This in the long run could result in 100s of shard regions getting created. So the question is am I heading down the path of creating a new anti-pattern for Akka?

[Patrik Nordwall]

Let's see if I understand. You call ClusterSharding(system).start, with unique typeName, dynamically when a new customer is created?

On Sun, Nov 20, 2016 at 10:37 PM, Richard Ney <kamisa...@gmail.com> wrote:

I have a application in development where in grouping collections of entities by the customer in a SaaS environment. Instead of creating a single sharding region for each entity type and mixing my customers actors I want to have each customer created get their own collection of shard regions for simplicity of control. This in the long run could result in 100s of shard regions getting created. So the question is am I heading down the path of creating a new anti-pattern for Akka?

--
>>>>>>>>>>      Read the docs: http://akka.io/docs/
>>>>>>>>>>      Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>>      Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups "Akka User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscribe@googlegroups.com.
To post to this group, send email to akka...@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

--

Patrik Nordwall
Akka Tech Lead
Lightbend -  Reactive apps on the JVM
Twitter: @patriknw

[Richard Ney]

The simple answer is yes. The more detailed answer is:

I have a cluster singleton for this specific role. It's responsible for listening for the creation of all new customers from the provisioning members in the cluster. Each of the cluster members associated with this specific role have a distributed pub/sub subscription for the creation of a new customer. When the cluster singleton gets a message that a new customer is provisioned it publishes a message out to the worker members to create 3 new sharding regions for that customer. Each of the cluster members makes the call for ClusterSharding(system).start for the three regions and each then replies back to the cluster singleton when they've completed the creation of the regions. After the cluster singleton gets the first response it then created a new actor for the new customer which on initialization makes three calls to ClusterSharding(system).startProxy to attach to the distributed regions for that customer. From that point onward the organization actors have responsibility for message handling for that customer and dispatching those messages to the various entity actors in each of the three child entity regions.

Richard Ney

On Friday, November 25, 2016 at 8:25:02 AM UTC-8, Patrik Nordwall wrote:

Let's see if I understand. You call ClusterSharding(system).start, with unique typeName, dynamically when a new customer is created?

On Sun, Nov 20, 2016 at 10:37 PM, Richard Ney <kamisa...@gmail.com> wrote:

I have a application in development where in grouping collections of entities by the customer in a SaaS environment. Instead of creating a single sharding region for each entity type and mixing my customers actors I want to have each customer created get their own collection of shard regions for simplicity of control. This in the long run could result in 100s of shard regions getting created. So the question is am I heading down the path of creating a new anti-pattern for Akka?

--
>>>>>>>>>>      Read the docs: http://akka.io/docs/
>>>>>>>>>>      Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>>      Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups "Akka User List" group.

To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+...@googlegroups.com.

To post to this group, send email to akka...@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

[Patrik Nordwall]

Ok, it should work.

Each entity type (customer) will have its own sharding coordinator and manage itself independent of other entity types. If you start to have many such (>100) I can imagine that there will be some peak loads when fail over of the coordinator node and such, but I have not tried.

It's important that you start it on all nodes.

Seems to be a lot of complexity and moving parts compared to running all together with same entity type.

/Patrik

[Justin du coeur]

On Sat, Nov 26, 2016 at 10:20 AM, Patrik Nordwall <patrik....@gmail.com> wrote:

Seems to be a lot of complexity and moving parts compared to running all together with same entity type.

Yeah, this is what I keep coming back to.  The architecture sounds like it could work, but I'm still not coming up with any way in which it seems *better* than simply prefixing each entity's ID with the customer ID.  I'm curious: what are the benefits you see in having separate shard regions for each one?

[Richard Ney]

Maybe I'm just missing something with addressing messages to actors in a shared sharding region. My original intention was to simplify the cases where I needed to send messages, start, or stop all actors associated with a customer. From people's experience is a call to Akka.system.actorSelection with a wildcard in the name is as performant as my segmented actor regions without the added complexity in deployment, management, and network overhead? Maybe I was seeing a problem that doesn't really exist.
 

[Justin du coeur]

Hmm.  I'll have to defer that one to others -- I haven't done much with wildcarding, so I can't speak to whether that's a significant issue or not...

--

>>>>>>>>>> Read the docs: http://akka.io/docs/
>>>>>>>>>> Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups "Akka User List" group.

To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscribe@googlegroups.com.

[Patrik Nordwall]

start message doesn't make sense, youmus know the identifiers and send va region anyway.

stop message would only be best effort, since another may be started right afterwards. Easiest if they passivate themselves after inactivity.

mån 28 nov. 2016 kl. 14:12 skrev Justin du coeur <jduc...@gmail.com>:

Hmm.  I'll have to defer that one to others -- I haven't done much with wildcarding, so I can't speak to whether that's a significant issue or not...

On Mon, Nov 28, 2016 at 2:19 AM, Richard Ney <kamisa...@gmail.com> wrote:

Maybe I'm just missing something with addressing messages to actors in a shared sharding region. My original intention was to simplify the cases where I needed to send messages, start, or stop all actors associated with a customer. From people's experience is a call to Akka.system.actorSelection with a wildcard in the name is as performant as my segmented actor regions without the added complexity in deployment, management, and network overhead? Maybe I was seeing a problem that doesn't really exist.
 
On Saturday, November 26, 2016 at 8:23:12 AM UTC-8, Justin du coeur wrote:

On Sat, Nov 26, 2016 at 10:20 AM, Patrik Nordwall <patrik....@gmail.com> wrote:

Seems to be a lot of complexity and moving parts compared to running all together with same entity type.

Yeah, this is what I keep coming back to.  The architecture sounds like it could work, but I'm still not coming up with any way in which it seems *better* than simply prefixing each entity's ID with the customer ID.  I'm curious: what are the benefits you see in having separate shard regions for each one?

--

>>>>>>>>>> Read the docs: http://akka.io/docs/
>>>>>>>>>> Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups "Akka User List" group.

To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+...@googlegroups.com.

To post to this group, send email to akka...@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

--
>>>>>>>>>> Read the docs: http://akka.io/docs/
>>>>>>>>>> Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups "Akka User List" group.

To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+...@googlegroups.com.