crossbar.io scalability features

[Iulian Nitescu]

Hi,

We are looking to build parts of our system with crossbar.io (mainly Python). We have played around with some examples and it looks great, so first I have to congratulate everyone working on crossbar.io - excellent initiative. My questions are highly related to this post: https://groups.google.com/forum/#!msg/autobahnws/RbYylAj_SZ4/Cq4bdC2s6VMJ.

What is the current state of the scalability features now? As the OP in the mentioned thread, we also need to be able to scale - basically worker + router load balancing (==> workers registering the same procedures). The scenario is to use Amazon Container Services and deploy Docker tasks with the crossbar.io stack on them. 

Is the feature set complete enough to allow running multiple workers and routers behind ELB? 

Is it possible to cluster workers (on separate instances) such that only one of them receives receives the message?

At the beginning of the project it is no problem if we have to use the trunk from git. There is also no problem with us managing and configuring the Docker containers, ELB etc. 

Thanks,

Iulian

[Alexander Gödde]

Hi Iulian!

Scalability features are under construction. There aren't yet any router-to-router features. This means that load balancing of workers is restricted to workers connecting to a single router. We're working on going beyond this, but unfortunately can't give a timeframe for this.

Regards,

Alex

[Iulian Nitescu]

Hi Alex,

Aha, right. So you are saying there is a way to configure workers into clusters behind a load balancer. Do these workers need a special configuration? For example is this bit of the documentation correct (http://crossbar.io/docs/Shared-Registrations/)?. Is there even need for a load balancer in that case? 

If I understand you correctly, it won't work if I put 2 router behind an LB? 

Thanks,

Iulian

[Tobias Oberstein]

Hi,

Crossbar.io _already_ is able to scale-up and scale-out application
components (container/guest workers). It fully supports shared
registrations. There is no LB involved, as Crossbar.io itself is acting
as the LB rgd the WAMP traffic routed to app components.

Crossbar.io (trunk) is also already able to scale-up Web services.

What's missing is scale-out for Web services, and both scale-up and
scale-out of actual WAMP routing (router workers).

When we have that, you will need a L4 LB in front of the Crossbar.io
nodes to (randomly) distribute incoming connection at the TCP level to
backend nodes.

Cheers,
/Tobias

Am 20.11.2015 um 15:30 schrieb Iulian Nitescu:
> Hi Alex,
>
> Aha, right. So you are saying there is a way to configure workers into
> clusters behind a load balancer. Do these workers need a special
> configuration? For example is this bit of the documentation correct
> (http://crossbar.io/docs/Shared-Registrations/)?. Is there even need for
> a load balancer in that case?
>
> If I understand you correctly, it won't work if I put 2 router behind an
> LB?
>
> Thanks,
> Iulian
>
> On Friday, November 20, 2015 at 3:19:59 PM UTC+1, Alexander Gödde wrote:
>
> Hi Iulian!
>
> Scalability features are under construction. There aren't yet any
> router-to-router features. This means that load balancing of workers
> is restricted to workers connecting to a single router. We're
> working on going beyond this, but unfortunately can't give a
> timeframe for this.
>
> Regards,
>
> Alex
>
> Am Freitag, 20. November 2015 12:59:45 UTC+1 schrieb Iulian Nitescu:
>
> Hi,
>
> We are looking to build parts of our system with crossbar.io

> <http://crossbar.io> (mainly Python). We have played around with

> some examples and it looks great, so first I have to
> congratulate everyone working on crossbar.io

> <http://crossbar.io> - excellent initiative. My questions are

> highly related to this post:
> https://groups.google.com/forum/#!msg/autobahnws/RbYylAj_SZ4/Cq4bdC2s6VMJ

> <https://groups.google.com/forum/#!msg/autobahnws/RbYylAj_SZ4/Cq4bdC2s6VMJ>.

>
> What is the current state of the scalability features now? As
> the OP in the mentioned thread, we also need to be able to scale
> - basically worker + router load balancing (==> workers
> registering the same procedures). The scenario is to use Amazon
> Container Services and deploy Docker tasks with the crossbar.io

> <http://crossbar.io> stack on them.

>
> Is the feature set complete enough to allow running multiple
> workers and routers behind ELB?
> Is it possible to cluster workers (on separate instances) such
> that only one of them receives receives the message?
>
> At the beginning of the project it is no problem if we have to
> use the trunk from git. There is also no problem with us
> managing and configuring the Docker containers, ELB etc.
>
> Thanks,
> Iulian
>

> --
> You received this message because you are subscribed to the Google
> Groups "Crossbar" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to crossbario+...@googlegroups.com
> <mailto:crossbario+...@googlegroups.com>.
> To post to this group, send email to cross...@googlegroups.com
> <mailto:cross...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/crossbario/a0f3276c-2f75-4ec9-838f-63dee0060572%40googlegroups.com
> <https://groups.google.com/d/msgid/crossbario/a0f3276c-2f75-4ec9-838f-63dee0060572%40googlegroups.com?utm_medium=email&utm_source=footer>.
> For more options, visit https://groups.google.com/d/optout.

[Iulian Nitescu]

Hi Tobias,

Thanks for the detailed explanation.

Do you have any best practices for redundancy of the router then? Until the scaling up/out is implemented. 

Regards,

Iulian 

> <mailto:crossbario+unsub...@googlegroups.com>.

[Tobias Oberstein]

Am 20.11.2015 um 16:02 schrieb Iulian Nitescu:
> Hi Tobias,
>
> Thanks for the detailed explanation.
>
> Do you have any best practices for redundancy of the router then? Until
> the scaling up/out is implemented.

HA (and scale-out) will require Crossbar.io nodes to connect to Crossbar
DevOps center. The latter will act as cluster monitor/manager. No, I
don't have a different approach - that's how it will work.

>
> Regards,
> Iulian
>
> On Friday, November 20, 2015 at 3:55:48 PM UTC+1, Tobias Oberstein wrote:
>
> Hi,
>
> Crossbar.io _already_ is able to scale-up and scale-out application
> components (container/guest workers). It fully supports shared
> registrations. There is no LB involved, as Crossbar.io itself is acting
> as the LB rgd the WAMP traffic routed to app components.
>
> Crossbar.io (trunk) is also already able to scale-up Web services.
>
> What's missing is scale-out for Web services, and both scale-up and
> scale-out of actual WAMP routing (router workers).
>
> When we have that, you will need a L4 LB in front of the Crossbar.io
> nodes to (randomly) distribute incoming connection at the TCP level to
> backend nodes.
>
> Cheers,
> /Tobias
>
>
> Am 20.11.2015 um 15:30 schrieb Iulian Nitescu:
> > Hi Alex,
> >
> > Aha, right. So you are saying there is a way to configure workers
> into
> > clusters behind a load balancer. Do these workers need a special
> > configuration? For example is this bit of the documentation correct
> > (http://crossbar.io/docs/Shared-Registrations/)

> <http://crossbar.io/docs/Shared-Registrations/)>?. Is there even

> > an email to crossbario+...@googlegroups.com <javascript:>
> > <mailto:crossbario+...@googlegroups.com <javascript:>>.

> > To post to this group, send email to cross...@googlegroups.com

> <javascript:>
> > <mailto:cross...@googlegroups.com <javascript:>>.

> > To view this discussion on the web visit
> >
> https://groups.google.com/d/msgid/crossbario/a0f3276c-2f75-4ec9-838f-63dee0060572%40googlegroups.com
> <https://groups.google.com/d/msgid/crossbario/a0f3276c-2f75-4ec9-838f-63dee0060572%40googlegroups.com>
>
> >

> <https://groups.google.com/d/msgid/crossbario/a0f3276c-2f75-4ec9-838f-63dee0060572%40googlegroups.com?utm_medium=email&utm_source=footer

> <https://groups.google.com/d/msgid/crossbario/a0f3276c-2f75-4ec9-838f-63dee0060572%40googlegroups.com?utm_medium=email&utm_source=footer>>.
>
> > For more options, visit https://groups.google.com/d/optout

> <https://groups.google.com/d/optout>.

>
> --
> You received this message because you are subscribed to the Google
> Groups "Crossbar" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to crossbario+...@googlegroups.com

> <mailto:crossbario+...@googlegroups.com>.

> To post to this group, send email to cross...@googlegroups.com
> <mailto:cross...@googlegroups.com>.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/crossbario/35e76134-0e82-4617-b3da-6cedee900c29%40googlegroups.com
> <https://groups.google.com/d/msgid/crossbario/35e76134-0e82-4617-b3da-6cedee900c29%40googlegroups.com?utm_medium=email&utm_source=footer>.

[Tim Keeler]

Really like the direction of the crossbar.io project!

I'd like to add my +1 for horizontal auto-scaling & HA. My use case is the same as lulian, I'd like to auto-scale EC2/Docker containers behind an ELB. This would be the preferred method instead of relying on a single router (or 2 w/HA). There was a SO comment that mentioned Redis would allow for auto-scalability. I support this approach as it allows for a more distributed architecture.

But if this is the only architecture, would the router run on one EC2 container and nodes run on several other EC2 containers?

Thanks,

Tim

>      > <mailto:crossbario+unsub...@googlegroups.com <javascript:>>.

>      > To post to this group, send email to cross...@googlegroups.com
>     <javascript:>
>      > <mailto:cross...@googlegroups.com <javascript:>>.
>      > To view this discussion on the web visit
>      >
>     https://groups.google.com/d/msgid/crossbario/a0f3276c-2f75-4ec9-838f-63dee0060572%40googlegroups.com
>     <https://groups.google.com/d/msgid/crossbario/a0f3276c-2f75-4ec9-838f-63dee0060572%40googlegroups.com>
>
>      >
>     <https://groups.google.com/d/msgid/crossbario/a0f3276c-2f75-4ec9-838f-63dee0060572%40googlegroups.com?utm_medium=email&utm_source=footer
>     <https://groups.google.com/d/msgid/crossbario/a0f3276c-2f75-4ec9-838f-63dee0060572%40googlegroups.com?utm_medium=email&utm_source=footer>>.
>
>      > For more options, visit https://groups.google.com/d/optout
>     <https://groups.google.com/d/optout>.
>
> --
> You received this message because you are subscribed to the Google
> Groups "Crossbar" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to crossbario+...@googlegroups.com

> <mailto:crossbario+unsub...@googlegroups.com>.

[Alexander Gödde]

Hi Tim!

Crossbar.io is agnostic to where application components run. At the moment, with routers not talking to each other, to scale up you'd indeed run the router on an EC2 instance and application components/workers on other instances.

Regards,

Alex

[Iulian Nitescu]

Hi Alex, Tobias,

I have another question. Given that the routers store the registrations in memory, what would happen in the following situation:

caller_1 ------------ [router_1] -------------- callee_1 f()

                                    |

                                    |

caller_2 ------------ [router_2] -------------- callee_2 g()

callee_1 registered a function f(), which caller_1 calls repeatedly. 

What happens when router_1 goes down? 

- Do I need to write code on callee_1s side to detect this and register f() again (which would presumably register on router_2 now)? or .. 

- Does the Autobahn framework automatically do this on callee_1s side? 

- Doesn't this mean that there will be downtime from caller_1s perspective? Would the calls caller_1 makes fail for some period of time?

Thanks,

Iulian

[Iulian Nitescu]

Or, is it the case that the state is replicated between all routers and therefore all routers hold all registrations all the time? 

[Alexander Gödde]

Hi Iulian,

since the features you're talking about are still under development, I can't give you a definitive answer.

Regards,

Alex