Shared subscriptions

[Dominique Burnand]

Hi,

Is there a way to have "shared subscriptions" in crossbar?

What do I mean by that? 

I think this article sums it pretty much up: https://www.hivemq.com/blog/mqtt-client-load-balancing-with-shared-subscriptions/

Why do I need this?

I have many front-end's which will publish a change to all other frontends. Including the backend which will save this change persistent.

Now I want to scale the backend horizontally: This means that I will have multiple backend-servers subscribed to the same topic, which leads to the issue that all servers will process the change of the frontend (even tough only one of them should handle that).

My current idea:

Implementing some kind of load-balancer:

To have another server in the backend which subscribes to the topic. The only task of that server is to issue a call whenever he receives a subscription.

That call can than be handled by the backend-servers via shared registrations: https://crossbar.io/docs/Shared-Registrations/

However: I do not think that this is the way to go. Or is it? I fear a bottleneck and a point of failure at the server which translates subscriptions to calls.

So the general question is: Is this something to come in crossbar? / If it's not planned, what is the reason? - What would my architecture need to look like to be inline with the ideas behind crossbar/WAMP?

Thanks a lot,

Dominique

[Tobias Oberstein]

Hi Dominique,

currently, WAMP only specifies "shared registrations", that is call
dispatching along one of the following policies:

'single'
'roundrobin'
'random'
'first'
'last'

it would be straight forward to translate this to "shared
subscriptions", where the first subscriber via SubscibeOptions makes the
subscription a shared one, and all subsequent subscribers will
automatically share the former, first subscription.

the broker would dispatch events no longer to all subscribers, but only
one of them based on the respective policy.

this would fit into the existing design, moreso: in a way, it would be a
strange asymmetry _not_ have this option in pubsub, but only in rpc.

Would above work for what you wanna do?

If so, the following would be needed to make it happen in WAMP:

1. issue on the WAMP repo for the feature
2. PR on the WAMP repo for spec text
3. issues on AutobahnPython and Crossbar.io repos
4. implementation PR for AutobahnPython
5. implementation PR for Crossbar.io

The time to get this merged: I'm +1, and I guess we should have it
sooner or later .. but we as a company/project don't have time right now
.. so timeline depends on community contributions ..

Cheers,
/Tobias