Distribute pubsub among many processes

[Nahuel Defossé]

I've been trying CrossbarIO and I could not see clearly how to solve this specific scenario:

1) a twisted based network process that should receive some notifications such as read the database again, stop one of its protocols. It produces some notifications.

2) a WSGI application that could send messages to 1 (this should be implemented with corssbar WSGI host service).

3) pubsub that send the last "known" value upon subscriptions. This will interact directly with browser. WAMP.

I see that 2 to 3 could use the Push Service. But I don't see how to spread notifications from 1 to 3. RPC?

If I had more processes like 1, that are twisted based, though they must run in a different address space, for some reason. Should I have some sort of proxy in 3?

Thanks.

[Tobias Oberstein]

Am 25.11.2014 03:22, schrieb Nahuel Defossé:
> I've been trying CrossbarIO and I could not see clearly how to solve
> this specific scenario:
>
> 1) a twisted based network process that should receive some
> notifications such as read the database again, stop one of its
> protocols. It produces some notifications.

You can run an AutobahnPython based WAMP client component together with
the other network stuff you want to do in 1 process. The WAMP client
component can connect to Crossbar.io, can be accessed from your "other
network stuff" and publish regular WAMP events.

As a bonus, you can have Crossbar.io start and monitor above process as
a "guest worker".

> 2) a WSGI application that could send messages to 1 (this should be
> implemented with corssbar WSGI host service).

The Crossbar.io WSGI service
(http://crossbar.io/docs/WSGI-Host-Service/) is a convenience mechanism
to start and monitor a regular WSGI app from and under Crossbar.io.

However, this service (currently) does not expose WAMP client functionality.

You can still achive this today. Here is how.

Using AutobahnPython and Twisted, you write a program that runs a WAMP
client connecting to the router plus a Twisted WSGI container. The only
thing to take care of is when publishing from WSGI code.

The WAMP component will run on the main thread while the WSGI code will
run on a background thread, and hence you will need to use callFromThread.

http://twistedmatrix.com/documents/current/api/twisted.internet.interfaces.IReactorThreads.callFromThread.html

When wrapping a regular WSGI application (which is blocking since it is
WSGI), and running this WSGI under Twisted, Twisted will run the WSGI
code on a background worker thread pool.

Depending on what you actually want to achieve with WSGI, you can also
go fully async, e.g. using Klein, a Flask clone:

http://tavendo.com/blog/post/mixing-web-and-wamp-code-with-twisted-klein/

> 3) pubsub that send the last "known" value upon subscriptions. This will
> interact directly with browser. WAMP.
>
> I see that 2 to 3 could use the Push Service. But I don't see how to
> spread notifications from 1 to 3. RPC?
> If I had more processes like 1, that are twisted based, though they must
> run in a different address space, for some reason. Should I have some
> sort of proxy in 3?
>
> Thanks.
>

> --
> You received this message because you are subscribed to the Google
> Groups "Crossbar" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to crossbario+...@googlegroups.com
> <mailto:crossbario+...@googlegroups.com>.
> To post to this group, send email to cross...@googlegroups.com
> <mailto:cross...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/crossbario/0f380eab-ccbc-42cb-9243-fbd77de64e5f%40googlegroups.com
> <https://groups.google.com/d/msgid/crossbario/0f380eab-ccbc-42cb-9243-fbd77de64e5f%40googlegroups.com?utm_medium=email&utm_source=footer>.
> For more options, visit https://groups.google.com/d/optout.