Guidance on application built with Autobahn

[Si]

Greetings,

I'm seeking guidance on building an application using Autobahn, hopefully it's ok to post here. 

Details on the project can be found here. This is my first project in python and also Autobahn, so any advice is welcome.

Originally I was just going to use websockets via Autobahn, but after some research it seems wamp sitting on top of websockets is a better approach, since it can perform authentication, authorization, as well as handling rpc calls and pubsub events, all of which I need.

After reviewing the autobahn code examples and source code, I'm unsure as to whether I should be using the wamp Application (or ApplicationSession?) class, and decorate my api accordingly, or decorate with exportRpc, exportPub and exportSub and use a class derived from the WampServerProtocol?

The web part of the project will use Django and Bootstrap, so I would be using websockets along with more typical html requests. Is there anything to watch out for with this? Since they are different protocols operating on different ports I can't think there would be, but not sure...

In regards to wamp authorization, I would like to bind the rights to api rpc calls and pubsub events based on Django user groups. The example code for wampcra points me in the right direction, but I'm unsure as to how to wire up my api. I'm a C# developer in my day job, and typically would use an attribute (decorator) based approach to define which roles/groups would have permission for functions. Is this a decent way to approach it in autobahn/python?

e.g.

@exportRpc or @app.register

@authorization('adminRole', 'userRole')

def someApiFunction():

    ...

Or should I follow the example shown here and manually define all the rpc and pubsub functions explicitly in one place?

Thanks in advance for any help.

regards, 

si

[Tobias Oberstein]

Si,

Am 21.08.2014 05:37, schrieb Si:
> Greetings,

Hello!

>
> I'm seeking guidance on building an application using Autobahn,
> hopefully it's ok to post here.

Sure.

>
> Details on the project can be found here

> <https://github.com/si618/pi-time>. This is my first project in python

> and also Autobahn, so any advice is welcome.

There is a learning curve. Have you done asynchronous / event driven
programming before? I mean in another language?

Because: having experience with that will be very valuable entering
Autobahn - which is asynch. from core to outside.

The second hint is understanding the core design of WAMP:

http://wamp.ws/why/

Well, plus Python;)

These 3 know-how areas will give you a good fundament, and you will then
find your way as it "starts to make sense";)

>
> Originally I was just going to use websockets via Autobahn, but after
> some research it seems wamp sitting on top of websockets is a better
> approach, since it can perform authentication, authorization, as well as
> handling rpc calls and pubsub events, all of which I need.

Yes, WAMP provides all of this. The practical short story is this:

You write app components (e.g. one that runs on the Pi), and those
connect via WAMP to a _router_.

If you want to use Python for writing an app component, then using
AutobahnPython for doing that is recommended.

AutobahnPython _also_ includes a basic router - but that doesn't come
with builtin authentication or authorization.

If you want to have a router with production ready features, one option
is Crossbar.io - this is open-source made by Tavendo. There are other
router implementations as well: please see the WAMP website. So you are
not "locked in" with WAMP.

Getting started with Crossbar.io and Python is trivial:

http://crossbar.io/docs/Getting-started-with-Python/

Crossbar.io can create complete apps for you ("scaffolding") which you
can then tweak.

>
> After reviewing the autobahn code examples and source code, I'm unsure
> as to whether I should be using the wamp Application (or
> ApplicationSession?) class, and decorate my api accordingly, or decorate

You can use both. ApplicationSession is fully supported. Application is
very new and might change - but we will also support it fully.

It's kinda taste question. Application has more stuff built into and
might be easier to use.

> with exportRpc, exportPub and exportSub and use a class derived from the

No. Forget those. That is WAMP _v1_. It is deprecated. We will soon
remove all WAMP v1 related code and examples from AutobahnPython.

> WampServerProtocol?

No. See above. With WAMP v2 you are purely concerned with
ApplicationSession or Application.

>
> The web part of the project will use Django and Bootstrap, so I would be

Boostrap is just css sugar. No concern. Django: see below.

> using websockets along with more typical html requests. Is there
> anything to watch out for with this? Since they are different protocols

Yes: it does not work;)

Django is based on blocking code. There is no (sane) WebSocket stuff,
not to speak of WAMP running directly inside Django.

What exactly are you trying to cover using Django?

I mean: it's useless for Pi/WAMP client components. It's also useless if
you are doing UIs as HTML5 single-page-apps.

> operating on different ports I can't think there would be, but not sure...

AutobahnPython runs under Twisted or asyncio. Django does not run under
either of those.

>
> In regards to wamp authorization, I would like to bind the rights to api
> rpc calls and pubsub events based on Django user groups. The example
> code

> <https://github.com/tavendo/AutobahnPython/blob/master/examples/twisted/wamp/authentication/wampcra/server.py>

> for wampcra points me in the right direction, but I'm unsure as to how
> to wire up my api. I'm a C# developer in my day job, and typically would

Using Crossbar.io, please see here:

http://crossbar.io/docs/Authorization/

> use an attribute (decorator) based approach to define which roles/groups
> would have permission for functions. Is this a decent way to approach it
> in autobahn/python?

No. Attaching authorization information directly to procedures is bad
design - at least this is Autobahn's and WAMP's view.

Whether a client 1 is allowed to call a procedure implemented in client
2 should (in general) be decided by the router (or a custom authorizer
in client 3)

Also you don't want to sprinkle roles in app code ..

>
> e.g.
>
> @exportRpc or @app.register
> @authorization('adminRole', 'userRole')
> def someApiFunction():
> ...

No. See above.

>
> Or should I follow the example shown here

> <https://github.com/tavendo/AutobahnPython/blob/master/examples/twisted/wamp1/authentication/server.py>

You should ignore everything that contains the word "wamp1". Library as
well as example code.

> and manually define all the rpc and pubsub functions explicitly in one
> place?

Not sure what that means.

>
> Thanks in advance for any help.

Welcome to WAMP and Autobahn!

I am aware that what I wrote above might only partially answer your
questions (or was too fast) and/or trigger more questions. No problem:
keep on asking;)

/Tobias

>
> regards,
> si
>
> --
> You received this message because you are subscribed to the Google
> Groups "Autobahn" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to autobahnws+...@googlegroups.com
> <mailto:autobahnws+...@googlegroups.com>.
> To post to this group, send email to autob...@googlegroups.com
> <mailto:autob...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/autobahnws/d4ba21df-0fa4-405d-9717-8a36c4c235d4%40googlegroups.com
> <https://groups.google.com/d/msgid/autobahnws/d4ba21df-0fa4-405d-9717-8a36c4c235d4%40googlegroups.com?utm_medium=email&utm_source=footer>.
> For more options, visit https://groups.google.com/d/optout.

[Si]

Super!

Thanks so much for the detailed response Tobias, it's exactly what I was looking for.

The front end (web) code is still the main part left to write, so I'm not locked into Django and I'm glad I asked the question now :) although I'm using its ORM and test framework, I don't think it will be too hard to swap over to other frameworks.

Is there an SPA framework that you would recommend for use with Crossbar?  

I've used MVVM pattern in C# quite a bit, so I'm liking the look of Knockout, but suggestions welcome.

Anyway, I will investigate crossbar.io further before asking more questions, and remove any wamp1 related references in my code.

Thanks again,

Si

btw, there is a 404 on the crossbar notification demo for the source code link. Looks like it needs to be updated to this.

[Tobias Oberstein]

Am 22.08.2014 03:02, schrieb Si:
> Super!
>
> Thanks so much for the detailed response Tobias, it's exactly what I was
> looking for.
>
> The front end (web) code is still the main part left to write, so I'm
> not locked into Django and I'm glad I asked the question now :) although
> I'm using its ORM and test framework, I don't think it will be too hard
> to swap over to other frameworks.

What database are you using?

>
> Is there an SPA framework that you would recommend for use with Crossbar?
> I've used MVVM pattern in C# quite a bit, so I'm liking the look of
> Knockout, but suggestions welcome.

Yep. We (Tavendo) use Knockout for every HTML5 UI we do. We either do
fully static Web sites or SPAs.

Knockout perfectly fits and works with AutobahnJS. It's light-weight,
gets out of your way, and it is _not_ opinioated / a full-stack thing.
Which I like. Alex knows much more on programming with Knockout than me.
He has upcoming blog posts also ..

Other SPA frameworks:

- AngularJS. More opinionated, more full stack. But large community and
growing. Others on this list are using it with WAMP and Crossbar.io.
There is also a page:
http://crossbar.io/docs/AngularJS-Application-Components/

- ExtJS. Kinda old school MVC thing (think: Microsoft MFC). But quite
capable and "enterprisy". E.g. this

https://www.youtube.com/watch?v=yIdmk2doU8w
http://record-evolution.com/

is built with ExtJS and Crossbar. There is

https://github.com/tavendo/AutobahnExtJS

which provides custom ExtJS data proxies - but it needs migration to
WAMP v2 and more love.

==

I'd say, learning curve with above is: Knockout < Angular < ExtJS

>
> Anyway, I will investigate crossbar.io further before asking more
> questions, and remove any wamp1 related references in my code.

Don't hesitate asking .. it can also help us improving the docs to know
where people are struggling ..

>
> Thanks again,
> Si
>
> btw, there is a 404 on the crossbar notification demo

> <https://demo.crossbar.io/demo/notification/index.html> for the source
> code
> <https://github.com/crossbario/crossbardemo/tree/master/web/demo/notification>

> link. Looks like it needs to be updated to this

> <https://github.com/crossbario/crossbardemo/tree/master/crossbardemo/crossbardemo/web/demo/notification>.

Ah, thanks
https://github.com/crossbario/crossbardemo/issues/12

>
>
> --
> You received this message because you are subscribed to the Google
> Groups "Autobahn" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to autobahnws+...@googlegroups.com
> <mailto:autobahnws+...@googlegroups.com>.
> To post to this group, send email to autob...@googlegroups.com
> <mailto:autob...@googlegroups.com>.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/autobahnws/873678af-eee6-4550-9625-34de5cbe5ff0%40googlegroups.com
> <https://groups.google.com/d/msgid/autobahnws/873678af-eee6-4550-9625-34de5cbe5ff0%40googlegroups.com?utm_medium=email&utm_source=footer>.

[Si]

On Friday, 22 August 2014 17:01:07 UTC+9:30, Tobias Oberstein wrote:

Am 22.08.2014 03:02, schrieb Si:
>
> The front end (web) code is still the main part left to write, so I'm
> not locked into Django and I'm glad I asked the question now :) although
> I'm using its ORM and test framework, I don't think it will be too hard
> to swap over to other frameworks.

What database are you using? 

At this stage, SQLite should suffice. 

I found an async ORM (twistar) also built on twisted, and have had some async experience in the .NET/C# worId, so understand once you go async it's best to go all in (turtles all the way down?) but I think it can wait until the front end is done.

 

> Is there an SPA framework that you would recommend for use with Crossbar?
> I've used MVVM pattern in C# quite a bit, so I'm liking the look of
> Knockout, but suggestions welcome.

Yep. We (Tavendo) use Knockout for every HTML5 UI we do. We either do
fully static Web sites or SPAs.

Great, thanks, I'll give that a go then, and keep a lookout for Alex's blog posts.

I spent some time researching Crossbar and it does seem to cover all of my bases.

Plan of attack is to create two crossbar applications:

- One is the lap timer (app+web server) which receives events triggered by sensors, stores them in a db, then notifies users.

- The other is the sensor, which initially will be on the same raspberry pi, but the plan is to have multiple sensors on the same network, sending events to the lap timer server.

That seems to fit the crossbar platform (I hope:).

cheers, si

[Tobias Oberstein]

Am 22.08.2014 09:58, schrieb Si:
>
>
> On Friday, 22 August 2014 17:01:07 UTC+9:30, Tobias Oberstein wrote:
>
> Am 22.08.2014 03:02, schrieb Si:
> >
> > The front end (web) code is still the main part left to write, so
> I'm
> > not locked into Django and I'm glad I asked the question now :)
> although
> > I'm using its ORM and test framework, I don't think it will be
> too hard
> > to swap over to other frameworks.
>
> What database are you using?
>
>
> At this stage, SQLite should suffice.
>

> I found an async ORM (twistar <https://github.com/bmuller/twistar>) also

> built on twisted, and have had some async experience in the .NET/C#
> worId, so understand once you go async it's best to go all in (turtles

Ah, very good. I see you already know the critical points;)

sqlite is fine!

The (low-level / SQL) way of doing database with Twisted is this:

https://twistedmatrix.com/documents/14.0.0/core/howto/rdbms.html

This stuff works with regular Python database drivers (incl. sqlite)
which are usually blocking. It runs DB things on a background worker
thread pool.

For PostgreSQL there is _also_ native async DB drivers.

If you want to go ORM (not pure SQL), twistar looks good. I haven't used
it myself, but yes.

> all the way down?) but I think it can wait until the front end is done.
>
> > Is there an SPA framework that you would recommend for use with
> Crossbar?
> > I've used MVVM pattern in C# quite a bit, so I'm liking the look of
> > Knockout, but suggestions welcome.
>
> Yep. We (Tavendo) use Knockout for every HTML5 UI we do. We either do
> fully static Web sites or SPAs.
>
>
> Great, thanks, I'll give that a go then, and keep a lookout for Alex's
> blog posts.
>
> I spent some time researching Crossbar and it does seem to cover all of
> my bases.
>
> Plan of attack is to create two crossbar applications:
> - One is the lap timer (app+web server) which receives events triggered
> by sensors, stores them in a db, then notifies users.
> - The other is the sensor, which initially will be on the same raspberry
> pi, but the plan is to have multiple sensors on the same network,
> sending events to the lap timer server.
>
> That seems to fit the crossbar platform (I hope:).

This is a perfect fit. It is exactly these kind of app where components
are diverse and distributed which we designed WAMP/CB for ..

/Tobias

>
> cheers, si

>
>
> --
> You received this message because you are subscribed to the Google
> Groups "Autobahn" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to autobahnws+...@googlegroups.com
> <mailto:autobahnws+...@googlegroups.com>.
> To post to this group, send email to autob...@googlegroups.com
> <mailto:autob...@googlegroups.com>.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/autobahnws/3af81e38-8103-4b11-9c79-aab9501c7398%40googlegroups.com
> <https://groups.google.com/d/msgid/autobahnws/3af81e38-8103-4b11-9c79-aab9501c7398%40googlegroups.com?utm_medium=email&utm_source=footer>.

[Si]

Hi again,

What's the recommended approach for logging in crossbar?  

I might have missed it, but couldn't find anything in the demos or documentation regarding logging, configuration of log levels and log output (console, file, etc.).

Looking at the source code it appears to be a mix of the twisted.python log and the crossbar Logger.

cheers, si

 

[Tobias Oberstein]

Am 27.08.2014 07:22, schrieb Si:
> Hi again,
>
> What's the recommended approach for logging in crossbar?

You can log from WAMP application components hosted by Crossbar.io by
just doing "print". Crossbar.io will forward anything received from
stdout from the component to it's own log mechanism.

>
> I might have missed it, but couldn't find anything in the demos or
> documentation regarding logging, configuration of log levels and log
> output (console, file, etc.).

Not your fault: https://github.com/crossbario/crossbar/issues/109

The docs still have gaps. I am sorry. This thing is still moving fast
and we are working hard to fill in missing stuff. Both features and docs.

>
> Looking at the source code it appears to be a mix of the twisted.python
> log and the crossbar Logger.

How logging _inside_ of Crossbar.io itself is done is only interesting
to Crossbar.io developers - not app developers. The mechnisms there are
a little involved, because of the forwarding from app components that
Crossbar.io does.

>
> cheers, si
>
> --
> You received this message because you are subscribed to the Google
> Groups "Autobahn" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to autobahnws+...@googlegroups.com
> <mailto:autobahnws+...@googlegroups.com>.
> To post to this group, send email to autob...@googlegroups.com
> <mailto:autob...@googlegroups.com>.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/autobahnws/48e181c7-0e0e-4d59-a75a-aeb19b9b9ed8%40googlegroups.com
> <https://groups.google.com/d/msgid/autobahnws/48e181c7-0e0e-4d59-a75a-aeb19b9b9ed8%40googlegroups.com?utm_medium=email&utm_source=footer>.

[Si]

No worries Tobias, 

I was looking for something a little more fine-grained than print i.e. being able to set log levels, but for the time being it looks like the twisted log is working fine, and injects logging output along with the rest of the crossbar logs inside a container. i.e. in my api code I have:

from twisted.python import log

log.msg("blah")

...with resulting console output once crossbar is started:

2014-08-27 22:12:44+0930 [Container    7608] blah

The other thing that took me a while to wrap my head around, and this is more a python issue than crossbar, but repeating here in case it helps others or is not advised? I want two separate crossbar apps to share common core code (api, model, db, etc.), and it looks like out how you structure your directories is pretty important. This is what I ended up with:

|- myApp (in PYTHONPATH)

  |- crossbarApp1

  |- crossbarApp2

  |- myApp

     __init__.py   

    |- core

       __init__.py

       api.py

From either crossbar app I can now reference the shared code:

from myApp.core.api import ...

The crossbar logging came in handy trying to figure this out, thanks :)

[Si]

Knockout perfectly fits and works with AutobahnJS. It's light-weight, 
gets out of your way, and it is _not_ opinioated / a full-stack thing. 
Which I like. Alex knows much more on programming with Knockout than me. 
He has upcoming blog posts also .. 

Hello again,

Made some progress and finding knockout to be a really nice framework, but I'm struggling to work out a decent approach using the websocket connection to transfer data to and from the view models. Do you guys have any advice or code examples? Just looked on the Tavendo blog and couldn't see any relevant posts from Alex...

I found this project, but it's doing pretty much what I am, and setting view model data from events wired to the connection, which doesn't seem quite right, as I will have many view models and lots of rpc and events to wire up. It feels like I should be using the websocket session and connection from inside my view models, which is how I've used MVVM pattern before.

thanks, si

[Roger Erens]

I'm interested in that, too.

If I'm not mistaken, Alex is preparing a tutorial in which knockout.js is used.

Maybe we can proofread already some of the stuff he has?

There's also this example already, did you see it?

https://github.com/crossbario/crossbardemo/tree/master/crossbardemo/crossbardemo/web/demo/form/knockout

but the database-functionality is too distracting to my liking.

Roger

Op vrijdag 3 oktober 2014 10:09:54 UTC+2 schreef Si:

[Si]

There's also this example already, did you see it?

https://github.com/crossbario/crossbardemo/tree/master/crossbardemo/crossbardemo/web/demo/form/knockout

Thanks Roger!

I hadn't seen those demos, and it definitely helps, in particular editform.js clears a few things up.

It seems event callbacks are wired to global functions, which then interact with view models, rather than trying to wire callbacks to functions in instances of view models (which I was struggling with:)

cheers, si

[Alexander Gödde]

Hi!

Sorry for the late response - I was on vacation!

A knockout tutorial is definitely on my to-do list - and at the moment I would like to base this on the edit form demo mentioned above.

The code is, however, quite old, and I like to think that my skills in using knockout have progressed since. So if you take a look at this in its present state, all you can be sure of is that it's working, but don't take anything about how the code is structured as gospel!

For the tutorial, the first thing on my list is a rewrite of the demo. 

Since there's still a clean slate - what would you like to see in a tutorial? How much of the basics of knockout need to be covered? How much of WAMP/Autobahn?

Regards,

Alex

[si]

Hi Alex,

No need for apologies, hope you enjoyed your vacation, I just got back from mine :)

From WAMP/Autobahn, I'm interested in good ways to handle connection and session management, and session interaction (rpc & events) with knockout view models. I'm also interested in working out a way to wrap up session.call method to provide generic logging of each call (both request and response|error), but still retain the nice .call(...).then(...) deferred syntax.

For Knockout, an example of lazy loading of view model collections from session rpc result would be great, and perhaps an example of updating a view model, but only after multiple rpc calls have been returned, e.g. I might want to update a property of the view model after each rpc is returned, but another vm property only after all of the rpc calls have returned.

cheers, si

--
You received this message because you are subscribed to a topic in the Google Groups "Autobahn" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/autobahnws/E2AsoUtH-vg/unsubscribe.
To unsubscribe from this group and all its topics, send an email to autobahnws+...@googlegroups.com.
To post to this group, send email to autob...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/autobahnws/0f574a17-43b2-419d-a104-d024ee024bbf%40googlegroups.com.

[Roger Erens]

Hi Alex,

my suggestion would be to follow the interactive Knockout.js tutorial very closely, since this would be familiar for people learning/having learned knockout.js.

In that tutorial, a mail client is built, but the interaction with the server is done by some hand-waving. I think it would be nice to show how that would be done using WAMP/Autobahn. E.g. start with obtaining the initial list of e-mails via RPC, and then using subscription to obtain new e-mails from the server-backend.

Might even attract Knockout.js students to the Autobahn/WAMP/Crossbar.io ecosystem!

I think Si's suggestion is already targeted at slightly more intermediate level Autobahn/Knockout.js students?

Cheers,

Roger

To view this discussion on the web visit https://groups.google.com/d/msgid/autobahnws/CAEgRcgAGBg62uJAuTcqJ58A5sfRF14LhzdENmORP33Udgf%3Dhkw%40mail.gmail.com.