WAMP reflection

[Tobias Oberstein]

Hi,

this topic came up a couple of times in the past: reflection.

How do you find out about a WAMP procedure arguments and return value?
At run-time?

Same for WAMP topics. How do discover those?

Lately, I've been thinking about this and came up with the following
experimental approach.

Here is a Markdown document that documents part of a WAMP based
interface exposed by some "store application":

https://github.com/tavendo/WAMP/blob/master/apidoc/test2.md

This document in fact is written in a WAMP-flavored Markdown style.

Which means: it contains code blocks like

https://github.com/tavendo/WAMP/blob/master/apidoc/test2.md#procedure-comexamplestoredelete_product

which actually declare a WAMP procedure in a machine readable way.

The approach is called "literate programming":
http://en.wikipedia.org/wiki/Literate_programming

It's applied to API documentation / metadata generation here.

Now, Crossbar.io trunk can read such files by configuration:

{
"workers": [
{
"type": "router",
"realms": [
{
"name": "realm1",
"decls": [
"../api.md"
],

And then application components can query the API at run-time via
regular WAMP RPC: see attached screenshot.

The other way this will be exposed in Crossbar is: have a Web path
service of resource type "apidoc", which renders above WAMP-flavored
Markdown into regular HTML with nice CSS styles applied.

==

I'd be interested in feedback on this.

Does it make sense? Is it useful? If it does, we should probably add a
description of both the WAMP reflection API and the WAMP-flavored
Markdown format to the WAMP specification.

Anway, what do you think?

Tobias

[Tobias Oberstein]

As an update to below, I've now got something working which uses
JSON-Schema when declaring WAMP procedures, topics and errors.

http://json-schema.org/

And those declaration can be embedded right in normal Markdown
documenation files. E.g.

https://github.com/tavendo/WAMP/blob/master/apidoc/api.md

Crossbar.io can load above WAMP-flavored Markdown and produce 2 things:

1. An online developer API documentation (left side of attached screenshot)

2. Clients can query the schema store at run-time via a single RPC
(right side of screenshot)

I'll now go on to the 3rd thing we can do with schemas:

3. Validate the application payload of calls and events according to the
schema.

E.g. when a procedure is called, all call arguments are checked before
the endpoint is invoked, and when the result comes back, it is again
checked and only then forwarded to the caller.

One can imagine different policies of checking and enforcing schemas in
the WAMP routers etc etc.

I think each of above is nice, but all three - just from allowing
schemas - is really nice;)

Cheers,
/Tobias

[Matt Bonneau]

Hi Tobias,

I do think discovery and description are great additions to the spec.

With your example, would the callee provide this information to the dealer when it registers the call?

Matt

[Tobias Oberstein]

Hi Matt,

Am 17.07.2014 02:12, schrieb Matt Bonneau:
> Hi Tobias,
>
> I do think discovery and description are great additions to the spec.
>
> With your example, would the callee provide this information to the
> dealer when it registers the call?

No, besides wamp.reflect.describe to query for metadata, there is
wamp.reflect.define to set metadata on a URI.

That procedure simply takes an URI and a config object like

{
"type": "procedure",
"title": "Create Product",
"description": "Create a new product in the store catalog."
"kwargs": {
"type": "object",
"properties": {
"id": {
"description": "The unique identifier for a product",
"type": "integer"
},
"name": {
"description": "Name of the product",
"type": "string"
},
"price": {
"type": "number",
"minimum": 0,
"exclusiveMinimum": true
}
},
"required": ["id", "name", "price"]
}
}

This procedure only takes keyword arguments, and the "kwargs" property
is a JSON-Schema object.

So the metadata stuff essentially is a service serparate in principle.

Requirements like all procedures need to be first defined before they
can be registered, or enforcing any calls to follow schemas etc are
router implementation specific.

What the spec will describe are the metadata WAMP API and the schema
language above.

Cheers,
/Tobias

> "../api.md <http://api.md>"

> ],
>
> And then application components can query the API at run-time via
> regular WAMP RPC: see attached screenshot.
>
> The other way this will be exposed in Crossbar is: have a Web path
> service of resource type "apidoc", which renders above WAMP-flavored
> Markdown into regular HTML with nice CSS styles applied.
>
> ==
>
> I'd be interested in feedback on this.
>
> Does it make sense? Is it useful? If it does, we should probably add a
> description of both the WAMP reflection API and the WAMP-flavored
> Markdown format to the WAMP specification.
>
> Anway, what do you think?
>
> Tobias
>
>
>
>

> --
> You received this message because you are subscribed to the Google
> Groups "WAMP" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to wampws+un...@googlegroups.com
> <mailto:wampws+un...@googlegroups.com>.
> To post to this group, send email to wam...@googlegroups.com
> <mailto:wam...@googlegroups.com>.
> Visit this group at http://groups.google.com/group/wampws.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/wampws/176d3366-7b5c-4b9a-b79e-419f371ed4ce%40googlegroups.com
> <https://groups.google.com/d/msgid/wampws/176d3366-7b5c-4b9a-b79e-419f371ed4ce%40googlegroups.com?utm_medium=email&utm_source=footer>.
> For more options, visit https://groups.google.com/d/optout.

[Joshua C Elliott]

Hi Tobias,

As an update to below, I've now got something working which uses
JSON-Schema when declaring WAMP procedures, topics and errors.

http://json-schema.org/

You might also be interested in http://apiblueprint.org/. There's a good ecosystem growing around it. I'll have to take a look at json-schema again.

[Konstantin Burkalev]

Hi, guys!

Tobias, very cool! 

I think, we should add service (RPC and Topics) discovery to WAMP advanced profile. 

This would allow us to implement a Service Registry, that is another one brick in SOA. 

[Tobias Oberstein]

Hi Joshua,

Am 17.07.2014 06:51, schrieb Joshua C Elliott:
> Hi Tobias,
>
> As an update to below, I've now got something working which uses
> JSON-Schema when declaring WAMP procedures, topics and errors.
>
> http://json-schema.org/
>
> You might also be interested in http://apiblueprint.org/. There's a good

Yes, I've been looking at this. Indeed quite interesting. Thanks for
pointing out.

They also make use of regular Markdown documentation files enriched with
additional markup for API description.

This is tailored to a REST/HTTP style API and hence cannot be used as is
for WAMP.

What they do syntax wise like

# GET /message
+ Response 200 (text/plain)

Hello World!

is using regular Markdown items with special meaning.

I tried to do likewise for WAMP

https://github.com/tavendo/WAMP/blob/master/apidoc/test.md#procedure-comexamplestoredelete_product-1

But I like the approach of embedded JSON blocks like

https://github.com/tavendo/WAMP/blob/master/apidoc/api.md#creating-a-product

better.

> ecosystem growing around it. I'll have to take a look at json-schema again.

IMO, it's quite pragmatic and not bad. And we don't reinvent wheels. We
reuse the JSON-Schema for the actual payload description and only add a
few extra attributes for WAMP.

Cheers,
/Tobias

>
> --
> You received this message because you are subscribed to the Google
> Groups "WAMP" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to wampws+un...@googlegroups.com
> <mailto:wampws+un...@googlegroups.com>.
> To post to this group, send email to wam...@googlegroups.com
> <mailto:wam...@googlegroups.com>.
> Visit this group at http://groups.google.com/group/wampws.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/wampws/457a670e-36c2-491f-b83f-85348c750b1e%40googlegroups.com
> <https://groups.google.com/d/msgid/wampws/457a670e-36c2-491f-b83f-85348c750b1e%40googlegroups.com?utm_medium=email&utm_source=footer>.

[Tobias Oberstein]

Am 17.07.2014 09:54, schrieb Konstantin Burkalev:
> Hi, guys!
>
> Tobias, very cool!
>
> I think, we should add service (RPC and Topics) discovery to WAMP
> advanced profile.

Hi Konstantin,

Yep. We could include 2 things in the AP spec:

1) description of the run-time API: the 2 procedures (define/describe)
and the corrsponding events (on_define/on_undefine). This is (re)using
JSON-Schema to describe args/kwargs payloads.

2) description of the WAMP-flavored Markdown format: the WAMP specific
syntax extension to standard (or GitHub flavored) Markdown

Not sure about 2) yet. E.g. here is an different style I am
experimenting with:

https://github.com/tavendo/WAMP/blob/master/apidoc/test2.md

> This would allow us to implement a Service Registry, that is another one
> brick in SOA.

A registry and automatic activation of application components - yes,
that's possible and would be really nifty! It is probably more a router
implementation specific feature. A very powerful one. And it perfectly
fits the "WAMP == SOA" story;)

Cheers,
/Tobias

>
> --
> You received this message because you are subscribed to the Google
> Groups "WAMP" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to wampws+un...@googlegroups.com
> <mailto:wampws+un...@googlegroups.com>.
> To post to this group, send email to wam...@googlegroups.com
> <mailto:wam...@googlegroups.com>.
> Visit this group at http://groups.google.com/group/wampws.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/wampws/979d66ed-5c3a-42fe-a840-c67f6bdf8b41%40googlegroups.com
> <https://groups.google.com/d/msgid/wampws/979d66ed-5c3a-42fe-a840-c67f6bdf8b41%40googlegroups.com?utm_medium=email&utm_source=footer>.

[42o8o1oo]

how is this related to https://github.com/tavendo/WAMP/blob/master/spec/advanced.md#reflection

Trying to undefine a previous defined URI (after a restart of the module) I get the error:

CrossbarRouterServiceSession instance has no attribute '_schemas'

Crossbar doesn´t recover from this Error and i have to restart it.

[Tobias Oberstein]

Am 26.08.2014 12:44, schrieb 42o8o1oo:
> how is this related to
> https://github.com/tavendo/WAMP/blob/master/spec/advanced.md#reflection

Reflection is an advanced feature that is still being discussed. It is
not yet frozen wrt to the spec.

In fact, now that the WAMP basic profile is reaching a plateau of
stability, we should focus on hashing out the AP features.

There are lots of these:

https://github.com/tavendo/WAMP/labels/WAMPv2%20Advanced%20Profile

The good thing is: I am quite confident that the basis we now have with
WAMP will allow us to do most of these - without any breakage of
existing implementations.

>
> Trying to undefine a previous defined URI (after a restart of the
> module) I get the error:
> CrossbarRouterServiceSession instance has no attribute '_schemas'
> Crossbar doesn´t recover from this Error and i have to restart it.

Undefine is not implemented

https://github.com/crossbario/crossbar/blob/master/crossbar/crossbar/router/session.py#L368

CrossbarRouterServiceSession is providing these metaservices.

The reflection stuff in Crossbar should be considered "experimental".

It is there, because I like to actually try things on real code instead
of just doing "paper work" on the spec. The proof is in the pudding;)

===

You seem to have dived into Crossbar.io code base and WAMP spec already
quite deeply. If you are interested, we do need more hands and
contributors. On WAMP. But also in particular on Crossbar.io. The code
base of CB isn't for the faint hearted. It's a powerful, and involved
piece of code. Even more so: if you are an experienced Python hacker and
want to get your feet wet in an exciting project - yees, please! ;)

>
> --
> You received this message because you are subscribed to the Google
> Groups "WAMP" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to wampws+un...@googlegroups.com
> <mailto:wampws+un...@googlegroups.com>.
> To post to this group, send email to wam...@googlegroups.com
> <mailto:wam...@googlegroups.com>.
> Visit this group at http://groups.google.com/group/wampws.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/wampws/b347d1dd-5c78-4aa5-baf1-469597040492%40googlegroups.com
> <https://groups.google.com/d/msgid/wampws/b347d1dd-5c78-4aa5-baf1-469597040492%40googlegroups.com?utm_medium=email&utm_source=footer>.

[matthia...@googlemail.com]

Hi!

For RPC why not describe the kind of transferred data (e.g. with such ja JSON scheme or sth else) while registering at the dealer with REGISTER. This could be done either through adding the information in an additional parameter or in the Options|dict.
Then the dealer would have the the information available without the need that this is defined in the dealer implementation or configuration. Clients could request the information through some tbd. messages, e.g. GET_FUNCTION_INFO, FUNCTION_INFO_RESULT and FUNCTION_INFO_ERROR.
Depending on the implmenetation of the Dealer it could either verify all requests/responses against the scheme if one is available or not.

An open question is how something similar could be achieved for pub/sub, because the publisher never explicitly registers at the server.

Matthias

[Tobias Oberstein]

Hi Matthias,

> For RPC why not describe the kind of transferred data (e.g. with such ja
> JSON scheme or sth else) while registering at the dealer with REGISTER.

Because any schema/validation information is associated with the
_procedure_ - not a _callee_ providing an implementation of the procedure.

> An open question is how something similar could be achieved for pub/sub,
> because the publisher never explicitly registers at the server.

As with RPC, the schema/validation is not associated with a _publisher_,
but the _topic_ being published to.

Cheers,
/Tobias