WAMP for embedded use

[Jens Kristian Jensen]

I have been using websockets for a while with some custom protocols for debugging SW in embedded systems. So far I have been using custom JSON protocols, but WAMP seems much more structured and also includes RPC. I consider using WAMP for more extended RPC use, but have a couple of questions:

1) Is there a client and/or server implementation that can be accessed from C++? I suspect not - not listed on the implementations page and I haven't been able to find one.

2) WAMP seems very flat in its RPC structure. Is there an intended way to return e.g. "objects" that can have methods called on them? I suspect you could have e.g. a "GetObject()" could return a string that would form part of the URI for calling methods on that "object". Something like http://myserver.com/api#GetObject would return "myObj" which could be used to call http://myserver.com/api/objects#MethodOnObject But this would have to be known explicitly on the client side and would not make it very easy to support something like client proxy objects unless the WAMP client library knew about the specific way to map objects to URIs.

3) Is there any kind of introspection support in WAMP? If not, I suspect you could first use the serverIdent from the WELCOME message to identify service type and API version, or perhaps call an RPC method that returns some kind of IDL. Is any kind of IDL defined for WAMP? I suspect the flexibility of the URI would make it hard to write a strict syntax for something like the "object" feature mentioned above?

4) How would you go about implement a server in a strongly typed, reflectionless language like C++? I currently use libwebsockets with handmade code for all the calls. The usual way is to have a separate bindingtool generate server classes server class stubs from an IDL input file or require server objects to inherit from some kind of framwork class that implements introspection. Anyone with a better idea for C and/or C++?

Thanks,

Jens Kristian Jensen

[Tobias Oberstein]

Hello Jens,

> 1) Is there a client and/or server implementation that can be accessed
> from C++? I suspect not - not listed on the implementations page and I
> haven't been able to find one.

Not that I know of. What networking abstraction are you using? ASIO?

>
> 2) WAMP seems very flat in its RPC structure. Is there an intended way
> to return e.g. "objects" that can have methods called on them? I suspect
> you could have e.g. a "GetObject()" could return a string that would
> form part of the URI for calling methods on that "object". Something
> like http://myserver.com/api#GetObject would return "myObj" which could
> be used to call http://myserver.com/api/objects#MethodOnObject But this
> would have to be known explicitly on the client side and would not make
> it very easy to support something like client proxy objects unless the
> WAMP client library knew about the specific way to map objects to URIs.

Thats not a protocol feature, but one of the specific WAMP
implementation. Mapping RPC endpoint URIs onto object methods for more
convenient calling can be done trivially i.e. in JavaScript (or other
dynamic languages). With WAMP-CRA, a client gets a list of RPC endpoint
URIs after authentication. Here is an example:
https://github.com/tavendo/AutobahnPython/tree/master/examples/wamp/authentication

>
> 3) Is there any kind of introspection support in WAMP? If not, I suspect
> you could first use the serverIdent from the WELCOME message to identify
> service type and API version, or perhaps call an RPC method that returns
> some kind of IDL. Is any kind of IDL defined for WAMP? I suspect the

An IDL is orthogonal to the protocol itself. You can of course require
all your RPC and PubSub payloads to conform to payload spec described
via your IDL, and have an introspection RPC endpoint that a client can
use at run-time.

> flexibility of the URI would make it hard to write a strict syntax for
> something like the "object" feature mentioned above?

Being able to call "obj1.method1(param1, param2)" is independent of
having param1/param2 respect some IDL regarding payload ..

>
> 4) How would you go about implement a server in a strongly typed,
> reflectionless language like C++? I currently use libwebsockets with

Good question. We have implementations for dynamically typed languages
(Python, JavaScript, PHP, ..) and statically typed languages that have
reflection (Java). The reflection allows to coerce values into the
respective static type.

I.e. in AutobahnAndroid you can issue an RPC like so:

mConnection.call("http://example.com/calc#add",
Integer.class,
new Autobahn.CallHandler() {

@Override
public void onResult(Object result) {
int res = (Integer) result;
Log.d(TAG, "calc:add result = " + res);
}

@Override
public void onError(String error, String info) {
}
},
23, 55
);

In this case, Integer.class is the return type the result is coerced
into, but it could be a user defined, structured "Person" class as well.
This magic is done by Jackson under the hood (JSON lib with object
mapping support).

> handmade code for all the calls. The usual way is to have a separate
> bindingtool generate server classes server class stubs from an IDL input
> file or require server objects to inherit from some kind of framwork
> class that implements introspection. Anyone with a better idea for C
> and/or C++?

I can see roughly 3 ways:


1. Write an interface spec in an IDL first. Then generate stubs from the
IDL.

I don't like that. It's old school .. CORBA .. WSDL .. crap. I don't
need that stuff for dynamic languages. It introduces a new languages
(the IDL).


2. Use a C++ type that supports heterogenous values like

boost::any
boost::variant

or stl::tuple / Boost.Tuple/.Fusion

http://www.boost.org/doc/libs/1_53_0/libs/fusion/doc/html/fusion/introduction.html

Something in this direction:

http://stackoverflow.com/questions/6561643/boost-variant-how-to-model-json
https://github.com/shokai/json_builder-cpp

Use values of such type throughout your application code.

IMHO, thats a valid/sane approach. It's simple. However, walking on this
road you will start to write a dynamic language core within your C++
app. At some point, I'd just kick that and link in LuaJIT.


3. Compile-time reflection

Wait until C++ provides the convenience of i.e. doing

===============

#pragma reflect // generate metadata for the next language element
sturct Reflected
{
int a;
string b;
double c;
};

vector<Reflected> myData;

CsvWriteHeader<Reflected>(myFile);
for (auto& entry: myData)
CsvWriteLine(myFile, entry);

===============

Above is from a comment here

http://boost.2283326.n4.nabble.com/OT-RFC-standardized-compile-time-reflection-td4573296.html


This has great potential. Should you find something as easy to use as
above code snipped together with clang, drop me a line ..

The stuff here look promising on first look:

https://bitbucket.org/dwilliamson/clreflect/wiki/Marking%20Primitives%20for%20Reflection

https://bitbucket.org/dwilliamson/clreflect

and this is a nice overview:

http://www.altdevblogaday.com/2011/09/25/reflection-in-c-part-1-introduction/


>
> Thanks,
> Jens Kristian Jensen
>
> --
> You received this message because you are subscribed to the Google
> Groups "WAMP" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to wampws+un...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

[Jens Kristian Jensen]

Thanks for the answer

On C implementations:

I use libwebsockets as raw WS library, which in turn uses 

On mapping RPC URIs to proxy objects

True, it does not have to be of the protocol itself. But then there has to be a common understanding to ensure than the URIs constituting and object in, say, AutobahnPython server and a client implementation made in another language. Is there more to the WAMP spec than what I already read? The proxy objects would be more than just a convenience IMO - also a way to say "there return value you get here is really a reference to an object, which has these methods". It provides an easier way to document an interface and provides structure to the API in a way similar to classes and methods vs. flat function calls. 

On IDLs:

For me they are at least of the following use:

* Checking that a client and a server speaks the same language (same interface, version, etc.)

* A way to generate client/server stubs in languages like C++ - the "old school way" as you said.

* A way to help document your interfaces

* Can be used as a way to interrogate a server about its provided interface. This way you can make a generic tool for inspecting or calling any server's interface, like e.g. D-Feet for DBus ( https://live.gnome.org/DFeet/ ). Currently I cannot see a way to do that with WAMP, at least not with the raw protocol. If I'm mistaken, please educate me.

Thanks for WAMP and the interesting pointers to C++ reflection. 

[Tobias Oberstein]

Hi Jens,

below my opinionated view on the matter.

Am 08.02.2013 23:01, schrieb Jens Kristian Jensen:
>
> Thanks for the answer
>
> On C implementations:
> I use libwebsockets as raw WS library, which in turn uses

Depending on your needs, you might also be interested in

https://github.com/zaphoyd/websocketpp

which is an ASIO based C++ WebSocket implementation that has 100% test
coverage via AutobahnTestsuite.

>
> On mapping RPC URIs to proxy objects
> True, it does not have to be of the protocol itself. But then there has
> to be a common understanding to ensure than the URIs constituting and
> object in, say, AutobahnPython server and a client implementation made

Parts of a distributed application always need some "understanding" of
each other. And this doesn't stop at the syntactic level of some
interface, but necessarily extends to the semantics of the interface as
well. I have yet to see an IDL that describes the semantics of an API.
For if you have an IDL that unambiguously and completely describes an
interface syntax _and_ semantics, that interface spec would be
"executable" in some interpreter .. and hence already constitute an
implementation of the component.

> in another language. Is there more to the WAMP spec than what I already
> read? The proxy objects would be more than just a convenience IMO - also
> a way to say "there return value you get here is really a reference to
> an object, which has these methods". It provides an easier way to

A RPC with WAMP calls a remote _procedure_, not a method on a specific
object of some instantiated class. This is not CORBA. Deliberately.
There are no protocol means to instantiate a server-side object for
client use. The server-side implementation might of course have a
session specific object instantiated providing the endpoint
implementation .. or it might be a singleton .. or whatever. Thats
hidden from the client.

> document an interface and provides structure to the API in a way similar
> to classes and methods vs. flat function calls.
>
> On IDLs:
> For me they are at least of the following use:
>
> * Checking that a client and a server speaks the same language (same
> interface, version, etc.)

Interface versioning (CORBA/DCOM land) introduces coupling .. WAMP is
all about simple, adaptive, dynamically typed interfaces ..

What do you do if you have deployed 100s of your embedded devices and
now want to extend an interface, say add some attribute to a return
value? Your statically generated stubs will break. You need to have a
new interface version. Complexity, brittleness arises ..

> * A way to generate client/server stubs in languages like C++ - the "old
> school way" as you said.

Supporting statically typed languages can be done by molding i.e. RPC
return values into the concrete implementation type .. skipping i.e.
attributes unknown by the implementation. This is how AutobahnAndroid
works .. via the magic provided by Jackson.

Of course that relies upon reflection capabilities of Java .. but C++ is
catching up, and using i.e.

https://bitbucket.org/dwilliamson/clreflect

it should be possible to replicate this approach.

Yes, this is different from having a separate IDL and generating
stubs/proxies ..

> * A way to help document your interfaces

This is a valid point .. we need something here.

I'd differentiate between:

a. documentation
b. validation
c. code generation (stubs/proxies)

IDLs and schema languages are for c. and b.. IMHO b. should be done on
app code, since you can't possibly specify all constraints via a schema
language anyway (unless you make it a full featured programming lang.)

> * Can be used as a way to interrogate a server about its provided
> interface. This way you can make a generic tool for inspecting or
> calling any server's interface, like e.g. D-Feet for DBus (
> https://live.gnome.org/DFeet/ ). Currently I cannot see a way to do that
> with WAMP, at least not with the raw protocol. If I'm mistaken, please
> educate me.

With WAMP Challenge-Response Authentication, the client already gets a
list of RPC endpoint URIs (and PubSub topics) he is authorized to use.
WAMP-CRA is implemented on top of 2 predefined RPCs

http://api.wamp.ws/procedure#authreq
http://api.wamp.ws/procedure#auth

This could be extended by providing a reflection API:

http://api.wamp.ws/procedure#describeProcedure
http://api.wamp.ws/procedure#describeTopic

This should at least include developer documentation for the parameters
and return values.

We could of course invent something along

http://json-schema.org/
http://tools.ietf.org/html/draft-zyp-json-schema-04

and return respective machine readable data with "describeProcedure".

This could be then used for validation and/or stub generation.

However, I don't believe in schemas for validation ..

Take the example here:

http://json-schema.org/example1.html

What if I want tags to follow some regular expression, but only if the
user is not admin?
What if I want the minimum allowed product price to depend to on product
category?

In any case: there is no need to extend the _wire protocol_.

If you want to further discuss the "describeProcedure/Topic" approach
above, I have created an issue for WAMPv2

https://github.com/tavendo/wamp/issues/22

Your opinions/ideas are welcome!

-- Tobias

>
> Thanks for WAMP and the interesting pointers to C++ reflection.
>

[Florian Meier]

This is exactly my use case. Do you have made any progress?

Greetings,

Florian

[Jens Kristian Jensen]

No, I gave up on WAMP, and continued to use a custom Websockets protocol using libwebsockets. I still think WAMP has obvious merits, compared to what I do, but I could not justify the investment to make it work for C++ given that this was for a hobbyist/debug project.

Regards,

Jens Kristian

[Tobias Oberstein]

FWIW, there now is a C++11 WAMP implementation: https://github.com/tavendo/AutobahnCpp/