ROS and Web Clients

[Hai Nguyen]

I'm not particularly familiar with DDS, but does it work nicely with browsers? either through some sort of translation or better yet, natively? I'm interested as the current ROS to WebSockets bridge is particularly ugly: the bridge has to subscribe to all the messages that any web client would need to listen to and then rebroadcast them, which introduces additional delays making it horribly painful to use for things like teleop with large messages like images or point clouds.

From my perspective, interest, especially commercially, in getting ROS to work with the web have only grown over time. There has been push by Bosch and Brown for a while, and then Willow joined in, right before it closed shop, to build a complementary web toolkit [1] for ROS. Savioke, from their job postings, seem to be doing something webby behind the scenes too. And even for hobby/research projects, it's just so much easier to access robots over a browser compared to with Ubuntu/RViz. The fact that you get iOS and Android support, no ROS java [2] needed, almost for free through their browsers is just fantastic (and I suggest trying this if you haven't already).

I might have missed it in my superficial lurking, but I haven't seen this issue of communicating with web clients raised with any seriousness yet. It would be a big missed opportunity if ROS 2.0 only supports talking to browsers at the level that it does now. Most projects are moving to the web. These days even my humble ipython runs a full-fledged web server in its default installation. Certainly, in the future, I feel like this will become a much more pressing issue.

[1] http://robotwebtools.org

[2] https://code.google.com/p/rosjava/

[William Woodall]

I think the limiting factor here is that browsers only support websockets as a means to communicate with other processes (without browser specific extensions). The overhead of doing socket communication over HTTP leads to the difficulty with high frequency messages like tf. I'll try to address some other things inline.

On Tue, Sep 16, 2014 at 1:01 PM, Hai Nguyen <hai...@gmail.com> wrote:

I'm not particularly familiar with DDS, but does it work nicely with browsers? either through some sort of translation or better yet, natively?

People have been using DDS with websockets:

http://www.slideshare.net/Angelo.Corsaro/dds-on-the-web-quick-recipes-for-realtime-web-applications

There is a DDS-web specification "in progress", the beta of the spec was released in November 2013:

http://www.omg.org/spec/DDS-WEB/

Briefly looking at it, they plan to do what we do, which is to run a native DDS program and have access to data from the browser go through that program to get data using websockets.

 

I'm interested as the current ROS to WebSockets bridge is particularly ugly: the bridge has to subscribe to all the messages that any web client would need to listen to and then rebroadcast them, which introduces additional delays making it horribly painful to use for things like teleop with large messages like images or point clouds.

I'm not convinced the rebroadcast is the core performance issue here, but you could imagine a system where every node has a websocket server, and then the rosbridge node acted as a lookup service for the web browser, allowing it to connect directly to any node. However, I'm not convinced this is a better solution because it adds a lot of complexity. I really think that the over head of converting to JSON and then sending data over HTTP using websockets is the main performance problem, but I could be wrong about that since I don't have any anecdotal evidence to support it. If that is the case though, there is nothing to be done about it, since the only portable way that I know of to communicate with web browsers is through websockets using JSON.

 

From my perspective, interest, especially commercially, in getting ROS to work with the web have only grown over time. There has been push by Bosch and Brown for a while, and then Willow joined in, right before it closed shop, to build a complementary web toolkit [1] for ROS. Savioke, from their job postings, seem to be doing something webby behind the scenes too. And even for hobby/research projects, it's just so much easier to access robots over a browser compared to with Ubuntu/RViz. The fact that you get iOS and Android support, no ROS java [2] needed, almost for free through their browsers is just fantastic (and I suggest trying this if you haven't already).

All of that is true, I don't think there is anything better DDS or ROS could do from a technical perspective though. I'm open to ideas on that front, it might be good to bring up with other robot web tools people.

 

I might have missed it in my superficial lurking, but I haven't seen this issue of communicating with web clients raised with any seriousness yet. It would be a big missed opportunity if ROS 2.0 only supports talking to browsers at the level that it does now. Most projects are moving to the web. These days even my humble ipython runs a full-fledged web server in its default installation. Certainly, in the future, I feel like this will become a much more pressing issue.

This is certainly true, but in my opinion the best way to support this is to have things like a native node.js client for ROS, which should be easy since it is extended using C++. Which gives you access to all the sophisticated web service development tools with a native connection to the ROS nodes.

ipython also using websockets just like rosbridge, there is nothing that I can see that they do fundamentally different from how rosbridge does things.

 

[1] http://robotwebtools.org

[2] https://code.google.com/p/rosjava/

--
You received this message because you are subscribed to the Google Groups "ROS SIG NG ROS" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ros-sig-ng-ro...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

William Woodall

ROS Development Team

wil...@osrfoundation.org

http://wjwwood.io/

[Robert Dean]

I think there is a somewhat obscured issue here. The issue is not necessarially the efficency of sending json or xml from a bridge but more fundamental one of data rate.  A web user does not necessarially need/want the full fire hose of tf data, but rather updates at 1-5Hz.  At these slower rates the overhead of converting to a web native format is trivial.   To the best of my knowledge, under current ROS1, there is no way to throttle the data flow from a publisher to subscribers on an individual basis. The throttling is done on the client side, yes?

Does DDS and/or the ros 2.0 design provide for a client stating "give me data X on topic Y at rate Z"?  This capability is potentially incredibly useful for managing run time resources.

On a somewhat related note, converting the to/from string representations created by the message generator to be json formatted would be very useful.  It is highly readable and easilly parsed (and parsed faster than xml).

[Jon Binney]

The solution I've used for this to create a separate topic, and publish throttled messages to it using http://wiki.ros.org/topic_tools/throttle . Then you have the original topic /foo, and the new topic /foo_throttled. The advantage of explicitly having a separate topic is that if you "rosbag record" from a topic, you get exactly the set of messages that a node who subscribes to that topic would get.

[Hai Nguyen]

I am not a WebSocket expert but from my understanding, HTTP is used just for the initial handshake so high frequency tf messages should work. And although I don't have measurements either, the current process where we encode to ROS, send, decode from ROS, encode to JSON, send, and finally decode from JSON is certainly much slower (high latency) and processor intensive than if messages started out as JSON. You also don't strictly need to send JSON at the server end—not that it's that much different from ROS messages—as WebSockets doesn't define a serialization format. For example, you can always send just binary blobs.

I agree that having WebSockets in each node, in addition to existing protocols, would be nice but perhaps complex. From what I can see with that linked DDS document, they're just talking about another bridge solution and that is the solution that we have now, which is kind of terrible. Although I didn't expect much since OMG, as you put it, is "slow to adapt to changes and therefore arguably doesn’t always keep up with the latest trends in software engineering." It looks like, for OMG, the web falls into this category of "latest trend in software engineering" that they don't "always keep up with." 

Personally, I feel like maybe having WebSockets be specifiable as a transport, instead of TCP or more exotic protocols, would seem like a good compromise.

Totally not related, but I've just (re)noticed that rosmaster and the parameter server runs on XMLRPC, which is a whole lot simpler to use and more sophisticated than ROS services (supporting things like exceptions), so why does ROS services need to be there at all? I feel like the master, in ROS 2.0, should hand out plain xmlrpc connections to services when requested by name.

On Tue, Sep 16, 2014 at 2:38 PM, William Woodall <wil...@osrfoundation.org> wrote:

--
You received this message because you are subscribed to a topic in the Google Groups "ROS SIG NG ROS" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/ros-sig-ng-ros/BfvcjD6Rsg0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to ros-sig-ng-ro...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Hai Nguyen

http://haidai.tumblr.com

[William Woodall]

On Tue, Sep 16, 2014 at 3:17 PM, Hai Nguyen <hai...@gmail.com> wrote:

I am not a WebSocket expert but from my understanding, HTTP is used just for the initial handshake so high frequency tf messages should work.

You're right, I was thinking the actual transport was over HTTP, but it is only for the handshaking, I suppose I confused this detail with how REST works.

 

And although I don't have measurements either, the current process where we encode to ROS, send, decode from ROS, encode to JSON, send, and finally decode from JSON is certainly much slower (high latency) and processor intensive than if messages started out as JSON. You also don't strictly need to send JSON at the server end—not that it's that much different from ROS messages—as WebSockets doesn't define a serialization format. For example, you can always send just binary blobs.

As Bob Dean suggested we could make it so that the message is deserialized directly into JSON (we want to support other in memory formats like OpenCV and PCL types as well). This would make the bridge more efficient. But I think the websockets need to still all go over 80 correct? meaning that you need a single point of entry to a remote machine. It seems like a more natural architecture to have one process serve as the gateway to a ROS system for the browser.

 

I agree that having WebSockets in each node, in addition to existing protocols, would be nice but perhaps complex. From what I can see with that linked DDS document, they're just talking about another bridge solution and that is the solution that we have now, which is kind of terrible. Although I didn't expect much since OMG, as you put it, is "slow to adapt to changes and therefore arguably doesn’t always keep up with the latest trends in software engineering." It looks like, for OMG, the web falls into this category of "latest trend in software engineering" that they don't "always keep up with." 

Well the fact that they have been discussing it for over two years even though websockets are only about three years old, and that they have a beta specification out is pretty good for a standards body, IMO. And I have yet to see a technical alternative to the bridge design for communicating with a middleware from a browser.

 

Personally, I feel like maybe having WebSockets be specifiable as a transport, instead of TCP or more exotic protocols, would seem like a good compromise.

Websockets are over TCP, I'm not sure what you mean here. I don't think it is a good idea to have multiple wire protocols for ROS officially. I think in this case having a bridge into another system like websockets is the right thing to do, but I might be convinced otherwise if we come up with a compelling technical solution which involves this.

 

Totally not related, but I've just (re)noticed that rosmaster and the parameter server runs on XMLRPC, which is a whole lot simpler to use and more sophisticated than ROS services (supporting things like exceptions), so why does ROS services need to be there at all? I feel like the master, in ROS 2.0, should hand out plain xmlrpc connections to services when requested by name.

I've seen that XMLRPC is not suitable for high frequency service requests and because it relies on XML it is difficult if not impossible to implement on embedded systems (see rosc's difficulties with XML parsing). Also ROS Services report errors too. In ROS 2.0 we will likely use DDS's RPC system once it becomes available, allowing us to keep one dependency for both pub/sub and services. Since ROS 2.0 will be using DDS, there will be no ROS master, DDS has a distributed discovery system which uses UDP multicast. This is another reason a bridge is technically necessary to communicate with a browser, since the browser cannot discover it's peers in the ROS network without a proxy in the ROS network (the bridge).

[William Woodall]

On Tue, Sep 16, 2014 at 2:50 PM, Robert Dean <bob....@gmail.com> wrote:

I think there is a somewhat obscured issue here. The issue is not necessarially the efficency of sending json or xml from a bridge but more fundamental one of data rate.  A web user does not necessarially need/want the full fire hose of tf data, but rather updates at 1-5Hz.  At these slower rates the overhead of converting to a web native format is trivial.

There are many tools in robot web tools to do just this, like slowing down tf and compressing pointclouds and such.

 

To the best of my knowledge, under current ROS1, there is no way to throttle the data flow from a publisher to subscribers on an individual basis. The throttling is done on the client side, yes?

Yes throttling must be done on the subscriber side, which is just much simpler (consider multiple publishers and multiple subscribers).

 

Does DDS and/or the ros 2.0 design provide for a client stating "give me data X on topic Y at rate Z"?  This capability is potentially incredibly useful for managing run time resources.

I'm not sure, but during our research this idea seemed to complicate things like udp multicast, where the pub -> sub communication is not connection oriented. I think it may be more visible if you have the explicitly throttled topics using throttle nodes like Jon Binney suggested. That way it is obvious from looking at the graph that this is occurring. This is sort of dog fooding with convention over special features with configuration seems to be a good strategy to me. In which case we need to make it low cost to create throttled topics (in terms of run time resources and developer time).

[Geoffrey Biggs]

On 17/9/2014, 4:50, Robert Dean wrote:
> Does DDS and/or the ros 2.0 design provide for a client stating "give me
> data X on topic Y at rate Z"? This capability is potentially incredibly
> useful for managing run time resources.

DDS provides the TIME_BASED_FILTER QoS policy. It is used by a
DataReader to state that it only wants a subset of the broadcast
messages, at a given frequency.

The DDS specification does not give any guidance on how to implement
this policy nor its impact on what actually goes over the wire. It only
describes the behaviour as seen by the DataReader. While an ideal
implementation may not send more samples than a DataReader wants to
conserve bandwidth, it is not as simple as that in reality. As William
indicated, the layout of the system as a whole complicates when to send
and when not to send. For example, if there are multiple DataReaders in
the same computing node receiving at different rates, then it costs less
in bandwidth to send all samples to the node and let the individual
DataReaders take what they want, but at the same time this may raise
computing costs for each DataReader.

This is the sort of thing that DDS implementers spend a lot of time
trying to get right.

Geoff

[Ben Kehoe]

Throttling wouldn't work on /tf, right? You'd have to have a special node that subscribed to /tf, and intelligently extracted the updates to the tree to send at a lower rate?

--
You received this message because you are subscribed to the Google Groups "ROS SIG NG ROS" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ros-sig-ng-ros+unsubscribe@googlegroups.com.

[William Woodall]

Yup, this is exactly what tf2_web_republisher does:

http://wiki.ros.org/tf2_web_republisher

https://github.com/RobotWebTools/tf2_web_republisher

To unsubscribe from this group and stop receiving emails from it, send an email to ros-sig-ng-ro...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Ben Kehoe]

That seems important enough to move to tf2 proper, no? Especially in the move to ROS 2.0. I guess that's off-topic though.

[William Woodall]

That maybe.

[Robert Dean]

The question comes from experiences with real-time and safety critical robotics. Granted web interfaces are a different beast.

4D/RCS style designs, for example, have strict definitions of how components are structured and how they communicate. Therefore if component A requires data X at 2Hz and component B requests data from the same publisher at 4Hz, then the publisher is designed to run at >4Hz, and the comms solution delivers the data on time. The consumers are designed to receive data at designed rates. It seems from Geoffrey Biggs's post that DDS handles these cases. RCSLIB definitely does.  RCSLIB may be more efficent in some ways, but that's a gray area related to threading I can debate with a DDS rep next time I run into one at a conference.

For these approaches, the comms solution does need to be more intelligent, and complex. The component and system designs, however, are much simpler and easier to validate and verify at design and runtime. From this viewpoint the need to add throttling or bridging nodes increases system complexity. The addition of queueing makes the real time bounds difficult (or impossible) to calculate and prove. 

[Hai Nguyen]

You are right that it's usually more ideal to have just one web socket connection rather than a connection for each topic. I was accepting it as an unfortunate downside of talking to ROS nodes, but now that you've raised the point, it seems to be more serious of a downside than I thought. Specifically, it'd be difficult to do things like implement access control for each topic, which is particularly vital for being on the web.

After looking into this a little more, however, it looks like whichever DDS system is used, if there is support for some sort of shared memory communication then having a bridge shouldn't be a big performance hit anyway, especially if the marshaling format ends up being JSON.

Since it's all contingent on shared memory access, is OSRF leaning towards a particular implementation of DDS that has this feature?

I do agree with Chad though. It's rather bold to assume that most of the best feedback for ROS 2.0 will be from people that are interested enough to join yet another mailing list, especially since this one is so relevant. From the earliest days, ROS's design has been informed and improved by constant interactions with roboticists. For the benefit of ROS 2.0, I think this process should continue since there is a very real possibility that bad design decisions will result in developers just continuing to use ROS 1.x or even worse, not updating their libraries to support it.

[William Woodall]

On Thu, Sep 18, 2014 at 3:02 PM, Hai Nguyen <hai...@gmail.com> wrote:

You are right that it's usually more ideal to have just one web socket connection rather than a connection for each topic. I was accepting it as an unfortunate downside of talking to ROS nodes, but now that you've raised the point, it seems to be more serious of a downside than I thought. Specifically, it'd be difficult to do things like implement access control for each topic, which is particularly vital for being on the web.

These are good points, the gateway enables better access control from the browser and simplifies the out going connections from the browser.

 

After looking into this a little more, however, it looks like whichever DDS system is used, if there is support for some sort of shared memory communication then having a bridge shouldn't be a big performance hit anyway, especially if the marshaling format ends up being JSON.

Since it's all contingent on shared memory access, is OSRF leaning towards a particular implementation of DDS that has this feature?

We will be trying to tackle the inter node communication performance from both directions, speeding up IPC and making it easier to avoid IPC. We will be working hard to make nodes which share a process (no need for IPC) more transparent, i.e. basically fix nodelets. Also if the DDS implementation supports them, optimizations like shared memory and/or UPDM will be used.

We haven't settled on a default, but unless RTI changes their license it will probably be PrismTech's OpenSplice (LGPL). We are also evaluating tinq (clear BSD). I believe that all three have shared memory, but I'm unsure about the ones with UDPM.

 

I do agree with Chad though. It's rather bold to assume that most of the best feedback for ROS 2.0 will be from people that are interested enough to join yet another mailing list, especially since this one is so relevant. From the earliest days, ROS's design has been informed and improved by constant interactions with roboticists. For the benefit of ROS 2.0, I think this process should continue since there is a very real possibility that bad design decisions will result in developers just continuing to use ROS 1.x or even worse, not updating their libraries to support it.

I don't assume that the best feedback for ROS 2 will be from people that join the sig mailing list. But I do assume that they are not interested in hearing about all the technical discussions, but rather would like to comment on the result of those discussions which result in either a set of proposals on how to proceed or some form of consensus from the sig members.

I believe the right way to proceed is by announcing on ros-users that a discussion is taking place on a sig mailing list in order to invite people to join the conversation, and then reporting back on ros-users when a proposal or consensus has been reached. But I completely disagree that every email in a discussion, for example all the emails in this thread, needs to go to the 1700+ members of ros-users. If the use of sig mailing lists prevents even one person from unsubscribing from ros-users then it is totally worth it in my opinion. ros-users is basically the only mechanism by which we can communicate with most of the community.

I asked you to move your conversation here because I identified it as part of the development effort and that it was not part of the core question that Chad asserted (imo correctly) should be discussed on ros-users. Your question was a question about support for a single technical feature amongst many which must be discussed for ROS 2. Your post had no proposal on how to proceed from a policy or technical perspective nor any consensus which was ready for consumption by all of ros-users. We've made progress on this topic on this mailing list, and once we reach some conclusion we can report back to ros-users.

I'm not trying to exclude anyone or prevent discussion, but I am advocating for isolating ros-users from lots of traffic. Minimizing the traffic on ros-users has been something the community has been working on from the earliest days (to borrow a phrase) of ROS, and it resulted in answers.ros.org, SIG's, API reviews on the wiki, and REP reviews off channel, all of which were moved away from ros-users at some point.

[Ingo Lütkebohle]

Am 19.09.2014 00:03 schrieb "Hai Nguyen" <hai...@gmail.com>:
>
> You are right that it's usually more ideal to have just one web socket connection rather than a connection for each topic.

Many Middleware packages can multiplex multiple streams over one TCP connection. Might be simmering to look for.

> I do agree with Chad though. It's rather bold to assume that most of the best feedback for ROS 2.0 will be from people that are interested enough to join yet another mailing list, especially since this one is so relevant. From the earliest days, ROS's design has been informed and improved by constant interactions with roboticists.

As someone whose research interests are founded on making teams more efficient, I can say that talking about the means of communication, while important, is secondary. The most important thing is the process (yeah, I know, that's what the architecture astronauts also say, and nobody likes those, but there is always a process anyway, so better consciously simplify it).

You don't get large involvement by exposing all people to every messy, technical discussion. That'll just scare them away, or make them ignore mails. You *also* don't get it by doing everything behind closed doors, of course.

Both you and Chad have asked for discussing, essentially, important things here. However, who's going to make that call?

*I* think there's always a tradeoff here, and that the only way to get around it is not to discuss more initially, but to release early and often, and taking release feedback into account.

For that to work, we can look at the Linux kernel approach, e.g. its commitment to compatibility between releases and Linus's style in general. We can also look at apache httpd 1.3 to 2.0, for what *not* to do (cf second system syndrome).

In that vein, it seems that the biggest problem right now is that ROS 2.0 is trying to do very many things at once, instead of staggering them. That's bound to be difficult, however you go about it.

I don't know the technical reasons for that, but I think it's interesting that other groups apparently managed to get DDS with ROS 1.0 APIs. Maybe something can be learned from that.

In any case, ROS has become larger and more complex now, and the core team actually became smaller. So, what worked the first time around is not necessarily the way to go now. Some adaptation will likely be necessary.

I for one would certainly be willing to help osrf by joining a special list. I can't commit to that right now because I'll be starting a new job soon, but just saying that I wouldn't consider that too large a burden. Of course, not everybody can or wants do that, that's why I think we need different forms of involvement.

Cheers,
Ingo

> _______________________________________________
> ros-users mailing list
> ros-...@lists.ros.org
> http://lists.ros.org/mailman/listinfo/ros-users
>