ROS 2 Update: Interface Description Format changes

[Jackie Kay]

We wanted to let everyone know about some changes in the interface (that’s ROS 2 parlance for `.msg` messages and `.srv` services) description format in ROS 2.

So far there are a few non-breaking extensions like optional upper bounds on dynamic arrays or strings and default values for primitive fields. You can read more about those in this work-in-progress design document:

http://design.ros2.org/articles/interface_definition.html

We're also looking at breaking changes to interfaces, which prevent you from copy-pasting a ROS 1 interface file into ROS 2 and having it work. Unfortunately, there are already a few instances of this, but they won't affect all interface files. A quick list of the existing issues:

Time and Duration are now messages and not built-in types. This is a good thing, because it makes our builtin types more portable. For example, you don't need to depend on rostime (a C++ library) to use our generated message code. Typically this means you only have to change `time stamp` to something like `builtin_interfaces/Time stamp`.

The Header message is no longer a "special" message type. In ROS 1 you can just use `Header` as a type regardless of the package you're in. In ROS 2 you must specify the prefix as you would with any other message that's not in the same package (an example of dog fooding [1]). This typically results in changes to lines like this: `Header header` changes into `std_interfaces/Header header`.

This brings us to another migration point. We've named the message packages differently, e.g. `std_interfaces` rather than `std_msgs` and `std_srvs`. We didn't do this on a whim, but because it makes creating a program that uses ROS 1 and ROS 2 at the same time possible. There are technical limitations with the CMake infrastructure which make it impossible to `find_package` a ROS 2 package and ROS 1 package of the same name at the same time. This is our current best approach for how to make the ROS 1/ROS 2 bridge. If a better strategy which allows packages of the same name to coexist comes up we will consider it.

If you want to discuss any of these issues in-depth, please start a new email thread for it.

We’ll follow up on this email with some RFC’s about other potentially breaking changes to the interfaces in ROS 2 which would add to the migration burden. So far all of the changes are potentially scriptable, except for the message package names (which could be scriptable with user input).

cheers,

Jackie

[1] https://en.wikipedia.org/wiki/Eating_your_own_dog_food

[Thibault Kruse]

Hi,

I am interested in how the differences will affect ROS1-2 interoperability.

I saw a github project offering bridges for specific topics and message types (https://github.com/ros2/ros1_bridge), which I believe to be a proof-of-concept prototype.

I also saw a package providing a ROS2 equivalent of ROS1 common_msgs (https://github.com/ros2/common_interfaces)

So is there any plan to have generic or standard mappings between ROS1 and ROS2 message definitions, and a plan to provide safe and performant generic bilateral mappings. By generic mappings I would understand a node that given a ROS1 and a ROS2 message definition plus maybe other namepace input would make a mapping by convention that fails if the conventions were not followed in the message definitions. By standard mapping I would understand a process that describes for a number of existing ROS1 and ROS2 message type pairs how to transform in either direction.

Things that seem a bit worrying are having a bridge node transmitting a message from ROS1 to ROS2, then this message then being back transmitted to ROS1 via the same or a different bridge. Or bridges disagreeing with the mapping of message types between ROS1 and ROS2 on different topics. Let alone the problems involved with modifying a message definition in either ROS2, and then having to make sure the equivalent message definition in ROS1 and all mappers being updated.

Or is ROS1-ROS2 interoperabilty regarded as something that is technically feasible, but probably too much effort?

[Dirk Thomas]

Hi Thibault,

yes, the referenced repository contains the current prototype for the bridge between ROS 1 and ROS 2.

Currently it only uses hard coded rules to map between ROS 1 and ROS 2 messages.

The approach is currently:

* Basically any ROS 1 package ending with `_msgs` corresponds to a ROS 2 package ending with `_interfaces`.

* If a message with the same name exists in these two packages it is considered.

* To actually be able to convert the message it also checks that all field names are equal (minus potential lower casing in ROS 2).

Beside that mapping-by-convention the idea is to also allow a configuration file which defines different mappings.

This has not been implemented in the prototype yet but it should allow to specify arbitrary mappings as long as the actual data is convertable.

The differences in the format (default values as well as upper boundaries) should not affect the bridge much.

The default value is not relevant since "on the wire" the value will be send anyway.

The upper boundary is already being checked in the current prototype.

If a message has an upper bound in ROS 2 but the received ROS 1 message is bigger the message will simply be dropped.

Regarding the mentioned problem of routing messages from the ROS 1 side of the bridge to the ROS 2 side and then back to ROS 1:

This has already been addressed in the current prototype.

For ROS 1 the "caller id" is being used to drop "loopback" messages within the bridge (https://github.com/ros2/ros1_bridge/blob/d0ee70e25555ac573a1b918d923f59008dad6e98/include/ros1_bridge/factory.hpp#L94-L99).

For ROS 2 this is done using the knowledge about the sending participant of a message which is usually available through vendor specific API (https://github.com/ros2/rmw/blob/0f0d7a2e31ca6b3e3a169e2bd04356324d19529d/rmw/include/rmw/rmw.h#L72).

The bridge inherently relies on receiving messages from one ROS version, deserializing them, converting them to the message instance of the other ROS version, serialize it again and then publish it on the other ROS version.

While the overhead will be noticeable it should be huge when running locally on the same machine.

But we haven't done any real bench marking - neither with the bridge nor with the ROS 2 prototype in general.

The next step for the bridge will be to let it connect topics when they come up on both sides.

The prototype does also not yet address bridging of services, actions, parameters or dynamic reconfigure.

Cheers,

- Dirk

--
You received this message because you are subscribed to the Google Groups "ROS SIG NG ROS" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ros-sig-ng-ro...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Jack O'Quin]

What is the use case for mixing ROS1 and ROS2 in the same package?​

I'd prefer that the message package names remain the same.

[Jack O'Quin]

​Let's drop the primitive types ``byte`` and ``char``. 

They have long been deprecated in ROS1. Their original definitions made no intuitive sense, anyway. In the C language standard, ``char`` could be either signed or unsigned. 

 http://wiki.ros.org/msg#Fields 

Let the ROS1 -> ROS2 bridge convert them to the corresponding aliases.

[Dirk Thomas]

The generated code for ROS 1 messages is very different from the generated code for ROS 2 messages.

E.g. the ROS 2 code includes new features (default values, upper boundaries) as well as new API (e.g. setter functions for all fields).

Additionally conversion functions from / to the various DDS vendors are generated for ROS 2 messages.

Therefore the dependencies of a message package in ROS 1 and ROS 2 are different, since they require different code generators.

Beside that ROS 1 and ROS 2 packages use a different CMake infrastructure (catkin vs. ament) to be built (hopefully we can publish an article about ament in the near future).

Later is necessary since ROS 2 require feature in the build process which are not available in ROS 1.

Therefore mixing ROS 1 and ROS 2 in the same package is not possible.

Based on the experience with "mixing" rosbuild with catkin that is also not a goal for ROS 2.

It is intended to keep ROS 1 and ROS 2 completely separate.

That being said there is one important use case which requires to make it necessary to use ROS 1 message and ROS 2 messages at the same time.

A bridge should be able to receive messages from one ROS version and relay them to the other ROS version.

This software must be able to use C++ headers from both ROS 1 as well as ROS 2 packages.

Since packages are based on CMake and are being identified by their name (`find_package(<pkgname>)`) these packages must have unique names across the ROS versions.

Based on these requirements the message packages in ROS 2 have different names which are not colliding with ROS 1 package names.

Cheers,

- Dirk

On Fri, Jul 3, 2015 at 2:21 PM, Jack O'Quin <jack....@gmail.com> wrote:

What is the use case for mixing ROS1 and ROS2 in the same package?​

I'd prefer that the message package names remain the same.

--

[Dirk Thomas]

That is a good point about the `byte` and `char` types.

We have left them in for now because we saw some value especially when considering encodings and other types (e.g. `wchar`).

We should reconsider this after having specified the exact mapping of these types to some languages (e.g. C++ and Python).

I have added a todo to the article: https://github.com/ros2/design/commit/f79b8fb87b2421e467e1a3ff113aacf83f47ac98

Thanks,

- Dirk

--

[Jack O'Quin]

On Fri, Jul 3, 2015 at 5:26 PM, Dirk Thomas <dth...@osrfoundation.org> wrote:

Beside that ROS 1 and ROS 2 packages use a different CMake infrastructure (catkin vs. ament) to be built (hopefully we can publish an article about ament in the near future).

Later is necessary since ROS 2 require feature in the build process which are not available in ROS 1.

I am sorry to hear that.

 

Therefore mixing ROS 1 and ROS 2 in the same package is not possible.

Based on the experience with "mixing" rosbuild with catkin that is also not a goal for ROS 2.

It is intended to keep ROS 1 and ROS 2 completely separate.

That seems fine to me. 

 

That being said there is one important use case which requires to make it necessary to use ROS 1 message and ROS 2 messages at the same time.

A bridge should be able to receive messages from one ROS version and relay them to the other ROS version.

This software must be able to use C++ headers from both ROS 1 as well as ROS 2 packages.

Since packages are based on CMake and are being identified by their name (`find_package(<pkgname>)`) these packages must have unique names across the ROS versions.

Based on these requirements the message packages in ROS 2 have different names which are not colliding with ROS 1 package names.

So, how does one handle messages defined in other ROS packages, which may not use the ``_msgs`` suffix?

Is that outside the scope of the message bridge?

If the only multi-version package build is for the bridge, why not invent a way to handle that special case, instead of forcing global renaming of message types?

--

 joq

[William Woodall]

We've given some thought to how to do just that (handle the special case), but we've not come up with a solution that works well in practice. It really just boils down to the fact that they both use CMake and you need to find_package(foo) twice, once from ROS 1 and once from ROS 2, and get different results each time. It would be great to have a solution or a pattern that would allow us to overcome that without introducing lots of special changes in either the ROS 1 or ROS 2 version of the package. It should be relatively straight forward scenario to setup, just create a package with messages in ROS 1 and then a package in ROS 2 with the same name and messages. Then try to write another package, in the style of the current bridge, which uses ROS 1 and ROS 2 at the same time, as well as both versions of the message packages. If anyone can make that work in a clean way I think it would interesting to discuss and pursue.

As a result we've been naming packages differently on purpose so that the core ROS 2 and ROS 1 stacks don't have name collision. That's worked so far because we can take the time to organize it in such a way, but I agree that's not an ideal reality for each ROS 1 package you'd like to migrate to ROS 2.

One hopeful idea is that we'll be able to have a bridge in the future that's perhaps not as performant, but that can work without needing to depend on generated code from the ROS 2 message packages. Instead it would find them at run time and dynamically publish and subscribe to them based on the message idl file alone. That would allow you to have the same package name in ROS 1 and ROS 2 without running into the issue of how to deal with both at build time. However, that's just an idea at this point and so for the first version of the bridge which relies on generated code, in both ROS 1 and ROS 2, it was easier to change the names.

Cheers,

--

 joq

--
You received this message because you are subscribed to the Google Groups "ROS SIG NG ROS" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ros-sig-ng-ro...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

William Woodall

ROS Development Team

wil...@osrfoundation.org

http://wjwwood.io/

[Jack O'Quin]

On Fri, Jul 3, 2015 at 7:56 PM, William Woodall <wil...@osrfoundation.org> wrote:

On Fri, Jul 3, 2015 at 4:56 PM, Jack O'Quin <jack....@gmail.com> wrote:

If the only multi-version package build is for the bridge, why not invent a way to handle that special case, instead of forcing global renaming of message types?

We've given some thought to how to do just that (handle the special case), but we've not come up with a solution that works well in practice. It really just boils down to the fact that they both use CMake and you need to find_package(foo) twice, once from ROS 1 and once from ROS 2, and get different results each time. It would be great to have a solution or a pattern that would allow us to overcome that without introducing lots of special changes in either the ROS 1 or ROS 2 version of the package. It should be relatively straight forward scenario to setup, just create a package with messages in ROS 1 and then a package in ROS 2 with the same name and messages. Then try to write another package, in the style of the current bridge, which uses ROS 1 and ROS 2 at the same time, as well as both versions of the message packages. If anyone can make that work in a clean way I think it would interesting to discuss and pursue.

I can't help much with the build issues. I just learned we are inventing yet another ROS build system. :-(

Looking at the ros1_bridge/CMakeLists.txt, I can imagine getting access to the ROS 1 messages using something like...

  find_package(ros1 REQUIRED std_msgs)

 

instead of... 

  find_package(std_msgs)

That could provide enough wrapper to allow forcing the ROS 1 messages into a distinguishable directory structure and handle the CMake differences.

As a result we've been naming packages differently on purpose so that the core ROS 2 and ROS 1 stacks don't have name collision. That's worked so far because we can take the time to organize it in such a way, but I agree that's not an ideal reality for each ROS 1 package you'd like to migrate to ROS 2.

Agreed, and once we get beyond the core, things get out of hand in a hurry. Every reference to a message in code or other messages, would have to be renamed.

 

One hopeful idea is that we'll be able to have a bridge in the future that's perhaps not as performant, but that can work without needing to depend on generated code from the ROS 2 message packages. Instead it would find them at run time and dynamically publish and subscribe to them based on the message idl file alone. That would allow you to have the same package name in ROS 1 and ROS 2 without running into the issue of how to deal with both at build time. However, that's just an idea at this point and so for the first version of the bridge which relies on generated code, in both ROS 1 and ROS 2, it was easier to change the names.

That seems difficult, but worthwhile. I have little idea when anyone might have time to work on something like that. It does not sound like it would have a very high priority.

--

 joq

[Jack O'Quin]

On Fri, Jul 3, 2015 at 8:32 PM, Jack O'Quin <jack....@gmail.com> wrote:

Looking at the ros1_bridge/CMakeLists.txt, I can imagine getting access to the ROS 1 messages using something like...

  find_package(ros1 REQUIRED std_msgs)

Correction, I meant...

  find_package(ros1 REQUIRED COMPONENTS std_msgs)

 

 instead of... 

  find_package(std_msgs)

That could provide enough wrapper to allow forcing the ROS 1 messages into a distinguishable directory structure and handle the CMake differences.

--

 joq

[Dirk Thomas]

Looking at the ros1_bridge/CMakeLists.txt, I can imagine getting access to the ROS 1 messages using something like...

  find_package(ros1 REQUIRED COMPONENTS std_msgs)

Could you please also elaborate how this would work internally?

I don't understand how the "ros1" CMake config file or CMake module will locate the ROS 1 package "std_msgs" is there is a ROS 2 package with the same name.

- Dirk

PS: ament is not "yet another ROS build system".

You could call it catkin 1.0 or catkin 2.0 which address problems in the current catkin version which have been identified in the last two/three years of usage.

Since we don't have the article about ament written yet it might not be obvious what the differences and similarities are - I will hopefully get to write that down in the near future.

[Jack O'Quin]

On Fri, Jul 3, 2015 at 8:58 PM, Dirk Thomas <dth...@osrfoundation.org> wrote:

Looking at the ros1_bridge/CMakeLists.txt, I can imagine getting access to the ROS 1 messages using something like...

  find_package(ros1 REQUIRED COMPONENTS std_msgs)

Could you please also elaborate how this would work internally?

I don't understand how the "ros1" CMake config file or CMake module will locate the ROS 1 package "std_msgs" is there is a ROS 2 package with the same name.

Since I don't know how it works for either ROS 1 or ROS 2, I am not likely to be able to help you with any details. I was just pointing out that you can use ``find_package(std_msgs)`` for the new headers, while still finding the old ones in some fashion similar to the way catkin did it.

Since this would be a special hack mainly for the message bridge, I imagine it might use some ugly CMake or environment variable to find the ROS 1 versions of the messages.  

PS: ament is not "yet another ROS build system".

You could call it catkin 1.0 or catkin 2.0 which address problems in the current catkin version which have been identified in the last two/three years of usage.

Since we don't have the article about ament written yet it might not be obvious what the differences and similarities are - I will hopefully get to write that down in the near future.

Giving it a completely new name will lead many to interpret it like I did: as a new build system. Calling it catkin 2.0 would suggest a different interpretation. 

I look forward to reading that article.
--

 joq

[Bob Dean]

If I understand the problem correctly, base on limited knowledge of ros build (we use rake, it's easier for our systems), the issue is specific to ros1 messages and the design goal of using rosbridge provides a single location in the infrastructure which contains he problem.  Instead of fixing it globally with cmake fix it locally in a cmake-y way.

I think the problem can be solved by a script and ros_bridge smarts that makes Jack's suggestion work: find_package(rosbridge ROS1_REQUiRED std_msgs ROS2_REQUIRED jack bob). 

1 Ros bridge calls a script which searches the directory tree, and copies ros1 msgs into a sandbox within rosbridge. Or an index file of some sort. Given all the package and manifests ands such, there should be a way to distinguish ros1 from ros2 repos. Or alternatively add a ros version variable to the ament stuff which defaults to tos1 if not defined.

2 rosbridge includes a slightly modified ros1 Msgen which places the messages in a nested ros1 namespace, such as ros1:std_msgs::Header, to prevent compile issues

3 the rosbridge find-package cmake file knows from the special tag you want the ros1 version, and checks the sandbox for the sub directory. Similar to boost.

So this requires rosbridge cmake to be run prior to the ros2 dependencies.  This is similar to how I import ros msgs into our build system, it's worked reliably for about two years now. Minus the ros1 namespace as there are no naming conflicts yet, but when the time corns I change a line in the import and it's fixed.  The search is very very fast and should be able to check the need to reimport msgs as well.

On the interface side, I would suggest allowing lower camel case as variable names. It is a very common convention, and allowed by ros1 msggen, and if a company has been using it for their messages due to company naming conventions, moving to ros2 becomes a substantially larger effort. Nearly every line of code may need to be touched in some cases. In an industry where engineer time is >$1000 a day that cost adds up very fast and is a factor in choosing ros2 over another solution. It would probably be cheaper for he company to hack the msggen scripts to allow it. 

Lower camel case is not an excuse for non descriptively  named variables.

Sent from my iPhone

--

[Vincent Rabaud]

ok, re-posting on that thread after following @tkruse:

In case some messages change for whatever reasons (and therefore have different ROS1 and ROS2 IDLs), would there be an incentive in following some ISO standards ?
http://www.iso.org/iso/home/store/catalogue_tc/catalogue_tc_browse.htm?commid=54138&published=on

(I do not have access to them but there is one about coordinate systems, and there is one about mobile platform vocabulary coming out: http://www.iso.org/iso/home/store/catalogue_tc/catalogue_detail.htm?csnumber=65658)

On Thursday, July 2, 2015 at 2:23:33 AM UTC+2, Jackie Kay wrote:

[William Woodall]

I think it would be great to make sure we're using the common (or even standard) vernacular in our messages, but that has to be weighed against disruption incurred in the migration between ROS 1 and ROS 2. I would be in favor of making any changes which make sense in ROS 2, but obviously that's not the opinion held by everyone and that's ok.

In any case where our terminology or practices diverge from ISO, it would be worth putting a comment about that in the affected messages. So at least people that are familiar with the ISO standards can related them to our style. Higher level discrepancies will have to be captured in wiki level documentation.

As with all of these suggestions (https://groups.google.com/d/msg/ros-sig-ng-ros/YAyfgrvUvs0/-rU78R7S10AJ remove PointCloud, rethink PointCloud2, change the name of fields in Pose, etc.), I think it's best if we address them one at a time as issues or pull requests. So if you have access to the standards and notice a difference with our messages, then suggest that we change them (in a narrow as possible scope) and we can decide one by one what to do.

I do not have access to them but...

Not saying we shouldn't try to reuse conventions which exist, but I never understood how standards which are not freely available are supposed to gain adoption through any other means outside of coercion. I can't even look at the whole document to see if I agree with their definitions. Does anyone know how people can access these ISO standards in a free and legal way? I didn't see them on this page:

http://standards.iso.org/ittf/PubliclyAvailableStandards/index.html

--

You received this message because you are subscribed to the Google Groups "ROS SIG NG ROS" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ros-sig-ng-ro...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.