Re: [ros-sig-ng-ros] What will ROS1 and ROS2 share...

[Tully Foote]

On Sun, Jul 19, 2015 at 3:40 AM, Thibault Kruse <tibo...@googlemail.com> wrote:

On Saturday, July 18, 2015 at 7:51:52 PM UTC+2, Jack O'Quin wrote:

On Sat, Jul 18, 2015 at 5:09 AM, Thibault Kruse <tibo...@googlemail.com> wrote:

On Friday, July 17, 2015 at 12:56:12 PM UTC+2, William Woodall wrote:

On Fri, Jul 17, 2015 at 1:49 AM, Thibault Kruse <tibo...@googlemail.com> wrote:

So the already tight maintainership situation in ROS will become even tighter, as maintainers will have to split their time, or support only one side of the trench.

As one of the major contributors and maintainers of ROS 1 (even before doing it full time) I'm keenly aware of this. I, and I'm certain many others, agree with the sentiment, but I'm not sure how else to reconcile this with the fact that we need to continue to make improvement to the core of ROS in order to meet the needs of more users. I think the approach we've taken will be slower and require more effort on our part and on the communities part, but it will also be the least risky and least disruptive to the ROS 1 ecosystem.

I am a bit tired of hearing this "non-disruptive of ROS1" argument. Basically ROS2 is the preparation for the mother of all disruptions to the ROS1 ecosystem. A slow, large-scale migration of thousands of packages to a different middleware. I believe it would be more honest if OSRF said that OSRF prefers disruption by migration over disruption by evolution, or something.

ROS 1 non-disruption is not an argument. It is a requirement that has been forcefully communicated to OSRF by many (probably most) of the current ROS 1 community.

I know, I am not tired of the community requesting this. I am tired of OSRF using that as an excuse for creating a new competing middleware.

There are multiple interpretations of what the community means when it says "We want less disruptions in the future":

A) Do not improve ROS1, create a new competing middleware to which we'll have to slowly and painfully migrate.
B) Put all future improvements into ROS1, but make the changes backwards compatible and well-tested.

I believe B reflects better than A what the community wants.

There is also C:
C) Create yet another middleware if you must, but continue to improve ROS1, and make changes to ROS1 backwards compatible and well-tested

But C obviously requires much more effort to everyone, and is not realistic. As as you can see with catkin and ament, it is very tempting for OSRF to just leave ROS1 behind, and make improvement only in the ROS2 world. You can also read it in all the OSRF answers saying: "Improving catkin could have caused regressions, so we did not improve catkin". With that argument, no further improvements will be made to ROS1. The same goes for all other packages maintained outside OSRF with the expectable low level of code sharing.


The alternative that the community could well have expected would be to make the communication layer exchangeable under a common client API (which can be changed/enhanced over time using tic-toc deprecation). That would mean all further improvements would have benefitted the community, but the community would not have to change much or migrate. This is the obvious benefit of maximum code sharing.

Apparently OSRF does not want to do that, and the consequence will be a huge disruption to the ROS community, as opposed to the wish expressed by the community.

Note that this is not fatal to anyone, and maybe in 10 years people will look back and praise OSRF for the courage and wisdom. I only explain that this is the opposite of what the community wants now.

That's what I mean when I say I am tired of the "non-disruptive of ROS1" argument.

You're right that all three of those paths are possible. And they have been considered. All of the options have tradeoffs. If we break the API we can make fundamental changes to the design. If we require full backwards compatibility and roll it out in place it limits how much change can be made, and also risks breakage of the current system as it changes underneath. And obviously we can do both or some combination of both it just takes more work.

As outlined in the first article on ROS 2 design page we've chosen to break the API and maintain ROS 1.0 in parallel. http://design.ros2.org/articles/why_ros2.html

This is a common approach used by many software projects to create a major versioned number change which installs side by side with the old version. We are committing even further to make sure that the two versions can interoperate as a heterogeneous system, which is often not supported in major version changes of projects.

Our decision was based on feedback from users and potential users. We specifically reached out to ROS users as well as people and companies we know have chosen not to use ROS so we could identify the use cases not supported by the current version of ROS. From those conversations we've identified several use cases which are in demand but are not well supported at the moment. Many of these use cases were not considered in the current design and are either incompatible with the current design or would be significantly more complicated for users to use based on extending the current API rather than a new API designed to support the use cases.

What you currently know as ROS is actually the 5th or so generation of ROS. We did a lot of prototyping and testing before the 1.0 release, with completely different paradigms. In each iteration we learned a lot and took that into account in the next design. Currently each time we fork core ROS packages for a new rosdistro we give them new minor versions in the 1.X series. We've been calling this ROS 2.0 to communicate that it's a major change and we're changing the major version number to reflect another iteration of the design cycle. We converged on the ROS 1.0 design based on a set of use cases which were prototyped and tested.

Our goal is to use the lessons learned from the many years of use of ROS to create a replacement which will have compelling enough features that people will want to migrate their software forward. If they do not find the new features compelling they can keep using the current system. If we develop a system that we cannot convince people to use that will be a major failure and we can switch our development back to the ROS 1.0 branch. However I strongly believe that as a community we have learned a lot about the capablities and limitations of ROS and have enough expertise to develop a new version which is enough better to be worth switching.

At OSRF are currently working hard on rapidly prototyping to provide a functioning system to be able to get effective feedback from the community. As we go we are asking for comments and developing in the open. We hope that the community can see what we're doing and will continue to provide us with feedback, suggestions, and contribute if they have the time available.

--
You received this message because you are subscribed to the Google Groups "ROS SIG NG ROS" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ros-sig-ng-ro...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Dirk Thomas]

On Sun, Jul 19, 2015 at 3:40 AM, Thibault Kruse <tibo...@googlemail.com> wrote:

On Saturday, July 18, 2015 at 7:51:52 PM UTC+2, Jack O'Quin wrote:

On Sat, Jul 18, 2015 at 5:09 AM, Thibault Kruse <tibo...@googlemail.com> wrote:

On Friday, July 17, 2015 at 12:56:12 PM UTC+2, William Woodall wrote:

On Fri, Jul 17, 2015 at 1:49 AM, Thibault Kruse <tibo...@googlemail.com> wrote:

So the already tight maintainership situation in ROS will become even tighter, as maintainers will have to split their time, or support only one side of the trench.

As one of the major contributors and maintainers of ROS 1 (even before doing it full time) I'm keenly aware of this. I, and I'm certain many others, agree with the sentiment, but I'm not sure how else to reconcile this with the fact that we need to continue to make improvement to the core of ROS in order to meet the needs of more users. I think the approach we've taken will be slower and require more effort on our part and on the communities part, but it will also be the least risky and least disruptive to the ROS 1 ecosystem.

I am a bit tired of hearing this "non-disruptive of ROS1" argument. Basically ROS2 is the preparation for the mother of all disruptions to the ROS1 ecosystem. A slow, large-scale migration of thousands of packages to a different middleware. I believe it would be more honest if OSRF said that OSRF prefers disruption by migration over disruption by evolution, or something.

ROS 1 non-disruption is not an argument. It is a requirement that has been forcefully communicated to OSRF by many (probably most) of the current ROS 1 community.

I know, I am not tired of the community requesting this. I am tired of OSRF using that as an excuse for creating a new competing middleware.

There are multiple interpretations of what the community means when it says "We want less disruptions in the future":

A) Do not improve ROS1, create a new competing middleware to which we'll have to slowly and painfully migrate.
B) Put all future improvements into ROS1, but make the changes backwards compatible and well-tested.

I believe B reflects better than A what the community wants.

There is also C:
C) Create yet another middleware if you must, but continue to improve ROS1, and make changes to ROS1 backwards compatible and well-tested

But C obviously requires much more effort to everyone, and is not realistic. As as you can see with catkin and ament, it is very tempting for OSRF to just leave ROS1 behind, and make improvement only in the ROS2 world. You can also read it in all the OSRF answers saying: "Improving catkin could have caused regressions, so we did not improve catkin". With that argument, no further improvements will be made to ROS1. The same goes for all other packages maintained outside OSRF with the expectable low level of code sharing.


The alternative that the community could well have expected would be to make the communication layer exchangeable under a common client API (which can be changed/enhanced over time using tic-toc deprecation). That would mean all further improvements would have benefitted the community, but the community would not have to change much or migrate. This is the obvious benefit of maximum code sharing.

Apparently OSRF does not want to do that, and the consequence will be a huge disruption to the ROS community, as opposed to the wish expressed by the community.

Note that this is not fatal to anyone, and maybe in 10 years people will look back and praise OSRF for the courage and wisdom. I only explain that this is the opposite of what the community wants now.

That's what I mean when I say I am tired of the "non-disruptive of ROS1" argument.

I think you are contradict yourself here.

On the one hand you argue for having future changes in ROS 1 but in a backward compatible way.

But on the other hand you suggest to use a tic-toc cycle to introduce changes.

A tic-toc cycle is intrinsically not backward compatible because one round deprecates things and the other round removes them.

And since this would imply breaking existing ROS 1 code every other release we don't want to follow that idea.

Another example are features mentioned in other threads on this mailing list (e.g. in ament) or other changes in ROS 2 discussed in the past (e.g. in the ROS client library) which *do* require breaking changes.

On the one hand you seem to agree with the usefulness of at least a few of the proposed features and/or changes.

But on the other hand you request to introduce them in ROS 1 which by the nature of the changes would imply breaking existing code again (which you don't want to either).

For some of these features it has been discussed and explained with much details why (we think) they can't be achieved in a backward compatible way.

Simply asking for both directions without a proposal how those two directions should be achieve together is imo "wishful thinking" and does not help to evolve the discussion.

If you can think of any way to achieve certain new features in a backward compatible way we would very much appreciate if you could describe these ideas in detail.

As William wrote a lot of times: nothing is set in stone - with reasonable arguments almost any current design decision can be revisited.

- Dirk

[Jack O'Quin]

On Sat, Jul 18, 2015 at 1:10 PM, Dirk Thomas <dth...@osrfoundation.org> wrote:

On Sat, Jul 18, 2015 at 10:51 AM, Jack O'Quin <jack....@gmail.com> wrote:

On Sat, Jul 18, 2015 at 5:09 AM, Thibault Kruse <tibo...@googlemail.com> wrote:

On Friday, July 17, 2015 at 12:56:12 PM UTC+2, William Woodall wrote:

On Fri, Jul 17, 2015 at 1:49 AM, Thibault Kruse <tibo...@googlemail.com> wrote:

So the already tight maintainership situation in ROS will become even tighter, as maintainers will have to split their time, or support only one side of the trench.

As one of the major contributors and maintainers of ROS 1 (even before doing it full time) I'm keenly aware of this. I, and I'm certain many others, agree with the sentiment, but I'm not sure how else to reconcile this with the fact that we need to continue to make improvement to the core of ROS in order to meet the needs of more users. I think the approach we've taken will be slower and require more effort on our part and on the communities part, but it will also be the least risky and least disruptive to the ROS 1 ecosystem.

I am a bit tired of hearing this "non-disruptive of ROS1" argument. Basically ROS2 is the preparation for the mother of all disruptions to the ROS1 ecosystem. A slow, large-scale migration of thousands of packages to a different middleware. I believe it would be more honest if OSRF said that OSRF prefers disruption by migration over disruption by evolution, or something.

ROS 1 non-disruption is not an argument. It is a requirement that has been forcefully communicated to OSRF by many (probably most) of the current ROS 1 community.

Thank you, Jack, for stating it that clearly.

I really hope that we (as the whole community) reaches a consensus on this part: that it is a *requirement* and not an argument to discuss about for longer.

You're welcome! :-)

While accepting ROS 1 stability as an important goal of the new design, much still remains to discuss about ROS 1 and 2 compatibility, coexistence, and ease of migration. Reasonable people could disagree about all those topics.

For example: supporting ROS 1 builds via ament tools may eventually prove valuable. As long as we can still build ROS 1 with catkin, it should not affect stability at all. I see no reason to support it right now, but designing with that possiblity in mind does make sense.

 

I believe the impetus for ROS 2 comes mainly from a different, more industrial product development  community, who would like to use ROS but for various reasons cannot in its current form.

This is somewhat over-simplified and just my opinion (OSRF speak quite well for themselves). But, if I am correct, the initial ROS 2 target users are not the research labs who make up much of the existing ROS 1 community. 

The academics I know want stability so they can run their experiments and publish papers. They do not care much about low-level infrastructure, and begrudge every minute spent upgrading. They hated the catkin migration. It was time-consuming, and for them rosbuild actually worked better. 

Many product developers want new features, like DDS, and light-weight C language interfaces with few dependencies. They need their wire-level protocols fully documented, sometimes even verified and certified. They are less interested in ease of migration from ROS 1.

Why not just build separate products for the different user communities? That is generally easier to do. But, both the research and development communities stand to benefit from synergy between the two versions.

Whenever we separate academia and industry usage we do that based on the "common" use cases of those two domains.

Yes. I called it "somewhat over-simplified" without apology. Such simplifications may help clarify the discussion. That's good as long as we don't completely forget the underlying complexities. 

 

We should keep in mind that academy might very well have use cases that are not well supported in ROS 1 and therefore would like to use ROS 2.

The same goes for industry: for a few applications ROS 1 might be well suited.

In general I agree to your command that initially it is likely that "academia" will stay with ROS 1 for a while and industrial users might start considering ROS 2 if they avoided ROS 1 for good reasons.

But I don't this has to be the case for long.

Wherever "academy" (or any other ROS 1 user) has use cases which are not satisfied well enough in ROS 1 - to name a few:

* communication over unreliable / wireless network

* multi robot scenarios

* real time requirements

* integration with embedded systems

I am certain they will consider to use ROS 2 since it is supposed to fulfill these requirements much better compared to ROS 1.

Simply because it will get them to their goal faster and with less effort.

Good points, and worth keeping in mind. 

For the UTexas CS researchers I associate with: (1) wireless communication and (2) multi-robot systems are important; (3) real time is not; and (4) integration with embedded systems is not their focus (but might be interesting to other groups, like Electrical Engineering).

I would nominate multi-robot support over wireless as a crucial use case for ROS 2 networking. Unless we can create something significantly better and more reliable than the existing multi-master implementations, we need to improve the design or implementation or something. 

I believe the UTexas researchers would use ROS 2 if it can solve that problem. 

--

 joq

[Thibault Kruse]

On Sunday, July 19, 2015 at 6:30:47 PM UTC+2, Tully Foote wrote:

On Sun, Jul 19, 2015 at 3:40 AM, Thibault Kruse <tibo...@googlemail.com> wrote:

On Saturday, July 18, 2015 at 7:51:52 PM UTC+2, Jack O'Quin wrote:

On Sat, Jul 18, 2015 at 5:09 AM, Thibault Kruse <tibo...@googlemail.com> wrote:

On Friday, July 17, 2015 at 12:56:12 PM UTC+2, William Woodall wrote:

On Fri, Jul 17, 2015 at 1:49 AM, Thibault Kruse <tibo...@googlemail.com> wrote:

So the already tight maintainership situation in ROS will become even tighter, as maintainers will have to split their time, or support only one side of the trench.

As one of the major contributors and maintainers of ROS 1 (even before doing it full time) I'm keenly aware of this. I, and I'm certain many others, agree with the sentiment, but I'm not sure how else to reconcile this with the fact that we need to continue to make improvement to the core of ROS in order to meet the needs of more users. I think the approach we've taken will be slower and require more effort on our part and on the communities part, but it will also be the least risky and least disruptive to the ROS 1 ecosystem.

I am a bit tired of hearing this "non-disruptive of ROS1" argument. Basically ROS2 is the preparation for the mother of all disruptions to the ROS1 ecosystem. A slow, large-scale migration of thousands of packages to a different middleware. I believe it would be more honest if OSRF said that OSRF prefers disruption by migration over disruption by evolution, or something.

ROS 1 non-disruption is not an argument. It is a requirement that has been forcefully communicated to OSRF by many (probably most) of the current ROS 1 community.

I know, I am not tired of the community requesting this. I am tired of OSRF using that as an excuse for creating a new competing middleware.

There are multiple interpretations of what the community means when it says "We want less disruptions in the future":

A) Do not improve ROS1, create a new competing middleware to which we'll have to slowly and painfully migrate.
B) Put all future improvements into ROS1, but make the changes backwards compatible and well-tested.

I believe B reflects better than A what the community wants.

There is also C:
C) Create yet another middleware if you must, but continue to improve ROS1, and make changes to ROS1 backwards compatible and well-tested

But C obviously requires much more effort to everyone, and is not realistic. As as you can see with catkin and ament, it is very tempting for OSRF to just leave ROS1 behind, and make improvement only in the ROS2 world. You can also read it in all the OSRF answers saying: "Improving catkin could have caused regressions, so we did not improve catkin". With that argument, no further improvements will be made to ROS1. The same goes for all other packages maintained outside OSRF with the expectable low level of code sharing.


The alternative that the community could well have expected would be to make the communication layer exchangeable under a common client API (which can be changed/enhanced over time using tic-toc deprecation). That would mean all further improvements would have benefitted the community, but the community would not have to change much or migrate. This is the obvious benefit of maximum code sharing.

Apparently OSRF does not want to do that, and the consequence will be a huge disruption to the ROS community, as opposed to the wish expressed by the community.

Note that this is not fatal to anyone, and maybe in 10 years people will look back and praise OSRF for the courage and wisdom. I only explain that this is the opposite of what the community wants now.

That's what I mean when I say I am tired of the "non-disruptive of ROS1" argument.

You're right that all three of those paths are possible. And they have been considered. All of the options have tradeoffs. If we break the API we can make fundamental changes to the design. If we require full backwards compatibility and roll it out in place it limits how much change can be made, and also risks breakage of the current system as it changes underneath. And obviously we can do both or some combination of both it just takes more work.

I am saying just don't go telling people that this will not disrupt ROS1, or you chose it in order not to disrupt ROS1. Tell people you prepare a large-scale migration to a different middleware, for whatever reasons you mention, and that you understand this is a *large* disruption. You can still argue that it's worthwhile, and it's normal (and acceptable) that some people will disagree because they assess the consequences differently.

If the risk of breakage to ROS1 is too great, that only means you lack in automated test coverage. The solution to that can never be to duplicate code into a new major version, because that makes that problem obviously larger than smaller. Instead of having one under-tested ecosystem, you'll have two under-tested ecosystems. Adding more tests to ROS1 so that you can make improvements with higher confidence. I myself started added integration tests to catkin IIRC, but I believe they are not used.

Regarding "more work", there is a trade-off between the work to be done in-house at OSRF (creating change), and the work in the community (creating, adapting and improving thousands of packages). Creating something backwards compatible creates more work at OSRF, but not outside OSRF. And the time invested by the community into all kinds of contributions to all kinds of packages is valuable, too. The value of ROS depends on the quality and maintenance of all the other packages maintained outside OSRF. By creating a middleware schism with minimal code-share, not only is every contribution halved in value (because it only affects one of the two middlewares), the whole process discourages contributions. People contributing to ROS1 may very well ask themselves why bother if OSRF plans to drop ROS1 anyway (E.g. the work Jon Bohren did for catkin_tools cannot easily make it into ament). And ROS2 on the other hand is not even sure to be a success. So contributing to either may be a waste of time. If making changes in a backwards-compatible way, OSRF has more in-house work, but makes better usage of all the outside work in the community. So don't just say something is "more work" or "less work", when you obviously only consider the in-house work, not the community output as a whole.

You can argue that you have set some internal deadline, like ROSCon2015 for a technical demo, and making changes in a backwards-compatible way would have made that deadline impossible. Or make some similar claim about the necessity to make the rapid-prototyping really rapid (like OSRF funding running out, or people getting impatient).

Our decision was based on feedback from users and potential users. We specifically reached out to ROS users as well as people and companies we know have chosen not to use ROS so we could identify the use cases not supported by the current version of ROS.

That's all good. It has nothing to do with not disrupting ROS1. It might be a good reason to disrupt ROS1 in a way it has never been disrupted before, sure.
 

Our goal is to use the lessons learned from the many years of use of ROS to create a replacement which will have compelling enough features that people will want to migrate their software forward. If they do not find the new features compelling they can keep using the current system. If we develop a system that we cannot convince people to use that will be a major failure and we can switch our development back to the ROS 1.0 branch.

Yes, we can look at both Python3 and Perl6 to see those two different possible future scenarios (the jury of course still being out for Perl6). Neither Python3 nor Perl6 can be described as being non-disruptive to their communities.

So I still maintain that OSRF decided for a large disruption (for maybe good reasons) against the explicit wish of the community, and that OSRF made that decision without involving the community in the decision-making (maybe also for good reasons).

[Thibault Kruse]

On Sunday, July 19, 2015 at 9:08:46 PM UTC+2, Dirk Thomas wrote:

On Sun, Jul 19, 2015 at 3:40 AM, Thibault Kruse <tibo...@googlemail.com> wrote:

[...]

I think you are contradict yourself here.

On the one hand you argue for having future changes in ROS 1 but in a backward compatible way.

But on the other hand you suggest to use a tic-toc cycle to introduce changes.

Tic-toc deprecation is a way to disrupt less, it's a compromise between improving and backwards compatibilty. I argue for staying backwards compatible where possible, with tic-toc deprecation being a viable alternative (with a voted REP) when compatibility is not possible. I believe the community never asked for no tiny change ever to be made again. I believe the phrasing was "Less disruption", not "Zero disruption".

[Bill Morris]

So, I am willing to give up being able to install ROS1 and ROS2 at the
same time if we can keep the message packages the same.

I propose that for every ROS1 message package we create a ROS2 package
that can be used for bridging and message generation. These packages
would be exact copies of the ROS1 packages except they would be have a
new name.

Ex. An exact copy of the ROS1 'camera_msgs' would be available in ROS2
as 'camera_msgs1' and the equivalent ROS2 message package could then be
named 'camera_msgs'.

For message packages with significant changes, a mapping package
('camera_msgs_map') could provide the necessary information to translate
messages.

[William Woodall]

Thanks for the suggestion. I think this is really similar to Dirk's suggestion in that it keeps the same package names for messages in ROS 2 by doing some ROS 1 message generation in ROS 2 in order to make the bridge work, but his proposal doesn't require an extra package in ROS 2 since it uses the ROS 1 message packages directly but through pkg-config and crawling for .msg files.

Does that sound right? Can we just close this one out and focus on that proposal? Do you see something else to pursue here? I don't want to ignore this thread if there's more to discuss.

--
You received this message because you are subscribed to the Google Groups "ROS SIG NG ROS" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ros-sig-ng-ro...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

William Woodall

ROS Development Team

wil...@osrfoundation.org

http://wjwwood.io/

[Bill Morris]

I think the pkg-config option from the "Alternative proposal for the ROS
bridge" looks like a reasonable way forward.
I still have some concerns about converting ros1 bags to ros2 bags, but
I think that is a problem for another day.
Let's close this thread and focus on the other one.

Thanks,
Bill