Building + Package management

744 views
Skip to first unread message

Andrew Hundt

unread,
Apr 15, 2016, 10:38:58 PM4/15/16
to ROS SIG NG ROS
I've been looking into ros2 a little bit, and I was curious about a couple of things regarding building and managing packages. 

First a brief aside, I'm very pleased with the decision to use an existing middleware solution. While my personal experience with OpenSplice has overall been negative due to the complexity of setup, bugs encountered, and lack of documentation, it may have improved since I was using it intimately in ~2011-2014. Plus, since it is open source it can be improved. Despite these experiences, it is great that ros2 isn't being started from scratch! 

catkin/amant/package managment

Much like the middleware, couldn't package management be handled by existing tools made elsewhere in the same manner as the choice of DDS? If not, why not?

From my perspective, choosing an existing tool could substantially reduce the need for such work during ros2 development and into the future. It would also prevent platform specific gotchas that come with a smaller user base. Plus, this space is much different today when compared to how it was 2009.

Existing full stack build tools

For example, there are cross platform and language build tools already available that scale to massive size:

Combining existing federated build and package tools

Alternately, adopting an existing package manager such as homebrew+linuxbrew or 0install.net for applications in combination with existing package managers that work well with their own language if needed. Particularly with the recent release of Ubuntu for Windows, *brew could be a powerful tool!

  • tight focus on making it easy to create installers for your own package
  • possible to provide full top to bottom versioning of packages and dependencies (avoid cross-os differences!)
  • federated build/install system
  • developer tools
  • "bottles" which are prepackaged binaries
  • automatic dependency installation
  • flags to customize the build configuration
  • continuous integration support
  • integration with many many languages (i.e. installing and setting them up)
  • OSRF already uses homebrew on OS X!

Of course this isn't the whole solution because the build tools matter. Fortunately, most languages have good existing options ros2 could integrate with and provide corresponding examples of how to use it with ros2. Here are some possible options for various languages:

C++
Java
  • maven
  • gradle
  • ant
Python - pip/pip3
Ruby - RubyGems
lua - luarocks

Conclusion

Ultimately, couldn't either one of the full stack build tools, or a federated setup (perhaps modifying existing tools when/if necessary) be sufficient compared to a ros specific toolset? 

I've at least read:

Thanks for your consideration!


William Woodall

unread,
Apr 15, 2016, 11:39:31 PM4/15/16
to ROS SIG NG ROS
On Fri, Apr 15, 2016 at 7:38 PM, Andrew Hundt <ath...@gmail.com> wrote:
I've been looking into ros2 a little bit, and I was curious about a couple of things regarding building and managing packages. 

First a brief aside, I'm very pleased with the decision to use an existing middleware solution. While my personal experience with OpenSplice has overall been negative due to the complexity of setup, bugs encountered, and lack of documentation, it may have improved since I was using it intimately in ~2011-2014. Plus, since it is open source it can be improved. Despite these experiences, it is great that ros2 isn't being started from scratch! 

catkin/amant/package managment

Much like the middleware, couldn't package management be handled by existing tools made elsewhere in the same manner as the choice of DDS? If not, why not?

That's a fair question. I'll do my best to express why I think there's not a good existing solution for us to leverage in this part of the system (the build tools).
 

From my perspective, choosing an existing tool could substantially reduce the need for such work during ros2 development and into the future. It would also prevent platform specific gotchas that come with a smaller user base. Plus, this space is much different today when compared to how it was 2009.

We didn't spend as much time researching alternatives for build systems as we did for middlewares, but we've been working on them for quite a while and we try to keep up to date on the latest trends. Given that, we didn't feel that there was a good replacement that's come around since we developed rosbuild and catkin way back when.

As for https://github.com/catkin/catkin_tools/issues/266, that issue isn't so much a platform issue as a combination of limitations from our tools, to cmake's generator for Xcode, to Xcode itself. Skipping ahead a bit, the new-ish alternatives to something like ament (buck, bazel, or pants) don't support working with Xcode like this either. For example, from http://bazel.io/faq.html:

> There is no support to invoke Bazel from Xcode (for example to re-generate generated sources such as Objective-C files based on protos), nor to open Xcode from Bazel directly.

Basically you have to run bazel, and then open xcode on the result. Which is more or less what you're able to do with a cmake package, except cmake can be reinvoked from within Xcode (same applies for QtCreator and Visual Studio).
 

Existing full stack build tools

For example, there are cross platform and language build tools already available that scale to massive size:
I've experimented with bazel more than the others, but all of these, in my opinion, don't address the federated model we have in ROS completely. They tend to be designed to accommodate a centralized, large repository with modular internal structure, and require everything to be written in or wrapped up in bazel (or the tool of choice). By comparison we're moving towards a set of build, install, and test conventions in ament, such that you don't even need cmake specifically. You should be able to use pure Python setuptools or bazel or whatever you want so long as it meets a few small requirements like being able to install to FHS layout and possibly others (I can't enumerate them all off-hand).

The other thing that is a deal breaker for me is a lack of support for Windows. Again, I only really tried out bazel, but looking briefly are the others, they also don't seem to support Windows. And as much as I hate Windows development sometimes (ok most of the time), it's something that has been requested of us again and again in ROS, and I think it's important that at least the core works and in order for that to be the case we need a build system that works on Windows too.
 
 
Combining existing federated build and package tools

Alternately, adopting an existing package manager such as homebrew+linuxbrew or 0install.net for applications in combination with existing package managers that work well with their own language if needed. Particularly with the recent release of Ubuntu for Windows, *brew could be a powerful tool!

I love Homebrew, and I think it's a really nice tool, but I think they lean towards bleeding edge developers. It lacks some of the rigor of other distribution systems like apt-get, especially in terms of controlling versions of packages you can install. It does a nice job of keeping old versions installed side-by-side, but shy's away from complex versioned dependencies (which I think is understandable given their users and how the tool is used typically).

I've tried to use Linuxbrew with limited success. For me, Homebrew + Linuxbrew doesn't cover enough platforms for us (notably lacking Windows support). As you mention, in some future case we might be able to use these on Windows, but having tried out their bash for Windows demos, my opinion is that it will be years before that's something we can rely on and will most likely have some serious limitations. Also we want people to be able to use our stuff within Visual Studio, which seems unlikely to be an option with bash on Windows. Overall, I don't see how Homebrew or Linuxbrew could replace our build tools, since they don't really address the developer user who builds lots of packages at the same time. I think it would be less convenient to use, for example, Homebew and plain CMake rather than our tools if you're hacking on more than one package at a time.

I also looked at 0install.net, but only briefly. Only recently the Windows support has started to catch up (looks like major improvements since the new year). I'd be open to using this as a distribution mechanism, but again I don't think it replaces the need for tools when building multiple packages from source at the same time.
 


  • tight focus on making it easy to create installers for your own package
  • possible to provide full top to bottom versioning of packages and dependencies (avoid cross-os differences!)
  • federated build/install system
  • developer tools
  • "bottles" which are prepackaged binaries
  • automatic dependency installation
  • flags to customize the build configuration
  • continuous integration support
  • integration with many many languages (i.e. installing and setting them up)
  • OSRF already uses homebrew on OS X!
 
Of course this isn't the whole solution because the build tools matter. Fortunately, most languages have good existing options ros2 could integrate with and provide corresponding examples of how to use it with ros2. Here are some possible options for various languages:

C++
I'm aware of all of these :)

Again, ament is more about automating the builds of multiple packages. We support plain cmake packages, for example we build FastRTPS and FastCDR without modifications in our build process. If you look at ament_cmake, it is basically just convenience macros, which you can choose to use or not, just like any other CMake "library". All the "ament cmake" packages can be built as a normal cmake package ("mkdir build; cd build; cmake ..; make"). We intend to rely heavily on newer CMake features. In fact our own Dirk Thomas and Jose Luis Rivero were some of the main actors involved in getting something better than cmake 3.2 into Xenial after the code freeze:


It involved us spinning up builds of several hundred ubuntu packages on our build.ros.org farm. So we're keen to capitalize on that effort.
 
Java
  • maven
  • gradle
  • ant
These could all be supported by ament I think.
 
Python - pip/pip3

We have many pure Python packages in ROS 2, being built with ament, that only use setuptools. Of course these packages could be released into pip. We typically do not just use pip though since apt-get packages cannot depend on pip packages.
 
Ruby - RubyGems
lua - luarocks

Same story as for Python and pip.
 

Conclusion

Ultimately, couldn't either one of the full stack build tools, or a federated setup (perhaps modifying existing tools when/if necessary) be sufficient compared to a ros specific toolset? 

I'd love to stop working on build tools and build systems and build conventions and just throw away our custom stuff and use something already out there. However, I neither see a system that already meets our goals nor a system that is willing to change their direction to meet our goals.

My conclusion is that none of them meet our goals (whether it be Windows support or allowing for integration with other build systems or supporting portable federated deployment). Also consider that many of our existing packages are written in CMake and many of our users are used to using it. So diverting away from CMake has some significant cognitive cost for our community. For those using catkin, we hope to continue to support it so that they can transition to ament when/if they think it appropriate (Brian and Morgan have been working on this recently, hopefully they'll have something to share about it soon). But I'd argue that the transition from catkin to ament would be much easier than a transition to a completely new system like SCons or bazel.

I'd also argue that ament is basically "a federated setup (perhaps modifying existing tools when/if necessary)" as you put it. We use and support plain CMake and Python's setuptools without modification or even a package.xml file. We could support pretty much anything that provides an "install to FHS layout" target. As for the ament_cmake stuff, it is really no different than any other CMake code that's embedded in pretty much every open source cmake project out there (have a look at opencv, pcl, gazebo, ogre, and similar projects). All of them have non-trivial cmake code to accomplish some common tasks. Ours just has a name, is modularized out of our other code so it's reusable, and could be used by others if they like it.

Just like with the on going work for ROS 1, we fully expect people to build software how ever they like and just use ROS 2 as a dependency. We still have to build our software somehow and we've settled on this for various reasons.

If you can point to a part of the system and ask "why didn't you use X instead?", then I think we can have a more productive discussion. Just pointing out that there are other tools out there isn't something I can weigh against what we're currently doing. I believe we've come to these decisions after well informed consideration, but that doesn't mean there aren't places we could throw away something custom and reuse an existing tool. I just haven't seen any concrete suggestions from our community on that point, but I'd be excited to get some suggestions.
I know our justifications exist largely in our (the ROS 2 team's) heads, and that sucks, but thankfully discussions like this can force us to put them into words. So thanks for getting the ball rolling on that point. I look forward to more nuanced discussions and proposals as we converge on a common understanding of the current state of things.
 


--
You received this message because you are subscribed to the Google Groups "ROS SIG NG ROS" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ros-sig-ng-ro...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.



--
William Woodall
ROS Development Team

Austin Hendrix

unread,
Apr 15, 2016, 11:51:14 PM4/15/16
to ros-sig...@googlegroups.com
The current build/package system that ROS uses provides:
* Dependencies on system packages, automated installation of those
packages, across different platforms (various flavors of linux, OSX,
windows)
* Dependency declaration and resolution between ROS packages
* Generation of distro-specific binary packages (deb, rpm, arch)
* Cross-platform support (linux, OSX, windows)

And OSRF kindly runs a build farm which takes public, source
repositories and produces binaries for some popular platforms (made
possible by close integration with the build system).

linuxbrew and 0install both appear to meet these basic requirements.
linuxbrew doesn't appear to support windows but otherwise sounds
interesting, and 0install looks interesting but a bit labor-intensive
(maybe that can be automated?).

I've been using bazel for a large project for a while, and I've found
that it doesn't meet most of these requirements - it doesn't
automatically resolve dependencies on the host system, and it doesn't
provide good tools for generating distro-specific packages.

I've also found that bazel is generally immature, and it's much harder
to debug issues with bazel than with cmake or make. To be fair, it's a
fairly new project and they're only on version 0.2.1.

I haven't used pants or buck, so I can't speak to their maturity or
suitability, but I think they're targeted at similar use case to bazel,
and probably have similar drawbacks. If anyone has used these, I would
be interested to hear about your experience.

I would not expect too much of the Ubuntu on Windows 10 support. It
looks like they'll have good support for the ELF binary format and build
tools, but it's not clear if they'll have X support (for rviz, rqt) or
not, and it doesn't sound like they will have hardware support.

-Austin

On 04/15/2016 07:38 PM, Andrew Hundt wrote:
> I've been looking into ros2 a little bit, and I was curious about a
> couple of things regarding building and managing packages.
>
> First a brief aside, I'm very pleased with the decision to use an
> existing middleware solution. While my personal experience with
> OpenSplice has overall been negative due to the complexity of setup,
> bugs encountered, and lack of documentation, it may have improved
> since I was using it intimately in ~2011-2014. Plus, since it is open
> source it can be improved. Despite these experiences, it is great that
> ros2 isn't being started from scratch!
>
> *catkin/amant/package managment*
>
> Much like the middleware, couldn't package management be handled by
> existing tools made elsewhere in the same manner as the choice of DDS?
> If not, why not?
>
> From my perspective, choosing an existing tool could substantially
> reduce the need for such work during ros2 development and into the
> future. It would also prevent platform specific gotchas
> <https://github.com/catkin/catkin_tools/issues/266> that come with a
> smaller user base. Plus, this space is much different today when
> compared to how it was 2009.
>
> *Existing full stack build tools*
>
> For example, there are cross platform and language build tools already
> available that scale to massive size:
>
> * buck build <https://github.com/facebook/buck> (facebook) c++,
> java, D, python, rust, lua (many more)...
> * bazel build <http://bazel.io/faq.html> (google) c++/java primarily
> * pants build <https://pantsbuild.github.io/> (twitter/foursquare)
>
>
> *Combining existing federated build and package tools*
>
> Alternately, adopting an existing package manager such as homebrew
> <http://brew.sh/>+linuxbrew <http://linuxbrew.sh/> or 0install.net
> <http://0install.net> for applications in combination with existing
> package managers that work well with their own language if
> needed. Particularly with the recent release of Ubuntu for Windows,
> *brew could be a powerful tool!
>
> homebrew <http://brew.sh/>+linuxbrew <http://linuxbrew.sh/> support:
>
> * tight focus on making it easy to create installers for your own
> package
> * possible to provide full top to bottom versioning of packages and
> dependencies (avoid cross-os differences!)
> * federated build/install system
> * developer tools
> * "bottles" which are prepackaged binaries
> * automatic dependency installation
> * flags to customize the build configuration
> * continuous integration support
> * integration with many many languages (i.e. installing and setting
> them up)
> * OSRF already uses homebrew on OS X!
>
>
> Of course this isn't the whole solution because the build tools
> matter. Fortunately, most languages have good existing options ros2
> could integrate with and provide corresponding examples of how to use
> it with ros2. Here are some possible options for various languages:
>
> C++
>
> * Standalone CMake (which has improved a lot now around 3.5 with
> namespaces and separate build/install configs,
> cpack,packageconfighelpers
> <https://cmake.org/cmake/help/v3.5/module/CMakePackageConfigHelpers.html>
> etc)
> * CMake + an existing convenience tool such as:
> o https://github.com/conan-io/conan (successor to biicode)
> o https://github.com/ruslo/hunter
> o https://github.com/iauns/cpm
> o https://github.com/schuhschuh/cmake-basis (I worked on this in
> 2014)
>
> Java
>
> * maven
> * gradle
> * ant
>
> Python - pip/pip3
> Ruby - RubyGems
> lua - luarocks
>
> *Conclusion*
>
> Ultimately, couldn't either one of the full stack build tools, or a
> federated setup (perhaps modifying existing tools when/if necessary)
> be sufficient compared to a ros specific toolset?
>
> I've at least read:
> - http://design.ros2.org/articles/ament.html
> -
> https://groups.google.com/forum/#!searchin/ros-sig-ng-ros/package/ros-sig-ng-ros/suTQfcddeh8/p3d90Ew8-ZkJ
>
> Thanks for your consideration!
>
>
> --
> You received this message because you are subscribed to the Google
> Groups "ROS SIG NG ROS" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to ros-sig-ng-ro...@googlegroups.com
> <mailto:ros-sig-ng-ro...@googlegroups.com>.

Jack O'Quin

unread,
Apr 16, 2016, 12:09:24 PM4/16/16
to ros-sig...@googlegroups.com
On Fri, Apr 15, 2016 at 10:39 PM, William Woodall <wil...@osrfoundation.org> wrote:

I know our justifications exist largely in our (the ROS 2 team's) heads, and that sucks, but thankfully discussions like this can force us to put them into words. So thanks for getting the ball rolling on that point. I look forward to more nuanced discussions and proposals as we converge on a common understanding of the current state of things.

+1 for Will's excellent summary of the rationale for creating and maintaining a ROS build system. His message all by itself is a good step towards a solid Rationale section for the ament design document.

My initial impression of ROS back in 2009 was colored by the natural skepticism of any experienced open-source programmer towards any project that creates its own build system. (Qmake was a well-known and annoying example at the time.) It was only after several months using ROS that I came to understand how much the build system enables our federated development goals.

So, a full rationale statement such as this remains quite important.
--
 joq

Andrew Hundt

unread,
Apr 16, 2016, 1:39:56 PM4/16/16
to ROS SIG NG ROS


Existing full stack build tools

For example, there are cross platform and language build tools already available that scale to massive size:
I've experimented with bazel more than the others, but all of these, in my opinion, don't address the federated model we have in ROS completely. They tend to be designed to accommodate a centralized, large repository with modular internal structure, and require everything to be written in or wrapped up in bazel (or the tool of choice). By comparison we're moving towards a set of build, install, and test conventions in ament, such that you don't even need cmake specifically.
I'll be honest, I haven't used these myself I've only read about them. Buck seems the closest but I can definitely see why they may not work as-is. However, hearing how important the federated model is really made a light bulb go off and it isn't any of these options. I'll explain below.


On Saturday, April 16, 2016 at 12:09:24 PM UTC-4, Jack O'Quin wrote:
On Fri, Apr 15, 2016 at 10:39 PM, William Woodall <wil...@osrfoundation.org> wrote:

I know our justifications exist largely in our (the ROS 2 team's) heads, and that sucks, but thankfully discussions like this can force us to put them into words. So thanks for getting the ball rolling on that point. I look forward to more nuanced discussions and proposals as we converge on a common understanding of the current state of things.

+1 for Will's excellent summary of the rationale for creating and maintaining a ROS build system. His message all by itself is a good step towards a solid Rationale section for the ament design document.,

My initial impression of ROS back in 2009 was colored by the natural skepticism of any experienced open-source programmer towards any project that creates its own build system. (Qmake was a well-known and annoying example at the time.) It was only after several months using ROS that I came to understand how much the build system enables our federated development goals.

This discussion and the one above made some of the goals clearer to me and something specific really stood out in my mind. Specifically:

Homebrew is the ros topics of build systems!

Why not go with something that matches one of your own best/proven models/designs in another area it can apply very well? Specifically, packaging! Please allow me to elaborate and address a couple of the concerns.

On Friday, April 15, 2016 at 11:39:31 PM UTC-4, William Woodall wrote:



I love Homebrew, and I think it's a really nice tool, but I think they lean towards bleeding edge developers. It lacks some of the rigor of other distribution systems like apt-get, especially in terms of controlling versions of packages you can install. It does a nice job of keeping old versions installed side-by-side, but shy's away from complex versioned dependencies (which I think is understandable given their users and how the tool is used typically).

You are right that Homebrew leans towards bleeding edge developers in the default packages they provide, however, their tool is specifically designed with the rigor you describe, they just happened to deliberately choose the bleeding edge rolling release model in their primary repository, but they make it trivial to set up a stable version locked repository. 

That is the one crucial detail that is critical to ROS, it is trivial to change the packages you're building against and set up your own package set. Everything revolves around the git repo, taps and recipes, and it is super easy to create your own. This will get pretty detailed because it is important to understand their model. 

Specifically, Recipes (ruby install scripts) are pinned to exact tarball hashes, binaries, github tags or other files at the user's option all precisely defined in a git repository. Think of it as ros topics for packaging. As evidence, let's start with a "recipe" by taking a look at the start of the opencv install ruby script:




class Opencv < Formula desc "Open source computer vision library"
homepage "http://opencv.org/"
url "https://github.com/Itseez/opencv/archive/2.4.12.tar.gz"
sha256 "8989f946a66fa3fc2764d637b1c866caf28d074ece187f86baba66544054eefc"
head "https://github.com/Itseez/opencv.git", :branch => "2.4"
revision 2
bottle do
sha256 "3e584f97f377b8ac0f7e55efa494bdb0fb108930212f5fb8a03cecc18f9b1d43" => :el_capitan
sha256 "c60dc79ecf71499e91945726f4e0fd3f7abaf2dac92e4a8d081c4be48d39dcac" => :yosemite
sha256 "f4e3fdf22515e9847f70fcd71ef96eef06e68077f054e0178417fe073594fd83" => :mavericks
end
option "32-bit"
option "with-java", "Build with Java support"
option "with-qt", "Build the Qt4 backend to HighGUI"
option "with-tbb", "Enable parallel code in OpenCV using Intel TBB"
option "without-tests", "Build without accuracy & performance tests"
option "without-opencl", "Disable GPU code in OpenCV using OpenCL"
option "with-cuda", "Build with CUDA support"
option "with-quicktime", "Use QuickTime for Video I/O instead of QTKit"
option "with-opengl", "Build with OpenGL support"
option "with-ximea", "Build with XIMEA support"
option "without-numpy", "Use a numpy you've installed yourself instead of a Homebrew-packaged numpy"
option "without-python", "Build without Python support"

 

 As you can see, this script excerpt:

- has a list of specific dependencies, hashed to an exact version, tarball, etc.
- provides transparent download of binaries for specific operating systems when the default package works for you.
- provides extremely easy to use configuration options if you need something other than the default

It would be very easy for ros to provide files like this in branches for long term support of specific versions in release distributions of ROS. Even better it would be easy for users to modify when they need to patch due to bugs in the release software, which is something I encounter on a daily basis. Furthermore, these scripts are incredibly easy to understand, configure, and they run not only cmake but dozens of other build tools as well. 

Now, let's say you aren't happy with the version provided there and you want to use your own. This is where "taps" come in. You can either fork the whole repository and modify one or two lines to change the version, or you can create your own ruby script that binds to a specific version. 

For example, switching to your specific fully versioned clone is as easy as:

brew uninstall opencv
brew tap ahundt/science
brew install ahundt/science/opencv

where ahundt/science is at https://github.com/ahundt/homebrew-science
versus the homebrew/science repo is https://github.com/Homebrew/homebrew-science

Or, if you want to just install a single specific file without any other changes:


Using this system, ROS could provide a full top to bottom package stack that is known to be stable and make it very consistent across many OSes!

It would also support the current release cadence versioning, and long term support ROS desires extremely well.

For pure binary installs and applications there is another system of "casks" as well, though I won't go into that quite as much, here is the link:

That said, there are a few specific outstanding functional components that would need to be fixed so it would work for ROS. I'm sure this isn't a complete enumeration but it hits some of the big points.

Specifically the following would need resolution: 
However, these are all issues which are all very practical to resolve with some development, although the windows issue probably outclasses the others in scale. Even so, I believe none of these shortcomings is insurmountable, some are trivial, and some look like they'll actively be addressed.

Why not adapt homebrew to work for ROS and take advantage of the dozen developers already working on homebrew and 5000+ contributors, incorporating the experiences of all the ros build systems plus the experience of thousands of other developers?

It would be quite easy to set up a homebrew-ros with branches and exactly the versions of the exact software that is desired to get started! It also would help that many of you are already familiar with homebrew when compared to any other pre-existing tool not currently used by ROS.

Thanks again for taking my questions and ideas into consideration!

Thibault Kruse

unread,
Apr 16, 2016, 1:56:47 PM4/16/16
to ROS SIG NG ROS
On Saturday, April 16, 2016 at 4:38:58 AM UTC+2, Andrew Hundt wrote:
Much like the middleware, couldn't package management be handled by existing tools made elsewhere in the same manner as the choice of DDS? If not, why not?

Back in 2012, I briefly tried motivating using the gnome buildsystem, jhbuild:
https://groups.google.com/forum/#!topic/ros-sig-buildsystem/xCYO2EOgd0M

I believed that given the gnome ecosystem has certain similarities to the ROS ecosystem, a joint effort in buildsystems might benefit both.

 
For example, there are cross platform and language build tools already available that scale to massive size:
Like Austin, i regard those as still too experimental for public use (all 3). So far their user-facing bits have not been optimized for the general public (steep learning curve, hard to debug, easy to shoot yourself in the foot).

Ultimately, couldn't either one of the full stack build tools, or a federated setup (perhaps modifying existing tools when/if necessary) be sufficient compared to a ros specific toolset? 
 
 
To understand a bit more about the history and culture involved, you can read up on the discussions during the introduction of catkin: http://wiki.ros.org/catkin/Reviews/2012-08-01_API_Review
It's a long read, and mostly ancient history now, but you can get a glimpse at the requirements that were considered.

Steven Peters

unread,
Apr 16, 2016, 2:48:57 PM4/16/16
to ROS SIG NG ROS
It is indeed very easy to set up your own homebrew tap to host your own custom packages. It's harder, though, to put version constraints on dependencies (for example requiring an older version of boost). The current state-of-the-art is to create a duplicate formula (such as boost155) typically hosted in homebrew/duplicates. I don't consider this a very elegant approach.

It is true that they allow customizable builds (such as --double-precision for bullet), and you can declare these options in your dependencies (gazebo requests that double-precision option from bullet). Using that example, if you install gazebo and bullet is not installed, it will install bullet with the requested options. If bullet is already installed, however, it won't check whether it has the desired configuration, it will just use it.

Don't get me wrong, I love homebrew very much. I just think it's not trying to do everything that debian does, so it's important to keep that in mind.

Steve
...

Andrew Hundt

unread,
Apr 17, 2016, 7:06:29 PM4/17/16
to ROS SIG NG ROS


On Saturday, April 16, 2016 at 2:48:57 PM UTC-4, Steven Peters wrote:
It is indeed very easy to set up your own homebrew tap to host your own custom packages. It's harder, though, to put version constraints on dependencies (for example requiring an older version of boost). The current state-of-the-art is to create a duplicate formula (such as boost155) typically hosted in homebrew/duplicates. I don't consider this a very elegant approach.

It is true that they allow customizable builds (such as --double-precision for bullet), and you can declare these options in your dependencies (gazebo requests that double-precision option from bullet). Using that example, if you install gazebo and bullet is not installed, it will install bullet with the requested options. If bullet is already installed, however, it won't check whether it has the desired configuration, it will just use it.

Don't get me wrong, I love homebrew very much. I just think it's not trying to do everything that debian does, so it's important to keep that in mind.

Steve
 
I suggest sending them feedback while the getting's hot!


I've started the ball rolling and the homebrew people are at least thinking about some of these issues, and they are planning to address at least some aspects. 


You have some good points there. I'm not intimate with the capabilities of debian packaging. Are there any quality resources you can recommend? I usually end up compiling most things from source anyway due to bugs I need to fix that can't wait for a future release or functionality I'd rather not reimplement that is available beyond the release versions. I'm not sure how, but is there any way .debs can even help with that? Is there a streamlined way for federated binaries in what currently exists?

Additionally, deb files don't help me much on OS X. :-) It would be amazing to have one system that can work on all platforms. How much would it take to add version parameters/constraints to homebrew? I don't think that would be overly complex.
 

Andrew Hundt

unread,
Apr 17, 2016, 7:51:07 PM4/17/16
to ROS SIG NG ROS
It seems someone is trying to cross-compile ros2 which is another use case very relevant to this discussion:

Andrew Hundt

unread,
May 3, 2016, 7:48:32 PM5/3/16
to ROS SIG NG ROS
This seems to have died out while I've been traveling. Any chance you might have a moment for my questions from April 17 below?

I've also thought of another problem I've encountered. As an LTS gets older some of the tools I use must be updated, which means compiling and patching from source, sometimes supplying later versions of items in /usr/lib in /usr/local/lib, sometimes for bugfixes there isn't another way to avoid. Can Ament do anything to make life easier in this situation?

Brew, upon discussion seems to have not found an easier to use (their top priority) way of dealing with versions other than files like github.com/username/homebrew-tap/boost155.rb instead of github.com/username/homebrew-tap/boost.rb. The one thing they may add is explicit versioned file support and aliasing for the latest version. 

Brew does seem to be looking into linux support for 1.0. A nice plus will be support for distros that don't have debian... maybe it will eventually prove easy enough to manually port the OS X ros install scripts back to linux, then I can at least utilize the package version/bugfix flexibility and easy to use scripts on linux too. Perhaps it really is just as easy to do for non-experts on linux/ubuntu as well and I'm just not familiar with the process?

Steven Peters

unread,
May 3, 2016, 8:12:17 PM5/3/16
to ros-sig...@googlegroups.com
On Tue, May 3, 2016 at 4:48 PM, Andrew Hundt <ath...@gmail.com> wrote:
This seems to have died out while I've been traveling. Any chance you might have a moment for my questions from April 17 below?

I've also thought of another problem I've encountered. As an LTS gets older some of the tools I use must be updated, which means compiling and patching from source, sometimes supplying later versions of items in /usr/lib in /usr/local/lib, sometimes for bugfixes there isn't another way to avoid. Can Ament do anything to make life easier in this situation?

Brew, upon disc. So I don't think it willussion seems to have not found an easier to use (their top priority) way of dealing with versions other than files like github.com/username/homebrew-tap/boost155.rb instead of github.com/username/homebrew-tap/boost.rb. The one thing they may add is explicit versioned file support and aliasing for the latest version. 

Brew does seem to be looking into linux support for 1.0. A nice plus will be support for distros that don't have debian... maybe it will eventually prove easy enough to manually port the OS X ros install scripts back to linux, then I can at least utilize the package version/bugfix flexibility and easy to use scripts on linux too. Perhaps it really is just as easy to do for non-experts on linux/ubuntu as well and I'm just not familiar with the process?


tldr: homebrew isn't made for stable software releases like LTS, so it's not a direct replacement for debian.

I participated in that homebrew discussion, and my takeaway is that homebrew is probably closest to debian unstable since it is always providing the most recent versions of software. Homebrew doesn't currently have an LTS, which requires freezing software versions for a long time and managing version-dependent dependencies. I asked in that thread about improving support for version-dependent dependencies, but they basically said it's up to the folks doing packaging to deal with that and they didn't want to add complexity to Homebrew to simplify this task.

Andrew Hundt

unread,
May 4, 2016, 12:27:51 AM5/4/16
to ROS SIG NG ROS
Steven, thank you for your reply. I'm running into some confusion between this reply and how things work with ROS. Could you clarify? Hopefully the following questions illustrate what I'm missing:

Does Ubuntu currently manage all your debian packaging of ROS releases for you from top to bottom? It is true the homebrew guys won't manage a ROS release for you, but could you describe how that's different from what you do already? 

Could ROS resolve this by cloning its own "LTS" branch of the necessary taps (or their own super tap) into a ROS managed repo for each release and freeze things that way?

Additionally, how does the stability of debian versions help ROS releases on other platforms where ROS is used but debian is not? 

Thanks, I'm just trying to understand the perspective from which your post was written!

Tully Foote

unread,
May 4, 2016, 6:16:05 PM5/4/16
to ros-sig...@googlegroups.com
These things are doable, but the main thing is is that we don't want to be maintaining the core OS and all it's dependencies in addition to the ROS distribution. We take advantage of the great work of the maintainers in the Debian and Ubuntu communities and use their releases as stable bases upon which to build our ROS packages. Leveraging that, on any given day we always expect ROS to install on our supported platforms. 

Comparatively OSX support based on homebrew is only known to work at a given point in time when tested. However after that time when upstream moves a major dependency we don't know when or if it will break our software. We could fork the entire system however to do so would incur the cost of maintaining every dependency. We regularly get reports of software previously tested as working is no longer working due to changed dependencies. 

We are working hard to keep the dependency tree smaller for ROS2 as can be seen by our small mostly self contained binary installations for our Alphas. However as a generic deployment mechanism when we start building up the community based stack I think we want to stick to the native packages managers on most platforms. 


Thanks, I'm just trying to understand the perspective from which your post was written!

--

Jack O'Quin

unread,
May 5, 2016, 12:13:22 PM5/5/16
to ros-sig...@googlegroups.com
At ROSCon last October, Mark Shuttleworth proposed "snap" as a secure, cross-platform packaging designed for the Internet of Things. I notice that Ubuntu 16.04 Xenial supports it. 

Does anyone here have experience with snap they'd like to share?

Is it a potential packaging option for ROS 2?

Andrew Hundt

unread,
May 9, 2016, 5:37:15 PM5/9/16
to ROS SIG NG ROS


On Wednesday, May 4, 2016 at 6:16:05 PM UTC-4, Tully Foote wrote:

Steven, thank you for your reply. I'm running into some confusion between this reply and how things work with ROS. Could you clarify? Hopefully the following questions illustrate what I'm missing:

Does Ubuntu currently manage all your debian packaging of ROS releases for you from top to bottom? It is true the homebrew guys won't manage a ROS release for you, but could you describe how that's different from what you do already? 

Could ROS resolve this by cloning its own "LTS" branch of the necessary taps (or their own super tap) into a ROS managed repo for each release and freeze things that way?

Additionally, how does the stability of debian versions help ROS releases on other platforms where ROS is used but debian is not? 

These things are doable, but the main thing is is that we don't want to be maintaining the core OS and all it's dependencies in addition to the ROS distribution. We take advantage of the great work of the maintainers in the Debian and Ubuntu communities and use their releases as stable bases upon which to build our ROS packages. Leveraging that, on any given day we always expect ROS to install on our supported platforms. 

Comparatively OSX support based on homebrew is only known to work at a given point in time when tested. However after that time when upstream moves a major dependency we don't know when or if it will break our software. We could fork the entire system however to do so would incur the cost of maintaining every dependency. We regularly get reports of software previously tested as working is no longer working due to changed dependencies. 

We are working hard to keep the dependency tree smaller for ROS2 as can be seen by our small mostly self contained binary installations for our Alphas. However as a generic deployment mechanism when we start building up the community based stack I think we want to stick to the native packages managers on most platforms. 


I've had a little more time to mull this over, and I think I have a narrower but very common situation worth discussing and perhaps streamlining:
  1. I add code that uses some functionality in a widely used ROS dependency like (pcl or OpenCV, for example) that I haven't used before.
  2. It turns out there is a bug in said call, I submit a patch or bug report on github
  3. (2) may or may not be added upstream right away so I create a temporary fork/branch with a fix.
  4. I can no longer use the packaged release included in ROS, so an arduous process of manually compiling/installing said patch and all dependencies begins on every platform (I at least try to support Ubuntu + OS X) and physically installed machine I have.
  5. Some users of my package aren't experienced enough to do this themselves, so they are simply stuck and out of luck unless I can give them step-by-step instructions.
  6. Eventually a patch to the dependency gets added upstream
  7. repeat step (5) and (6) to go back to depending on upstream version, but things still need to be manually compiled until a new ROS release updates the dependency, which is often quite a while.

Is this something any of you have encountered?
 
Does anyone else have ideas to make this process much easier in ROS2? 

When considering the above, perhaps it will make sense why my first instinct was suggesting the mechanisms in brew to streamline this process. I've found it makes fixes for users simpler because I can push a change to my forked formula + post a couple simple brew commands on the web and everyone is good to go. However, it seems you have good reasons to stick to the native package manager on each platform. 

I think the people helped the most would be users who may simply lack the skills/experience to maintain their own versions of packages they depend on. I also think some developers may see value in reducing the complexity of this process. Thoughts?

Andrew Hundt

unread,
May 9, 2016, 5:39:38 PM5/9/16
to ROS SIG NG ROS


On Monday, May 9, 2016 at 5:37:15 PM UTC-4, Andrew Hundt wrote:
7. repeat step (5) and (6) to go back to depending on upstream version, but things still need to be manually compiled until a new ROS release updates the dependency, which is often quite a while.

 
sorry, this was supposed to be repeat steps (4) and (5)

Jonathan Bohren

unread,
May 9, 2016, 5:52:48 PM5/9/16
to ROS SIG NG ROS
> When considering the above, perhaps it will make sense why my first instinct was suggesting the mechanisms in brew to streamline this process.

Andrew, the example you gave is definitely something that happens all the time, but it seems like this use case has more to do with the management of forks of repos than it does with the distribution of "official" releases. Really there's nothing in ROS or ROS2 preventing this functionality other than a convenient place in the community for users to put forks.

rosdistro [1] doesn't have the model to capture forks, but if it did, it would probably take a day to write a script that uses something like vcstool to all of the appropriate dependencies into a source workspace. ROSIndex [3] does model forks of repos, and again, it wouldn't be hard to add a script to that website which generates a YAML file that vcstool can use to clone a bunch of packages into a workspace.

Then you could just share a list of your variants, pipe it into a program, and get all of the code you need to run that variant. 


-j

--

Andrew Hundt

unread,
May 9, 2016, 8:37:05 PM5/9/16
to ROS SIG NG ROS


On Monday, May 9, 2016 at 5:52:48 PM UTC-4, Jonathan Bohren wrote:
> When considering the above, perhaps it will make sense why my first instinct was suggesting the mechanisms in brew to streamline this process.

Andrew, the example you gave is definitely something that happens all the time, but it seems like this use case has more to do with the management of forks of repos than it does with the distribution of "official" releases. Really there's nothing in ROS or ROS2 preventing this functionality other than a convenient place in the community for users to put forks.

rosdistro [1] doesn't have the model to capture forks, but if it did, it would probably take a day to write a script that uses something like vcstool to all of the appropriate dependencies into a source workspace. ROSIndex [3] does model forks of repos, and again, it wouldn't be hard to add a script to that website which generates a YAML file that vcstool can use to clone a bunch of packages into a workspace.

Then you could just share a list of your variants, pipe it into a program, and get all of the code you need to run that variant. 


-j

Yes, this sounds like exactly the right idea! Can forking ROS, modifying, then sharing be made a first class citizen by design and very easy for users to do?

That would be brilliant!

Jonathan Bohren

unread,
May 10, 2016, 1:19:49 AM5/10/16
to ROS SIG NG ROS
> Yes, this sounds like exactly the right idea! Can forking ROS, modifying, then sharing be made a first class citizen by design and very easy for users to do?

You mean in rosdistro? I don't think that's the place for this, since it's meant specifically for distributions. For source support, and unofficial variants, forks are already first-class on rosindex. 

For example, consider the known forks of `conman`:

"Publishing" a fork to rosindex just requires a PR to the `rosforks` repo above, then on the next index generation, it will be included:

This could be turned into a dynamic website instead of a static one, modulo the resources required to develop and maintain it. At the end of the day, if the community cares about these features, the community needs to step up and support them. To start supporting what you're talking about, really all you need is an extension to rosindex which generates / update your vcstool YAML file, and a script to consume it and clone all the necessary repos. 

-j

--

Daniel Stonier

unread,
May 10, 2016, 12:00:27 PM5/10/16
to ros-sig...@googlegroups.com
On 6 May 2016 at 01:13, Jack O'Quin <jack....@gmail.com> wrote:
At ROSCon last October, Mark Shuttleworth proposed "snap" as a secure, cross-platform packaging designed for the Internet of Things. I notice that Ubuntu 16.04 Xenial supports it. 

Does anyone here have experience with snap they'd like to share?

We didn't use it, but we spent a day reviewing it. Seems like they are taking the web idea of sandboxing served applications with their own esoteric dependencies (think virtual machine/virtualenv/bitnami/docker) and  applying the same principle to regular operating system apps which traditionally share their resources with other applications. This gives you the same isolation benefits and the ability to concoct an app with esoteric dependencies.

This could give you a means of providing a snap for a particular package/node that uses an updated api and dependency chain, but you'd need to make sure it uses the same versions of messages to communicate with the rest of the system. 

So...problems of building aside, you'd also not want to do this too much as every 'updated' library you load with your snap will invariably be already loaded in duplicate by other nodes on your system. Unlike web served applications, or desktop applications, our robots tend not to have this many resources lying around.

Daniel

.





Is it a potential packaging option for ROS 2?

--
You received this message because you are subscribed to the Google Groups "ROS SIG NG ROS" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ros-sig-ng-ro...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply all
Reply to author
Forward
0 new messages