libral dependencies and platforms

[Andy Henroid]

Greetings, libral mailing list,

 

I’ve been very pleased to see that libral is heading down the path of building a lightweight, extensible RAL implementation that is also light on library dependencies (though with Boost in there, it could probably be lighter). The static libral/ralsh binaries available on the site are an excellent demonstration of this. Can you imagine a future with a single RAL binary that works across all (many) Linux platforms? That might be hard, but it’s a goal worth working towards.

 

In light of this, I was wondering about which Linux platforms today’s libral binaries work on. Today’s binaries are built on CentOS 6 and glibc 2.12, and while libral is still very much a work in progress, I thought it would be interesting to find out. The README says the binaries will work on any Linux platform with glibc 2.12 or later, but I’ve lost my glibc to Linux distro decoder ring, so I ran a quick experiment using all of the Linux platforms that I have easy access to.

 

Here’s what worked:

• RHEL 6, 7 (and related distros: CentOS, OracleLinux, ScientificLinux)
  
• Debian 6, 7, 8, 9
  
• SLES 11, 12
  
• Ubuntu 10.04, 12.04, 14.04, 15.04, 15.10, 16.04, 16.10
  
• Fedora 21, 22, 23, 24, 25

These did not work (based on glibc 2.3-2.9):

• RHEL 4, 5 (and CentOS 4, 5, etc.)
  
• SLES 10
  
• OpenSUSE 11

That’s a lot of platforms that libral works on right out of the box! And certainly there are many other platforms/variants/versions that would work but I haven’t tested.

• The README says the binaries may work with glibc 2.8 or 2.9, and I had mixed results. SLES 11 (glibc 2.11) worked fine, but OpenSUSE 11 (glibc 2.9) did not work.
  
• The binaries also worked on at least one Amazon Linux variant (a 2014.03 snapshot) but I imagine they would work on any release based on CentOS 6 and later.
  
• The binaries are currently built on 64-bit, and they do not work on any 32-bit platform. If they were built on 32-bit, would they work for both 32 and 64-bit? Maybe not. I should find out.
  
• Also, there may not be many interested in support for RHEL/CentOS 5, but with some build wrangling, that might also be possible.

Thanks for letting me share this quick survey. David and fellow libral contributors, great work! I’d like to join in too. And I’m hoping to see the libral project, as it grows and evolves, remain true to its “lightweight” goals.

 

Andy

 

[David Lutterkort]

Hi Andy,

first off, thanks for doing all this ! This is really great !

On Wed, Feb 22, 2017 at 3:17 PM, Andy Henroid <andy.h...@puppet.com> wrote:

 

I’ve been very pleased to see that libral is heading down the path of building a lightweight, extensible RAL implementation that is also light on library dependencies (though with Boost in there, it could probably be lighter).

As a sidenote: the libraries that libral uses right now are mostly based on expediency, and any change that reduces those dependencies would be very welcome.

 

Here’s what worked:

• RHEL 6, 7 (and related distros: CentOS, OracleLinux, ScientificLinux)
  
• Debian 6, 7, 8, 9
  
• SLES 11, 12
  
• Ubuntu 10.04, 12.04, 14.04, 15.04, 15.10, 16.04, 16.10
  
• Fedora 21, 22, 23, 24, 25

These did not work (based on glibc 2.3-2.9):

• RHEL 4, 5 (and CentOS 4, 5, etc.)
  
• SLES 10
  
• OpenSUSE 11

These results are really cool and informative. They would probably be really useful in the README, though I don't want to overwhelm that with a big compat matrix. Maybe a separate section/table at the end of that file ?
 

That’s a lot of platforms that libral works on right out of the box! And certainly there are many other platforms/variants/versions that would work but I haven’t tested.

• The README says the binaries may work with glibc 2.8 or 2.9, and I had mixed results. SLES 11 (glibc 2.11) worked fine, but OpenSUSE 11 (glibc 2.9) did not work.
  

That statement was mostly based on my spelunking with nm/readelf and looking at symbol versions. Good to see results of some actual experiments.

• The binaries also worked on at least one Amazon Linux variant (a 2014.03 snapshot) but I imagine they would work on any release based on CentOS 6 and later.
  
• The binaries are currently built on 64-bit, and they do not work on any 32-bit platform. If they were built on 32-bit, would they work for both 32 and 64-bit? Maybe not. I should find out.
  

This would be really neat, too. To be a bit flippant, a 32bit build should be good enough for everybody (if libral needs more than 4GB of memory, there's something really bad going on)

• Also, there may not be many interested in support for RHEL/CentOS 5, but with some build wrangling, that might also be possible.

I thought for five minutes about building on EL5, but then couldn't find a machine with it quickly enough. It should help in going even farther back in terms of glibc deps.

Another thing I'd been thinking about was to make a fully static build with musl, but I have never used it, and am not sure how far back that goes, and whether there are any worries about backwards compat issues with syscalls (since you're now running against the naked kernel).

Tbh, another thing on the build front that I would be really interested in is having an easily usable build for OSX, as a lot of people seem to develop on that platform (I know Gareth spent some time on fighting Xcode which seems to be stricter than g++-6.3.1 and was throwing up compile errors)

And finally: some sort of Travis builds to make sure things continue building would be really neat, too.

Thanks for letting me share this quick survey. David and fellow libral contributors, great work! I’d like to join in too. And I’m hoping to see the libral project, as it grows and evolves, remain true to its “lightweight” goals.

I am looking forward to your contributions !

David

[Andy Henroid]

David, I'm glad this was useful information. Yes, I'll capture the relevant bits in a PR for the README, but keep it unobtrusive, because it's too early for the project to get bogged down in compatibility maintenance mode.

Also, it should be easy to do a quick first pass on each of these build ideas (32-bit, EL5, musl, OSX, extending Travis) and report back to the list.

Andy

--
You received this message because you are subscribed to the Google Groups "libral" group.
To unsubscribe from this group and stop receiving emails from it, send an email to libral+unsubscribe@googlegroups.com.
To post to this group, send email to lib...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/libral/CAHN%2BA%2BXRmJ7_RrMvngK8u%2BSf%3DOcr67T6THVoiWdMe0uzZpvLcA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[David Lutterkort]

Hi Andy,

On Wed, Feb 22, 2017 at 6:37 PM, Andy Henroid <andy.h...@puppet.com> wrote:

Also, it should be easy to do a quick first pass on each of these build ideas (32-bit, EL5, musl, OSX, extending Travis) and report back to the list.

I think the two most useful items from that list would be build instructions for OSX and Travis - the other things I don't see as super-urgent right now, and things like writing more providers, or reducing/simplifying dependencies are probably more valuable right now.

David
 

[Andy Henroid]

I've posted two PRs to the libral project

* https://github.com/puppetlabs/libral/pull/5: This fixes the build for OS X. However, the resulting ralsh executable does not appear to work fully. More work ahead.

* https://github.com/puppetlabs/libral/pull/6: A Docker container based build environment for libral and leatherman. This has been useful to me (using an OS X development system). It's not clear to me if this would be useful to others, though, or if it belongs in the libral repo. The branch is there, however, if anyone wants to try it out.

David, by the time you sent the last email, I had already tried out two of the low-priority items (only took a few minutes):

* A static build on CentOS 5 works great. If this is interesting, I can build you a new static image or send you a quick script to rebuild on the internal VMpooler systems. (CentOS 4 VMs are also available there, and we have pl-tools for CentOS 4. However, CentOS 4 is no longer a Puppet supported platform, so there is no puppet-agent package available--and I met some resistance doing a static Augeas build. Oh well.)

* Libral builds and runs fine on 32-bit, but as I feared, that same binary will not run on 64-bit systems, aborting with an ELF format error. If I recall, I've done this successfully on Windows (build on 32-bit, run on 64-bit) so, well, that's one small thing they've done better.

I'll continue dabbling, ticking off work items, and report back.

Andy

--

You received this message because you are subscribed to the Google Groups "libral" group.
To unsubscribe from this group and stop receiving emails from it, send an email to libral+unsubscribe@googlegroups.com.
To post to this group, send email to lib...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/libral/CAHN%2BA%2BWNSP3X-SXYQcJS41BCnGLZsE6Awc%2B1CqU%2B7SSAbrCn_g%40mail.gmail.com.

[David Schmitt]

On 24 February 2017 at 08:10, Andy Henroid <andy.h...@puppet.com> wrote:

I've posted two PRs to the libral project

* https://github.com/puppetlabs/libral/pull/5: This fixes the build for OS X. However, the resulting ralsh executable does not appear to work fully. More work ahead.

* https://github.com/puppetlabs/libral/pull/6: A Docker container based build environment for libral and leatherman. This has been useful to me (using an OS X development system). It's not clear to me if this would be useful to others, though, or if it belongs in the libral repo. The branch is there, however, if anyone wants to try it out.

David, by the time you sent the last email, I had already tried out two of the low-priority items (only took a few minutes):

* A static build on CentOS 5 works great. If this is interesting, I can build you a new static image or send you a quick script to rebuild on the internal VMpooler systems. (CentOS 4 VMs are also available there, and we have pl-tools for CentOS 4. However, CentOS 4 is no longer a Puppet supported platform, so there is no puppet-agent package available--and I met some resistance doing a static Augeas build. Oh well.)

* Libral builds and runs fine on 32-bit, but as I feared, that same binary will not run on 64-bit systems, aborting with an ELF format error. If I recall, I've done this successfully on Windows (build on 32-bit, run on 64-bit) so, well, that's one small thing they've done better.

Interesting, I would have expected it to work, if you have the 32bit libraries available on the 64bit install.

In other news, I've poked a bit at the travis.yml. See https://travis-ci.org/DavidS/libral for first results. With only minor changes it at least passes CPPCHECK. Now the sources need some tightening up to make the rest of the tools pass/

Cheers, David

[David Lutterkort]

Hi Andy,

On Fri, Feb 24, 2017 at 12:10 AM, Andy Henroid <andy.h...@puppet.com> wrote:

I've posted two PRs to the libral project

* https://github.com/puppetlabs/libral/pull/5: This fixes the build for OS X. However, the resulting ralsh executable does not appear to work fully. More work ahead.

* https://github.com/puppetlabs/libral/pull/6: A Docker container based build environment for libral and leatherman. This has been useful to me (using an OS X development system). It's not clear to me if this would be useful to others, though, or if it belongs in the libral repo. The branch is there, however, if anyone wants to try it out.

Nice work !

 

David, by the time you sent the last email, I had already tried out two of the low-priority items (only took a few minutes):

* A static build on CentOS 5 works great. If this is interesting, I can build you a new static image or send you a quick script to rebuild on the internal VMpooler systems. (CentOS 4 VMs are also available there, and we have pl-tools for CentOS 4. However, CentOS 4 is no longer a Puppet supported platform, so there is no puppet-agent package available--and I met some resistance doing a static Augeas build. Oh well.)

Nice. It looks like RHEL4 is going EOL at the end of March (as much as these releases ever go EOL) so RHEL5 seems like a time-travel far enough ago. And yes, I'd love to have that script.

 

* Libral builds and runs fine on 32-bit, but as I feared, that same binary will not run on 64-bit systems, aborting with an ELF format error. If I recall, I've done this successfully on Windows (build on 32-bit, run on 64-bit) so, well, that's one small thing they've done better.

That's strange .. at least on RH distros, you can definitely run 32bit and 64bit binaries side-by-side, though you'd need (for the static build) a 32bit glibc on x86_64 which most systems won't have installed.

But as I said, I don't think that's a huge priority: for installation, there will have to be some checking/decision based on the platform anyway at some point (is it Linux ? is it OS/X ?) and these differences just mean that the checking needs to be a bit more fine-grained.

David

[David Lutterkort]

Nice; too bad the actual build fails ;) The build failures look like they might be caused by a g++ that's just a tad too old. I'll have a look at them next week. For now, I'd be totally happy with a travis build without the DOXYGEN and CPPLINT targets, too.

David