Possible issue on long RPATH

[luigi calori]

I am trying to use Spack for building an environment for ( remote) visualization on a graphics cluster.

my main goal is to do:
spack install paraview+qt+python+tcl+opengl2%g...@4.8.2

After working around issues regarding current compiler and qt dependencies, I'm currently stuck on something that seems quite specific of Spack:

I got this error:

-- Installing: /galileo/home/userinternal/lcalori0/Spack/spack/opt/spack/linux-x
86_64/gcc-4.8.2/paraview-4.4.0-ksr4rzmlbotutfemnhjkjnwenujxx5ow/lib/paraview-4.4
/libvtkPVPythonCatalystPython27D-pv4.4.so
RegularExpression::compile(): Expression too big.
CMake Error at cmake_install.cmake:61 (file):
  file RPATH_CHANGE could not write new RPATH:

    /galileo/home/userinternal/lcalori0/Spack/spack/opt/spack/linux-x86_64/gcc-4
.8.2/paraview-4.4.0-ksr4rzmlbotutfemnhjkjnwenujxx5ow/lib:/galileo/home/userinter
nal/lcalori0/Spack/spack/opt/spack/linux-x86_64/gcc-4.8.2/paraview-4.4.0-ksr4rzm
lbotutfemnhjkjnwenujxx5ow/lib64:/galileo/home/userinternal/lcalori0/Spack/spack/
opt/spack/linux-x86_64/gcc-4.8.2/bzip2-1.0.6-4btagkaopoe2wfr2vug25mkuf6jmirm5/li
b:/galileo/home/userinternal/lcalori0/Spack/spack/opt/spack/linux-x86_64/gcc-4.8
.2/freetype-2.5.3-42cf6gupxnqwwnyj6sx4ecgvnvtkygs3/lib:/galileo/home/userinterna
l/lcalori0/Spack/spack/opt/spack/linux-x86_64/gcc-4.8.2/libpng-1.6.16-bgnhh427rb
3eifze6m24vxe773j5fgji/lib:/galileo/home/userinternal/lcalori0/Spack/spack/opt/s
.................................................really long RPATH string.............................



I think this is related to the large number of dependencies that build up a really huge path.

Anyone noted this?

also seems that extracting the graph has problems:
spack graph paraview+qt+python+tcl+opengl2%g...@4.8.2

crash with error IndexError: list index out of range

[Gamblin, Todd]

Hi Luigi,

Spack is probably overly cautious about RPATHs. It adds RPATHs for the entire dependency tree, but it probably only needs to add RPATHs for immediate dependencies since ld.so will search RPATHs of parent objects after those of the loading object.

I'm at a meeting today but you could try changing the get_rpaths function here:

https://github.com/LLNL/spack/blob/develop/lib/spack/spack/build_environment.py#L237

So that it uses pkg.spec.dependencies.values() instead of pkg.traverse() like it currently does. That should get you only immediate deps in your RPATH.

If it works, can you send a PR?

If you want to do a sanity check, I have a rough command here:

https://github.com/LLNL/spack/blob/features/libs-command/lib/spack/spack/cmd/libs.py

It's in a branch -- it runs ldd on libs in a package directory and checks for any dependencies that lie outside the Spack tree.  If the RPATHs are correct, there should not be any besides some basic system libraries.  I say it's rough because it has a hard coded list of "acceptable" system libs -- I want to clean that up before I merge the command.

Todd

 

---

From: sp...@googlegroups.com on behalf of luigi calori
Sent: Thursday, February 04, 2016 5:03:19 AM
To: Spack
Subject: [spack] Possible issue on long RPATH

--
You received this message because you are subscribed to the Google Groups "Spack" group.
To unsubscribe from this group and stop receiving emails from it, send an email to spack+un...@googlegroups.com.
To post to this group, send email to sp...@googlegroups.com.
Visit this group at https://groups.google.com/group/spack.
For more options, visit https://secure-web.cisco.com/18KxGEYL9pfTsSRXVwaIjrqpBTpX1keswmXOFE0BklBSloerbmAQ0DyVA1x2acMRUMpithYpC_C77UUOBW1bkMtZMIS3NtGu_w-arHYfOJEaxsUa6c_b44RdSUaCwkFrfsYY0nM1WuMsuC9nLTGttUfEVoUCjSFDmx-EaPqrJxlYiNqSazWdHznWCZ-XiOEUcjLV43q_d60PYC8BE3gkHXITZddqYYr0MqHomYs99oT2Kij922uCk_jAMocRZ8FdEUIkhOQLDWSjHRUwvaCcj2W3Ozg89knFRthTkcRPyirOx6pdCPpQLbDDnT3kEKY92mH1KKtSyL89UxX9CIrAu8DC54RiaETqkt20XrWsFrxFRnKaojieR8a7akq1DiDZD/https%3A%2F%2Fgroups.google.com%2Fd%2Foptout.

[Ben Boeckel]

On Thu, Feb 04, 2016 at 13:36:39 +0000, Gamblin, Todd wrote:
> So that it uses pkg.spec.dependencies.values() instead of
> pkg.traverse() like it currently does. That should get you only
> immediate deps in your RPATH.

Dependency types should also help this (i.e., no need to add Python to
the rpath if all it is used for is to run scons or whatever for the
build).

--Ben

[luigi calori]

How the type of dependencies is specified? are there already examples or is it a new feature?
Nevertheless, in the case of ParaView, I' m unsure about python: I think that ParaView links to pythonlib.
CMake is one dep  that could be specified as build only.

I think that Todd suggestion could short the path much more.

I' ll try it as soon as possible

[Ben Boeckel]

On Thu, Feb 04, 2016 at 10:43:05 -0800, luigi calori wrote:
> How the type of dependencies is specified? are there already examples or is
> it a new feature?

New feature, but it should look like:

depends_on('python', deptype=DEP_BUILD)
depends_on('coreutils', deptype=DEP_RUN)
depends_on('netcdf', deptype=(DEP_BUILD, DEP_INSTALL)) # this is the default
depends_on('mpi', deptype=DEP_ALL)

> Nevertheless, in the case of ParaView, I' m unsure about python: I think
> that ParaView links to pythonlib.

Indeed, this would be the case here (with +python).

> CMake is one dep that could be specified as build only.

Yep, (though I don't think it installs a lib/ directory, so it shouldn't
show up on the rpath).

> I think that Todd suggestion could short the path much more.

Yeah; I've put it into my branch as well (since I'm also changing code
around rpath anyways to only care about certain dependencies).

--Ben