how to set library paths for applications that use spack package libraries on Cray systems

[Phil Carns]

Hi all,

I'm struggling a little bit with this scenario:

• compiling software on a Cray XC platform (e.g., Theta at the ALCF or Cori at NERSC)
• several libraries are installed and loaded via spack
• the end user application is *not* a spack package, but it wishes to use libraries from the above

The user application has a conventional autotools build system with pkg-config to find libraries and cflags.  Everything compiles, links, and installs without any problems (great!).

When I try to run the resulting executable, however, the dynamic linker cannot find any of the libraries that were provided by the spack packages.  I've checked the usual suspects in terms of how it might find these libraries:

• The libraries in question are not in the default system path (of course: they are in spack directories)
• "spack load" does not set LD_LIBRARY_PATH (or any of the auxiliary *_LIBRARY_PATH variables in the Cray environment) to point to the package paths
  
• spack strips *.la files from the library packages at install time (had they been present, then presumably the application's libtool would pick them up and use them to guide RPATH/RUNPATH settings when linking executable binary).  The library packages create and install .la files if installed by hand, but they are not present in the spack installs.
  

What's the best practice for my use case?  I suspect that I'm doing something wrong :-)

I *could* hack up a script to construct an LD_LIBRARY_PATH for the application at run time by iterating through "spack location -i" calls, but that doesn't seem like the right thing to ask a potential user of our libraries to do.  We have quite a few libraries possibly in play, not just one.

Cmake may handle this differently than autotools, but I can't make our users all adopt cmake.

If the application itself were built with spack, then I suspect spack would take care of the paths automatically, but that's not a viable option either.

Oddly enough, my laptop (Ubuntu 20.04) *does* set LD_LIBRARY_PATH on spack load, so I don't see this problem during local development.  I'm not sure if that's intentional or not that for this behavior to be different across platforms, though, so I don't know whether to report it as a bug or not.

thanks!

-Phil