tell conda build to not use non-system libs ?

[Chris Barker]

Hi all,

(testing on OS-X for now)

I'm trying to set up a recipe, and do a build, that depends on various third-party libs. Many for these are available as conda packages, but not all.

However, I have have who knows what other stuff on my system from macports, homebrew, etc. So when the configure command is run inside conda build, it finds libs it's looking for outside conda, and outside the system libs. So I end up with a package that depends on some vagaries of my system -- not so useful for others.

Example:

$ otool -L ~/miniconda/lib/libgd.3.dylib 

/Users/chris.barker/miniconda/lib/libgd.3.dylib:

@rpath/./libgd.3.dylib (compatibility version 4.0.0, current version 4.0.0)

/usr/lib/libiconv.2.dylib (compatibility version 7.0.0, current version 7.0.0)

@rpath/libjpeg.8.dylib (compatibility version 13.0.0, current version 13.0.0)

@rpath/libz.1.dylib (compatibility version 1.0.0, current version 1.2.8)

/usr/lib/libbz2.1.0.dylib (compatibility version 1.0.0, current version 1.0.5)

@rpath/libpng16.16.dylib (compatibility version 34.0.0, current version 34.0.0)

@rpath/libfreetype.6.dylib (compatibility version 18.0.0, current version 18.1.0)

/usr/local/lib/libfontconfig.1.dylib (compatibility version 10.0.0, current version 10.0.0)

@rpath/libtiff.5.dylib (compatibility version 7.0.0, current version 7.0.0)

/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 169.3.0)

you can see that it's mostly linked against conda libs (@rpath) and system libs, but there is a pesky libfontconfig in there that came from homebrew.

I'm pretty sure I can add a flag to the configure command for it to ignore fontconfig, but this is really a pain to get right -- I need to build, and check what libs it's found, etc, etc....

Is there a way to make sure that a conda package does not link against any non-system and non-conda libs?

I understand that it's the configure script that is finding these libs -- but is it possible to mangle the environment so that those non-system locations aren't found? Are there environment variables that tell configure where to look? or is there a really smart configure script that looks everywhere (/local, /opt, etc...)

OR, if not, maybe conda could at least have warning :: CAUTION, package built depends on non-system libraries.

Any other ideas welcome.

-Chris

--

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris....@noaa.gov

[Stuart Berg]

Hi Chris,

>Is there a way to make sure that a conda package does not link against any non-system and non-conda libs?

In general, this is beyond conda's control. One option, of course, is
to build your recipes within a virtual machine, which you can ensure
hasn't been polluted with packages from homebrew, macports, etc.

Beyond that, you'll need to pass the appropriate flags to your
configure script, as you said.

I have no personal experience with libgd, but it appears that project
supports both autotools (configure) and cmake. I have more experience
with cmake, so I'll recommend using that.

You can tell cmake where to look for libraries in general via
CMAKE_PREFIX_PATH. If they can be found there, then (in theory),
cmake shouldn't accidentally find the other stuff on your system. For
example, I bet this will be a good start for your recipe:

mkdir build
cd build
cmake .. \
-DCMAKE_INSTALL_PREFIX="${PREFIX}" \
-DCMAKE_PREFIX_PATH="${PREFIX}" \
-DCMAKE_BUILD_TYPE=Release \

If necessary, you can set the individual cache variables for each dependency:

-DPNG_LIBRARY=${PREFIX}/lib/libpng.dylib \
-DPNG_PNG_INCLUDE_DIR=${PREFIX}/include \

-DFONTCONFIG_LIBRARY=${PREFIX}/lib/libfontconfig.dylib \
-DFONTCONFIG_INCLUDE_DIR=${PREFIX}/include \

...of course, you'd be wise to double-check the output of otool as you
did before, to make sure the right libraries were picked up.

HTH,
Stuart

> --
> You received this message because you are subscribed to the Google Groups
> "conda - Public" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to conda+un...@continuum.io.
> To post to this group, send email to co...@continuum.io.
> Visit this group at http://groups.google.com/a/continuum.io/group/conda/.

[Chris Barker]

On Wed, Sep 30, 2015 at 9:50 AM, Stuart Berg <stuar...@gmail.com> wrote:

>Is there a way to make sure that a conda package does not link against any non-system and non-conda libs?

In general, this is beyond conda's control.  One option, of course, is
to build your recipes within a virtual machine, which you can ensure
hasn't been polluted with packages from homebrew, macports, etc.

yeah, I probably should do that -- but seems like a lot of overhead right now

:-)

 

Beyond that, you'll need to pass the appropriate flags to your
configure script, as you said.

yup -- now I have:

./configure --prefix=$PREFIX \

            --with-png=$PREFIX \

            --with-freetype=$PREFIX \

            --with-tiff=$PREFIX \

            --with-fontconfig=no \

            --with-xpm=no \

 

I have no personal experience with libgd, but it appears that project
supports both autotools (configure) and cmake.  I have more experience
with cmake, so I'll recommend using that.

I started down that path, and  I couldn't get cmake to work for me. Actually, I couldn't get autotools to work, either, but then I found a tarball with the configure script pre-built, and presto!

You can tell cmake where to look for libraries in general via
CMAKE_PREFIX_PATH.  If they can be found there, then (in theory),
cmake shouldn't accidentally find the other stuff on your system. 

That would be a nice feature

 

And then it would pick-up new ibs if they show up. And I"d ony need to add the dependency in  meta.yaml.

For
example, I bet this will be a good start for your recipe:

mkdir build
cd build
cmake .. \
    -DCMAKE_INSTALL_PREFIX="${PREFIX}" \
    -DCMAKE_PREFIX_PATH="${PREFIX}" \
    -DCMAKE_BUILD_TYPE=Release \

If necessary, you can set the individual cache variables for each dependency:

    -DPNG_LIBRARY=${PREFIX}/lib/libpng.dylib \
    -DPNG_PNG_INCLUDE_DIR=${PREFIX}/include \

    -DFONTCONFIG_LIBRARY=${PREFIX}/lib/libfontconfig.dylib \
    -DFONTCONFIG_INCLUDE_DIR=${PREFIX}/include \

hmm -- anaconda does provide cmake -- maybe a I should give that another try.

Could help with the Windows build, too...

...of course, you'd be wise to double-check the output of otool as you
did before, to make sure the right libraries were picked up.

yup -- I'm thinking of writing a script to do that -- which maybe could be merged into conda -- it would be nice to get a warning after a build id non-system libs were picked up. Someone might want that, but they also might not....

Thanks,

   -CHB