Caffe without GPU: make runtest error

[Sully Chen]

I'm trying to compile caffe without GPU support. I successfully ran "make all" and "make test", but when I try to run "make runtest", I get the following output:

   .build_release/tools/caffe
    dyld: Library not loaded: libcaffe.so.1.0.0-rc3
      Referenced from: /Users/SullyChen/Desktop/caffe/.build_release/tools/caffe
      Reason: image not found
    make: *** [runtest] Trace/BPT trap: 5

Any ideas on how to fix?

[Mateusz Herczka]

 I have the exact same problem. GPU off, Anaconda on. 

.build_release/tools/caffe

dyld: Library not loaded: libcaffe.so.1.0.0-rc3

  Referenced from: /Users/mh/caffe/.build_release/tools/caffe

  Reason: image not found

make: *** [runtest] Trace/BPT trap: 5

Some earlier posts suggest using DYLD_FALLBACK_LIBRARY_PATH, so I tried adding a path to the directory where libcaffe.so.1.0.0-rc3 is, but no improvement. Here is how it looks:

echo $DYLD_FALLBACK_LIBRARY_PATH

/usr/local/cuda/lib:/Users/mh/anaconda/lib:/usr/local/lib:/usr/lib:/Users/mh/caffe/build/lib

I'm on MacOS 10.9.5.

[Jan C Peters]

I have no experience with MacOS, but that error seems to be essentially the same as on linux: At runtime the caffe lib is not found, since it is not in the standard library paths. In Linux you can give additional lib search paths using the env variable LD_LIBRARY_PATH. I don't know if there is an equivalent in MacOS.

Jan

[Sully Chen]

There is an OS X equivalent, and I tried the following before I posted this question:

export LD_LIBRARY_PATH=/Users/SullyChen/Desktop/caffe/.build_release/lib

export DYLD_LIBRARY_PATH=/Users/SullyChen/Desktop/caffe/.build_release/lib

I'm not sure why none of that worked. I compiled Caffe a while ago on an older version of OS X, but I'm running 10.11.2 and struggling to compile.

[Bruno Korbar]

Any luck with this? 

I've trying to manually load the libraries and still no luck. 

[Sully Chen]

I still haven't found a fix :(

[David Flanagan]

I have this same error. It happens when building with GPU support as well.  I believe that it only happens on MacOS 10.11 El Capitan.

The problem is that Apple has added security features that prevent the DYLD_LIBRARY_PATH (and all DYLD_* environment variables) from being passed through to subshells.  This is probably a good thing, but it broke a lot of Unix-style software build processes.  In this case the problem is that make creates a subshell to run all of its commands.  The subshell does not get the environment variables that specify where to find the shared libraries.  And beause Caffe doesn't have a 'make install' step, libcaffe is not found in the standard places like /usr/lib.

Run `make -n runtest`  That will print out the commands that make wants to run.

If you copy and paste those commands directly into your shell, they should work. It is only when make tries to run them in a subshell that they fail.

I'm not sure if this is only a problem while building Caffe, or if I'm going to keep seeing it when I try to work through some of the Caffe tutorials. If it does keep happening, I'll try copying libcaffe into /usr/local/lib and see if it just works there.  If not, I'll try /usr/lib.  If that doesn't work, it will be back to the internet to find some other solution. Apple's security feature that is causing this is called SIP, and I have seen descriptions of how to disable SIP, though that seems like a drastic step to take.

Hope this helps!

  David

[Sully Chen]

Manually running the commands printed by make -n runtest didn't work, but copying the lib files from caffe/.build_release/lib to /usr/local/lib allowed me to run make runtest. Thanks, yay!

[Sina Afrooze]

I'm not sure how the linking bug was introduced in the latest caffe, but running the following two lines after `make all`and `make test` fixes the problem for good:

install_name_tool -change libcaffe.so.1.0.0-rc3 @executable_path/../lib/libcaffe.so.1.0.0-rc3 .build_release/tools/caffe

install_name_tool -change libcaffe.so.1.0.0-rc3 @executable_path/../lib/libcaffe.so.1.0.0-rc3 .build_release/test/test_all.testbin

[Sina Afrooze]

Also if using pycaffe, the following command fixes the shared lib:

install_name_tool -change libcaffe.so.1.0.0-rc3 @loader_path/../../.build_release/lib/libcaffe.so.1.0.0-rc3 python/caffe/_caffe.so