Build error on OSX 10.12 : Undefined symbols for architecture x86_64: "_H5LTfind_dataset" (...)

[Dr. Dietrich Davidstein]

So the main build error is 

Undefined symbols for architecture x86_64:

  "_H5LTfind_dataset", referenced from:

(...)

but hdf5 is installed on my system (via brew).

I also tried building it with 2 different versions of hdf5:

1.10.0-patch1 and 1.8.16

but it didn't make any difference...

When i run cmake, for some reason hdf5 is not listed under the dependencies:

-- Dependencies:
--   BLAS              :   Yes (vecLib)
--   Boost             :   Yes (ver. 1.63)
--   glog              :   Yes
--   gflags            :   Yes
--   protobuf          :   Yes (ver. 3.2.0)
--   lmdb              :   Yes (ver. 0.9.19)
--   LevelDB           :   Yes (ver. 1.19)
--   Snappy            :   Yes (ver. 1.1.4)
--   OpenCV            :   Yes (ver. 2.4.13.2)
--   CUDA              :   Yes (ver. 8.0)

Do i need to manually add hdf5 somewhere?

Update: The compiler is definitely finding my hdf5 install:

-- HDF5: Using hdf5 compiler wrapper to determine C configuration
-- HDF5: Using hdf5 compiler wrapper to determine CXX configuration
-- Found HDF5: /usr/local/lib/libhdf5_cpp.dylib;/usr/local/lib/libhdf5.dylib;/usr/local/lib/libsz.dylib;/usr/lib/libz.dylib;/usr/lib/libdl.dylib;/usr/lib/libm.dylib (found version "1.10.0.1") found components:  HL

It's already in the Makefile:

LIBRARIES += glog gflags protobuf boost_system boost_filesystem m hdf5_hl hdf5 opencv_imgcodecs

Should i install hdf5 from source – would this fix this error?

Or is this a caffe specific problem?

I've tried some solutions i found on github, but couldn't figure out what's wrong with my build config.

Hope you can help me out here!

PS: Below is the full terminal output when i try to run make:

---

(...)
[ 78%] Building CXX object src/caffe/CMakeFiles/caffe.dir/util/math_functions.cpp.o

[ 78%] Building CXX object src/caffe/CMakeFiles/caffe.dir/util/signal_handler.cpp.o
/Users/david/github/caffe-projects/learn-reflectance/caffe/src/caffe/util/blocking_queue.cpp:50:7: warning: unused typedef 'INVALID_REQUESTED_LOG_SEVERITY' [-Wunused-local-typedef]
      LOG_EVERY_N(INFO, 1000)<< log_on_wait;
      ^
/usr/local/include/glog/logging.h:917:30: note: expanded from macro 'LOG_EVERY_N'
                             INVALID_REQUESTED_LOG_SEVERITY);           \
                             ^
[ 79%] Building CXX object src/caffe/CMakeFiles/caffe.dir/util/upgrade_proto.cpp.o
1 warning generated.
[ 79%] Linking CXX shared library ../../lib/libcaffe.dylib
Undefined symbols for architecture x86_64:
  "_H5LTfind_dataset", referenced from:
      caffe::SGDSolver<float>::RestoreSolverStateFromHDF5(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) in solver.cpp.o
      caffe::SGDSolver<double>::RestoreSolverStateFromHDF5(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) in solver.cpp.o
      void caffe::hdf5_load_nd_dataset_helper<float>(int, char const*, int, int, caffe::Array<float>*) in hdf5.cpp.o
      void caffe::hdf5_load_nd_dataset_helper<double>(int, char const*, int, int, caffe::Array<double>*) in hdf5.cpp.o
      void caffe::hdf5_load_nd_dataset_helper<float>(int, char const*, int, int, caffe::Blob<float>*) in hdf5.cpp.o
      void caffe::hdf5_load_nd_dataset_helper<double>(int, char const*, int, int, caffe::Blob<double>*) in hdf5.cpp.o
  "_H5LTget_dataset_info", referenced from:
      caffe::hdf5_info(int, char const*, int*) in fast_hdf5_layer.cpp.o
      void caffe::hdf5_load_nd_dataset_helper<float>(int, char const*, int, int, caffe::Array<float>*) in hdf5.cpp.o
      void caffe::hdf5_load_nd_dataset_helper<double>(int, char const*, int, int, caffe::Array<double>*) in hdf5.cpp.o
      void caffe::hdf5_load_nd_dataset_helper<float>(int, char const*, int, int, caffe::Blob<float>*) in hdf5.cpp.o
      void caffe::hdf5_load_nd_dataset_helper<double>(int, char const*, int, int, caffe::Blob<double>*) in hdf5.cpp.o
      caffe::hdf5_load_string(int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) in hdf5.cpp.o
  "_H5LTget_dataset_ndims", referenced from:
      caffe::hdf5_info(int, char const*, int*) in fast_hdf5_layer.cpp.o
      void caffe::hdf5_load_nd_dataset_helper<float>(int, char const*, int, int, caffe::Array<float>*) in hdf5.cpp.o
      void caffe::hdf5_load_nd_dataset_helper<double>(int, char const*, int, int, caffe::Array<double>*) in hdf5.cpp.o
      void caffe::hdf5_load_nd_dataset_helper<float>(int, char const*, int, int, caffe::Blob<float>*) in hdf5.cpp.o
      void caffe::hdf5_load_nd_dataset_helper<double>(int, char const*, int, int, caffe::Blob<double>*) in hdf5.cpp.o
  "_H5LTmake_dataset", referenced from:
      caffe::FastHDF5OutputLayer<float>::saveData(std::__1::vector<caffe::Blob<float>*, std::__1::allocator<caffe::Blob<float>*> > const&) in fast_hdf5_layer.cpp.o
      caffe::FastHDF5OutputLayer<double>::saveData(std::__1::vector<caffe::Blob<double>*, std::__1::allocator<caffe::Blob<double>*> > const&) in fast_hdf5_layer.cpp.o
  "_H5LTmake_dataset_double", referenced from:
      void caffe::hdf5_save_nd_dataset<double>(int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, caffe::Array<double> const&) in hdf5.cpp.o
      void caffe::hdf5_save_nd_dataset<double>(int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, caffe::Blob<double> const&, bool) in hdf5.cpp.o
  "_H5LTmake_dataset_float", referenced from:
      void caffe::hdf5_save_nd_dataset<float>(int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, caffe::Array<float> const&) in hdf5.cpp.o
      void caffe::hdf5_save_nd_dataset<float>(int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, caffe::Blob<float> const&, bool) in hdf5.cpp.o
  "_H5LTmake_dataset_int", referenced from:
      caffe::hdf5_save_int(int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int) in hdf5.cpp.o
  "_H5LTmake_dataset_string", referenced from:
      caffe::hdf5_save_string(int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) in hdf5.cpp.o
  "_H5LTread_dataset", referenced from:
      caffe::StaticHDF5InputLayer<float>::LayerSetUp(std::__1::vector<caffe::Blob<float>*, std::__1::allocator<caffe::Blob<float>*> > const&, std::__1::vector<caffe::Blob<float>*, std::__1::allocator<caffe::Blob<float>*> > const&) in fast_hdf5_layer.cpp.o
      caffe::StaticHDF5InputLayer<double>::LayerSetUp(std::__1::vector<caffe::Blob<double>*, std::__1::allocator<caffe::Blob<double>*> > const&, std::__1::vector<caffe::Blob<double>*, std::__1::allocator<caffe::Blob<double>*> > const&) in fast_hdf5_layer.cpp.o
      caffe::FastHDF5InputLayer<float>::load_batch(caffe::FastHDF5InputLayer<float>::Batch*) in fast_hdf5_layer.cpp.o
      caffe::FastHDF5InputLayer<double>::load_batch(caffe::FastHDF5InputLayer<double>::Batch*) in fast_hdf5_layer.cpp.o
  "_H5LTread_dataset_double", referenced from:
      void caffe::hdf5_load_nd_dataset<double>(int, char const*, int, int, caffe::Array<double>*) in hdf5.cpp.o
      void caffe::hdf5_load_nd_dataset<double>(int, char const*, int, int, caffe::Blob<double>*) in hdf5.cpp.o
  "_H5LTread_dataset_float", referenced from:
      void caffe::hdf5_load_nd_dataset<float>(int, char const*, int, int, caffe::Array<float>*) in hdf5.cpp.o
      void caffe::hdf5_load_nd_dataset<float>(int, char const*, int, int, caffe::Blob<float>*) in hdf5.cpp.o
  "_H5LTread_dataset_int", referenced from:
      caffe::hdf5_load_int(int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) in hdf5.cpp.o
  "_H5LTread_dataset_string", referenced from:
      caffe::hdf5_load_string(int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) in hdf5.cpp.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [lib/libcaffe.dylib] Error 1
make[1]: *** [src/caffe/CMakeFiles/caffe.dir/all] Error 2
make: *** [all] Error 2

[Kay Zhong]

Hi Dr. Dietrich Davidstein,

I have exactly the same situation. I was wondering if you find any solution for it.

Best wishes,

Yiqi

[Salil Tambe]

Hi Dr. Dietrich,

I have run into the exact same issue while trying to install caffe on OSX 10.12 using cmake. Did you find a fix at all?

thanks,

Salil