Caffe Compile Error (test_net.bin failed)

[Ali]

Hi,

I am installing Caffe in Fedora 23 and solved all the dependencies (hopefully).

I am using Anaconda and OpenBLAS (instead of ATLAS).

When I try to compile Caffe using "make all", the following error occurs:

-----------------------------------------------------------------------------------------------------------------------

CXX/LD -o .build_release/tools/test_net.bin

/usr/bin/ld: cannot find -lopeblas

collect2: error: ld returned 1 exit status

Makefile:616: recipe for target '.build_release/tools/test_net.bin' failed

make: *** [.build_release/tools/test_net.bin] Error 1

-----------------------------------------------------------------------------------------------------------------------

I tried to compile using cmake instead. But I could not conceive cmake to use OpenBLAS instead of ATLAS:

-----------------------------------------------------------------------------------------------------------------------

-- Boost version: 1.58.0

-- Found the following Boost libraries:

--   system

--   thread

--   filesystem

-- Found gflags  (include: /usr/include, library: /usr/lib64/libgflags.so)

-- Found glog    (include: /usr/include, library: /usr/lib64/libglog.so)

-- Found PROTOBUF Compiler: /usr/bin/protoc

-- Found lmdb    (include: /usr/local/include, library: /usr/local/lib/liblmdb.so)

-- Found LevelDB (include: /usr/include, library: /usr/lib64/libleveldb.so)

-- Found Snappy  (include: /usr/include, library: /usr/lib64/libsnappy.so)

-- CUDA detected: 7.5

-- Automatic GPU detection failed. Building for all known architectures.

-- Added CUDA NVCC flags for: sm_20 sm_21 sm_30 sm_35 sm_50

-- OpenCV found (/usr/lib64/OpenCV)

CMake Error at /usr/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:148 (message):

  Could NOT find Atlas (missing: Atlas_CBLAS_LIBRARY Atlas_BLAS_LIBRARY

  Atlas_LAPACK_LIBRARY)

Call Stack (most recent call first):

  /usr/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:388 (_FPHSA_FAILURE_MESSAGE)

  cmake/Modules/FindAtlas.cmake:43 (find_package_handle_standard_args)

  cmake/Dependencies.cmake:88 (find_package)

  CMakeLists.txt:43 (include)

-- Configuring incomplete, errors occurred!

See also "/home/boroujerdi/14 NEW TREND/Deep Learning Frameworks/Caffe/installation/9 Caffe/caffe-master/build/CMakeFiles/CMakeOutput.log".

See also "/home/boroujerdi/14 NEW TREND/Deep Learning Frameworks/Caffe/installation/9 Caffe/caffe-master/build/CMakeFiles/CMakeError.log".

-----------------------------------------------------------------------------------------------------------------------

Is there any solution to remove the first error and solve the problem without installing ATLAS?

my Makefile.config is:

-----------------------------------------------------------------------------------------------------------------------

## Refer to http://caffe.berkeleyvision.org/installation.html

# Contributions simplifying and improving our build system are welcome!

# cuDNN acceleration switch (uncomment to build with cuDNN).

# USE_CUDNN := 1

# CPU-only switch (uncomment to build without GPU support).

CPU_ONLY := 1

# uncomment to disable IO dependencies and corresponding data layers

# USE_OPENCV := 0

# USE_LEVELDB := 0

# USE_LMDB := 0

# uncomment to allow MDB_NOLOCK when reading LMDB files (only if necessary)

# You should not set this flag if you will be reading LMDBs with any

# possibility of simultaneous read and write

# ALLOW_LMDB_NOLOCK := 1

# Uncomment if you're using OpenCV 3

# OPENCV_VERSION := 3

# To customize your choice of compiler, uncomment and set the following.

# N.B. the default for Linux is g++ and the default for OSX is clang++

# CUSTOM_CXX := g++

# CUDA directory contains bin/ and lib/ directories that we need.

CUDA_DIR := /usr/local/cuda

# On Ubuntu 14.04, if cuda tools are installed via

# "sudo apt-get install nvidia-cuda-toolkit" then use this instead:

# CUDA_DIR := /usr

# CUDA architecture setting: going with all of them.

# For CUDA < 6.0, comment the *_50 lines for compatibility.

CUDA_ARCH := -gencode arch=compute_20,code=sm_20 \

-gencode arch=compute_20,code=sm_21 \

-gencode arch=compute_30,code=sm_30 \

-gencode arch=compute_35,code=sm_35 \

-gencode arch=compute_50,code=sm_50 \

-gencode arch=compute_50,code=compute_50

# BLAS choice:

# atlas for ATLAS (default)

# mkl for MKL

# open for OpenBlas

BLAS := open

# Custom (MKL/ATLAS/OpenBLAS) include and lib directories.

# Leave commented to accept the defaults for your choice of BLAS

# (which should work)!

BLAS_INCLUDE := /home/boroujerdi/local/OpenBLAS/include

BLAS_LIB := /home/boroujerdi/local/OpenBLAS/lib

# Homebrew puts openblas in a directory that is not on the standard search path

# BLAS_INCLUDE := $(shell brew --prefix openblas)/include

# BLAS_LIB := $(shell brew --prefix openblas)/lib

# This is required only if you will compile the matlab interface.

# MATLAB directory should contain the mex binary in /bin.

# MATLAB_DIR := /usr/local

# MATLAB_DIR := /Applications/MATLAB_R2012b.app

# NOTE: this is required only if you will compile the python interface.

# We need to be able to find Python.h and numpy/arrayobject.h.

#PYTHON_INCLUDE := /usr/include/python2.7 \

#/usr/lib/python2.7/dist-packages/numpy/core/include

# Anaconda Python distribution is quite popular. Include path:

# Verify anaconda location, sometimes it's in root.

ANACONDA_HOME := /home/boroujerdi/anaconda

PYTHON_INCLUDE := $(ANACONDA_HOME)/include \

$(ANACONDA_HOME)/include/python2.7 \

$(ANACONDA_HOME)/lib/python2.7/site-packages/numpy/core/include \

# Uncomment to use Python 3 (default is Python 2)

# PYTHON_LIBRARIES := boost_python3 python3.5m

# PYTHON_INCLUDE := /usr/include/python3.5m \

#                 /usr/lib/python3.5/dist-packages/numpy/core/include

# We need to be able to find libpythonX.X.so or .dylib.

# PYTHON_LIB := /usr/lib

PYTHON_LIB := $(ANACONDA_HOME)/lib

# Homebrew installs numpy in a non standard path (keg only)

# PYTHON_INCLUDE += $(dir $(shell python -c 'import numpy.core; print(numpy.core.__file__)'))/include

# PYTHON_LIB += $(shell brew --prefix numpy)/lib

# Uncomment to support layers written in Python (will link against Python libs)

# WITH_PYTHON_LAYER := 1

# Whatever else you find you need goes here.

INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include

LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib

# If Homebrew is installed at a non standard location (for example your home directory) and you use it for general dependencies

# INCLUDE_DIRS += $(shell brew --prefix)/include

# LIBRARY_DIRS += $(shell brew --prefix)/lib

# Uncomment to use `pkg-config` to specify OpenCV library paths.

# (Usually not necessary -- OpenCV libraries are normally installed in one of the above $LIBRARY_DIRS.)

# USE_PKG_CONFIG := 1

BUILD_DIR := build

DISTRIBUTE_DIR := distribute

# Uncomment for debugging. Does not work on OSX due to https://github.com/BVLC/caffe/issues/171

# DEBUG := 1

# The ID of the GPU that 'make runtest' will use to run unit tests.

TEST_GPUID := 0

# enable pretty build (comment to see full commands)

Q ?= @

LIBRARIES += opeblas

-----------------------------------------------------------------------------------------------------------------------

Thanks.

[Jan C Peters]

First of all, the cmake build is not related to the Makefile.config at all! These are two different and completely separate ways of building caffe:

- plain make build, using Makefile.config, or

- generate Makefiles (optimally in another directory) with cmake, then do "make" there.

For CMake to use the BLAS you want, you need to set the cmake cache variable BLAS to the correct setting. A user-friendly way to do that is using cmake-gui instead of the plain cmake-command. If you are not familiar with cmake, either acquaint yourself or use the other build method.

For the Makefile.config based build you did almost everything right, with one exception: You do not need to give the openblas library in the LIBRARIES variable (last line), that is done automatically when you choose BLAS := open above. But when you give an invalid library name there (which you did, there is an 'n' missing), this will make your build fail, because the linker searches for 'opeblas', that does not exist.

Jan

[Ali]

@Jan C Peter Many thanks for your reply.

When I remove the last line and run "make all", I have this situation:

-------------------------------------------------------------------------------

PROTOC src/caffe/proto/caffe.proto

CXX .build_release/src/caffe/proto/caffe.pb.cc

CXX src/caffe/net.cpp

CXX src/caffe/layer_factory.cpp

CXX src/caffe/parallel.cpp

CXX src/caffe/internal_thread.cpp

CXX src/caffe/layer.cpp

CXX src/caffe/util/im2col.cpp

CXX src/caffe/util/insert_splits.cpp

CXX src/caffe/util/benchmark.cpp

CXX src/caffe/util/db.cpp

CXX src/caffe/util/db_lmdb.cpp

CXX src/caffe/util/math_functions.cpp

CXX src/caffe/util/cudnn.cpp

CXX src/caffe/util/blocking_queue.cpp

CXX src/caffe/util/hdf5.cpp

CXX src/caffe/util/db_leveldb.cpp

CXX src/caffe/util/signal_handler.cpp

CXX src/caffe/util/upgrade_proto.cpp

CXX src/caffe/util/io.cpp

CXX src/caffe/layers/hinge_loss_layer.cpp

CXX src/caffe/layers/cudnn_softmax_layer.cpp

CXX src/caffe/layers/bnll_layer.cpp

CXX src/caffe/layers/embed_layer.cpp

CXX src/caffe/layers/conv_layer.cpp

CXX src/caffe/layers/inner_product_layer.cpp

CXX src/caffe/layers/softmax_layer.cpp

CXX src/caffe/layers/base_data_layer.cpp

CXX src/caffe/layers/loss_layer.cpp

CXX src/caffe/layers/image_data_layer.cpp

CXX src/caffe/layers/hdf5_data_layer.cpp

CXX src/caffe/layers/deconv_layer.cpp

CXX src/caffe/layers/base_conv_layer.cpp

CXX src/caffe/layers/accuracy_layer.cpp

CXX src/caffe/layers/data_layer.cpp

CXX src/caffe/layers/batch_reindex_layer.cpp

CXX src/caffe/layers/reshape_layer.cpp

CXX src/caffe/layers/spp_layer.cpp

CXX src/caffe/layers/im2col_layer.cpp

CXX src/caffe/layers/slice_layer.cpp

CXX src/caffe/layers/cudnn_lrn_layer.cpp

CXX src/caffe/layers/sigmoid_cross_entropy_loss_layer.cpp

CXX src/caffe/layers/mvn_layer.cpp

CXX src/caffe/layers/log_layer.cpp

CXX src/caffe/layers/tile_layer.cpp

CXX src/caffe/layers/cudnn_conv_layer.cpp

CXX src/caffe/layers/dropout_layer.cpp

CXX src/caffe/layers/softmax_loss_layer.cpp

CXX src/caffe/layers/memory_data_layer.cpp

CXX src/caffe/layers/batch_norm_layer.cpp

CXX src/caffe/layers/silence_layer.cpp

CXX src/caffe/layers/cudnn_lcn_layer.cpp

CXX src/caffe/layers/power_layer.cpp

CXX src/caffe/layers/sigmoid_layer.cpp

CXX src/caffe/layers/relu_layer.cpp

CXX src/caffe/layers/cudnn_pooling_layer.cpp

CXX src/caffe/layers/window_data_layer.cpp

CXX src/caffe/layers/reduction_layer.cpp

CXX src/caffe/layers/neuron_layer.cpp

CXX src/caffe/layers/cudnn_relu_layer.cpp

CXX src/caffe/layers/scale_layer.cpp

CXX src/caffe/layers/dummy_data_layer.cpp

CXX src/caffe/layers/pooling_layer.cpp

CXX src/caffe/layers/cudnn_sigmoid_layer.cpp

CXX src/caffe/layers/prelu_layer.cpp

CXX src/caffe/layers/eltwise_layer.cpp

CXX src/caffe/layers/euclidean_loss_layer.cpp

CXX src/caffe/layers/hdf5_output_layer.cpp

CXX src/caffe/layers/input_layer.cpp

CXX src/caffe/layers/contrastive_loss_layer.cpp

CXX src/caffe/layers/concat_layer.cpp

CXX src/caffe/layers/lrn_layer.cpp

CXX src/caffe/layers/flatten_layer.cpp

CXX src/caffe/layers/split_layer.cpp

CXX src/caffe/layers/filter_layer.cpp

CXX src/caffe/layers/absval_layer.cpp

CXX src/caffe/layers/multinomial_logistic_loss_layer.cpp

CXX src/caffe/layers/elu_layer.cpp

CXX src/caffe/layers/exp_layer.cpp

CXX src/caffe/layers/infogain_loss_layer.cpp

CXX src/caffe/layers/tanh_layer.cpp

CXX src/caffe/layers/bias_layer.cpp

CXX src/caffe/layers/cudnn_tanh_layer.cpp

CXX src/caffe/layers/argmax_layer.cpp

CXX src/caffe/layers/threshold_layer.cpp

CXX src/caffe/syncedmem.cpp

CXX src/caffe/common.cpp

CXX src/caffe/data_transformer.cpp

CXX src/caffe/blob.cpp

CXX src/caffe/solver.cpp

CXX src/caffe/data_reader.cpp

CXX src/caffe/solvers/adadelta_solver.cpp

CXX src/caffe/solvers/sgd_solver.cpp

CXX src/caffe/solvers/adam_solver.cpp

CXX src/caffe/solvers/nesterov_solver.cpp

CXX src/caffe/solvers/adagrad_solver.cpp

CXX src/caffe/solvers/rmsprop_solver.cpp

AR -o .build_release/lib/libcaffe.a

LD -o .build_release/lib/libcaffe.so.1.0.0-rc3

CXX tools/test_net.cpp

CXX/LD -o .build_release/tools/test_net.bin

/usr/bin/ld: warning: libgfortran.so.1, needed by /home/boroujerdi/anaconda/lib/libopenblas.so, not found (try using -rpath or -rpath-link)

/home/boroujerdi/anaconda/lib/libopenblas.so: undefined reference to `_gfortran_compare_string'

/home/boroujerdi/anaconda/lib/libopenblas.so: undefined reference to `_gfortran_pow_r8_i4'

/home/boroujerdi/anaconda/lib/libopenblas.so: undefined reference to `_gfortran_concat_string'

/home/boroujerdi/anaconda/lib/libopenblas.so: undefined reference to `_gfortran_copy_string'

/home/boroujerdi/anaconda/lib/libopenblas.so: undefined reference to `_gfortran_pow_i4_i4'

/home/boroujerdi/anaconda/lib/libopenblas.so: undefined reference to `_gfortran_pow_r4_i4'

/home/boroujerdi/anaconda/lib/libopenblas.so: undefined reference to `_gfortran_etime'

collect2: error: ld returned 1 exit status

Makefile:616: recipe for target '.build_release/tools/test_net.bin' failed

make: *** [.build_release/tools/test_net.bin] Error 1

-------------------------------------------------------------------------------

Any idea to solve that?

[Ali]

Also, gcc-gfortran-5.3.1-2.fc23.x86_64 is already installed.

[Jan C Peters]

Mhm, that is strange...

Ah, did you install openblas somehow through Anaconda? That would explain a lot. In that case the "missing" gfortran library is probably somewhere in /home/boroujerdi/anaconda/lib too. Since this is not a standard lib search path, it is not found by ld. You can use either some LD_LIBRARY_PATH based solution, or look for other solutions on the internet. This is a quite common "problem". Although that could result in version clashes between your system's libraries and the anaconda-provided ones. In my opinion anaconda is not very useful in a unixoid environment, it even makes things more difficult since you already have a working python environment and add another, which almost inevitably to problems. You could simply use the python provided by your distro and install additional python modules with either your distro's package manager or (if you want more recent package versions or don't have root access) with pip install --user. Anaconda is great for easily establishing a python ecosystem in a windows machine, but for linux it usually brings more problems than solutions.

And for the question why installing gcc-gfortran-5.3.1-2.fc23.x86_64 does not solve the problem: This package provides libgfortran.so.3, whereas your libopenblas needs libgfortran.so.1 (an older version). Such things usually only happen if you install binary packages from different (and as such usually inconsistent) sources, like in your case the fedora repos as well the conda system.

Jan


Am Donnerstag, 3. März 2016 18:20:13 UTC+1 schrieb Ali:

Also, gcc-gfortran-5.3.1-2.fc23.x86_64 is already installed.

[Jan C Peters]

I should add: probably the best solution in your case is simply using the fedora-provdided openblas package (by dnf install openblas), and adjust the include and lib paths. And as I said, I wouldn't recommend using anaconda, but that is a different matter.

Jan