DIGITS unable to find caffe binary [UBUNTU 14.04]

[Mohit Jain]

Hey,
    I've been trying to get DIGITS running but am stuck with this trivial looking error for a long time now. When I run digits-devserver, i get the following prompt.

mohit@mohit-nightfury:~/research/cloudcv/digits/digits$ ./digits-devserver
  ___ ___ ___ ___ _____ ___
 |   \_ _/ __|_ _|_   _/ __|
 | |) | | (_ || |  | | \__ \
 |___/___\___|___| |_| |___/


cudaRuntimeGetVersion() failed with error #38
CAFFE_HOME "/home/mohit/research/cloudcv/caffe/caffe" is invalid:
 caffe binary not found at "/home/mohit/research/cloudcv/caffe/caffe"
Default value for caffe_root "<PATHS>" invalid:
 caffe binary not found in PATH
==================================== Caffe =====================================
Where is caffe installed?


 Suggested values:
 (P*) [PATH/PYTHONPATH] <PATHS>
>>

 I have caffe installed in the folder, 

/home/mohit/research/cloudcv/caffe/caffe

Ive tried the following as the paths for the prompt.

>>  /home/mohit/research/cloudcv/caffe/caffe
>> /home/mohit/research/cloudcv/caffe/caffe/.build_release/tools
>> /home/mohit/research/cloudcv/caffe/caffe/.build_release/tools/caffe
>> /home/mohit/research/cloudcv/caffe/caffe/build
>> /home/mohit/research/cloudcv/caffe/caffe/build/tools
>> /home/mohit/research/cloudcv/caffe/caffe/build/tools/caffe

and all of them lead to the same error.

ERROR: "<path-here>" is not a directory

Invalid input

Can someone help me out? I've installed the NVIDIA version of caffe from github and have the [[[  _caffe.so, libcaffe-nv.so, libcaffe-nv.so.0.11.0,  libgtest.a, libcaffe-nv.a, libcaffe-nv.so.0 , libcaffe.so, libproto.a   ]]] files in my caffe/.build_release/tools/ (if it matters).

Regards,
Mohit

[Luke Yeager]

You do seem to be trying all the right things. Sorry the error message isn't more helpful for you. 

If you enter "CAFFE_ROOT", DIGITS expects to find the caffe executable at CAFFE_ROOT/build/tools/caffe

So from what you've said above, I would expect /home/mohit/research/cloudcv/caffe/caffe to work. Have you tried just /home/mohit/research/cloudcv/caffe? Are you simply forgetting the layout of your directory structure?

You should be able to use tab-completion to check yourself. The caffe root should include LICENSE, build/, src/, include/, etc.

Other things to try:

Did you do "make all" AND "make py"?

What OS are you running?

Do you have the latest version of DIGITS?

[Mohit Jain]

Hi Luke,
    Thanks for taking time to help me. I have a folder 'digits' in which i cloned the digits repo from github and hence the same named folders (similarly for caffe). Basically, my directory structure is fine. However, when I type 'make py' I get the following error.

mohit@mohit-nightfury:~/research/cloudcv/caffe/caffe$ make py
make: Nothing to be done for `py'.

even here,

mohit@mohit-nightfury:~/research/cloudcv/caffe/caffe/build$ make py

make: *** No rule to make target `py'.  Stop.

while 'make all runs fine.

mohit@mohit-nightfury:~/research/cloudcv/caffe/caffe/build$ make all

[  1%] Built target proto

[ 81%] Built target caffe

[ 82%] Built target caffe.bin

[ 84%] Built target compute_image_mean

[ 86%] Built target convert_imageset

[ 86%] Built target device_query

[ 87%] Built target extract_features

[ 87%] Built target finetune_net

[ 89%] Built target net_speed_benchmark

[ 89%] Built target test_net

[ 91%] Built target train_net

[ 93%] Built target upgrade_net_proto_binary

[ 94%] Built target upgrade_net_proto_text

[ 94%] Built target convert_cifar_data

[ 96%] Built target convert_mnist_data

[ 98%] Built target convert_mnist_siamese_data

[100%] Built target pycaffe

Can you tell me something based on this knowledge? I missed out on the 'make py' command.

Regards,
Mohit

[Mohit Jain]

Also, I am using Ubuntu 14.04 and have the latest DIGITS version cloned from here : https://github.com/NVIDIA/DIGITS

On Monday, June 8, 2015 at 9:51:35 PM UTC+5:30, Luke Yeager wrote:

[Luke Yeager]

"Nothing to be done for py" just means you've already built it, that's not an error.

It might be worth doing a "make clean" and then rebuilding with "make all -j && make py".

[Mohit Jain]

I am using cmake for Caffe installation. If I use the simple "make all && make py", I get the error on the 'make all' run as follows :

CXX/LD -o .build_release/tools/caffe.bin

.build_release/lib/libcaffe-nv.so: undefined reference to `cv::imread(cv::String const&, int)'

.build_release/lib/libcaffe-nv.so: undefined reference to `cv::imencode(cv::String const&, cv::_InputArray const&, std::vector<unsigned char, std::allocator<unsigned char> >&, std::vector<int, std::allocator<int> > const&)'

.build_release/lib/libcaffe-nv.so: undefined reference to `cv::imdecode(cv::_InputArray const&, int)'

collect2: error: ld returned 1 exit status

make: *** [.build_release/tools/caffe.bin] Error 1

Is installing with cmake not supported?

Regards,
Mohit

[Mohit Jain]

Also, I have opencv installed on my machine. So I dont really understand why I get the errors of undefined reference to cv.

[Axel Straminsky]

That's weird. In /usr/include, can you see the opencv folder?

When I installed DIGITS, I based mostly on this guide, see if it's of use for you: http://www.joyofdata.de/blog/gpu-powered-deeplearning-with-nvidia-digits/

[Luke Yeager]

I am using cmake for Caffe installation

Ah, that's your problem.

This commit in the NVIDIA/caffe fork adds versioning to caffe, but only to the Make path, not the CMake path.

It's on the TODO list (and you're welcome to fix it yourself if you're a CMake pro!), but for now you should just follow these instructions exactly.

By the way, does CMake only generate build/tools/caffe and not build/tools/caffe.bin? DIGITS should have given you a more helpful error message.

[Mohit Jain]

Finally got DIGITS working! Thank you so much guys for your help! The problem in my general make all operation was that without cmake, the opencv libraries were not being linked to the make. As Axel suggested, I added the /usr/include to my LIBRARY_DIRS variable in Makefile.config and set the PKG_CONFIG flag and everything works smoothly! Thank you so much for your help.

PS : Luke, Im not really a cmake pro (rather just started using it a month ago) but I think its a really elegant way of doing thing (especially with the GUI options) and really hope someone goes ahead and adds this functionality to DIGITS-caffe! All the best!

Regards,
Mohit

--
You received this message because you are subscribed to a topic in the Google Groups "DIGITS Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/digits-users/mIef4eiXjx0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to digits-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/digits-users/fa180de6-50ad-418f-9e23-41e83e43efb0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Pratheesh V R]

HI Luke ,
I trying to setup digits in my Ubuntu machine 14.4 and i am laso getting the error same as Mohit got.
and i have followed the steps as per  the link you have provided.

http://www.joyofdata.de/blog/gpu-powered-deeplearning-with-nvidia-digits/
and the error message i am getting  pasted belw.


wishary@wishary:~/digits$ ./digits-devserver

  ___ ___ ___ ___ _____ ___
 |   \_ _/ __|_ _|_   _/ __|
 | |) | | (_ || |  | | \__ \
 |___/___\___|___| |_| |___/

[Luke Yeager]

Hi Pratheesh,

That's not an error, DIGITS just wants you to enter the path to your caffe installation. Something like "/home/user/caffe".

What version of the code are you using? That warning doesn't get printed anymore. Try using the DIGITS 2 Preview by grabbing the v2.0.0-preview tag.

[Ibrahim Sarker]

can you told me,why this is happening?

2015-08-25 16:34:22 [20150825-163104-971b] [ERROR] Train Caffe Model: Check failed: *ptr host allocation of size 116160000 failed
2015-08-25 16:34:28 [20150825-163104-971b] [ERROR] Train Caffe Model task failed with error code -6

Job Status Error

Initialized at 04:31:04 PM (1 second)
Running at 04:31:05 PM (3 minutes, 25 seconds)
Error at 04:34:31 PM
(Total - 3 minutes, 26 seconds)

Train Caffe Model Error

Initialized at 04:31:04 PM (1 second)
Running at 04:31:06 PM (3 minutes, 23 seconds)
Error at 04:34:29 PM
(Total - 3 minutes, 24 seconds)

ERROR: Check failed: *ptr host allocation of size 116160000 failed

conv2 needs backward computation.
pool1 needs backward computation.
norm1 needs backward computation.
relu1 needs backward computation.
conv1 needs backward computation.
label_data_1_split does not need backward computation.
data does not need backward computation.
This network produces output accuracy
This network produces output loss
Collecting Learning Rate and Weight Decay.
Network initialization done.
Memory required for data: 831529208
Solver scaffolding done.
Starting Optimization
Solving
Learning Rate Policy: step
Iteration 0, Testing net (#0)
Test net output #0: accuracy = 0.34
Test net output #1: loss = 1.09667 (* 1 = 1.09667 loss)
Check failed: *ptr host allocation of size 116160000 failed

[Sameer Khan]

HI mohit i have the same problem will you please figure it out

[Mohit Jain]

Hi Sameer,
    I wrote a small blog learning from the problems faced during these procedures, https://cloudcv.wordpress.com. Hope that it helps you out :)

Regards,
Mohit

--
You received this message because you are subscribed to a topic in the Google Groups "DIGITS Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/digits-users/mIef4eiXjx0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to digits-users...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/digits-users/a608211d-4676-46aa-83f7-e4aae93570a5%40googlegroups.com.

[Sameer Khan]

 ___ ___ ___ ___ _____ ___
 |   \_ _/ __|_ _|_   _/ __|
 | |) | | (_ || |  | | \__ \
 |___/___\___|___| |_| |___/

Default value for caffe_root "<PATHS>" invalid:
    caffe binary not found in PATH
==================================== Caffe =====================================
Where is caffe installed?

    Suggested values:
    (P*) [PATH/PYTHONPATH] <PATHS>

 i am still having this error

[Mohit Jain]

That is not an error, provide the path to your caffe-home (eg. /home/user/stuff/caffe/) directory there. (Make sure you have the Nvidia's clone of caffe)

-Mohit

To view this discussion on the web visit https://groups.google.com/d/msgid/digits-users/f1aae4e8-23c4-45df-81e7-e5fcb6255fd9%40googlegroups.com.

[Sameer Khan]

Sir can you provide the installation guide for nvidias clone caffe

[Mohit Jain]

Please read through the blog I mentioned above, it has everything you'll need.

-Mohit

[Sameer Khan]

i did provide the path in which caffe is installed but it still shows the same error

[elaine...@gmail.com]

Hi Luke,

Is pycaffe necessary for the installation of digits? I have caffe that works smoothly with command-line interface, but I didn't build pycaffe. When I tried to install digits, I kept getting this error: ValueError: Caffe executable not found at "/home/Elaine/caffe/build/tools". Not sure what to do now. 

Thanks in advance!

在 2015年6月9日星期二 UTC-5上午11:08:36，Luke Yeager写道：