Error linking CXX executables

[Mythic Cocoa]

Hi,
I just had a weird experience while building caffe..

I executed make --jobs=4 and it ran all well till it reached 92%. Then while it was linking the CXX executable compute_image_mean, convert_image set, extract_features, and caffe it threw the same error - ../lib/libcaffe.so.1.0.0-rc3: undefined reference to 'cudnnConvolutionBackwardData_v3'.

What worries me here is..I have successfully built caffe before, with the same installation, same paths, the same device..basically the same everything. What exactly is going on? I haven't touched cudnn and installed DIGITS yesterday. I had some installation errors because of the caffe path not being recognized (that's why I'm here and building it again), but no issue in its requirement for cudnn, which proves nothing is wrong with cudnn.

Aside from advice on how to fix it, I would like to know if caffe automatically resets its paths over time or something similar.

Operating system: Ubuntu 14.04 LTS
CUDA version: 7.5

This is the full output:

Linking CXX shared library ../../lib/libcaffe.so
[ 90%] Built target caffe
Scanning dependencies of target caffe.bin
[ 91%] Building CXX object tools/CMakeFiles/caffe.bin.dir/caffe.cpp.o
Linking CXX executable caffe
../lib/libcaffe.so.1.0.0-rc3: undefined reference to `cudnnConvolutionBackwardData_v3'
../lib/libcaffe.so.1.0.0-rc3: undefined reference to `cudnnConvolutionBackwardFilter_v3'
collect2: error: ld returned 1 exit status
make[2]: *** [tools/caffe] Error 1
make[1]: *** [tools/CMakeFiles/caffe.bin.dir/all] Error 2
make: *** [all] Error 2

Thanks

[Przemek D]

Check the last 5 posts in this thread: https://groups.google.com/forum/#!topic/caffe-users/rFxaXqqwc-c - are they relevant?
Also, post which version(s) of cuDNN you're using.

[Mythic Cocoa]

I'm using cudnn version 4 and cuda version 7.5. I'll look into the discussion and update further. Thanks :)

[Mythic Cocoa]

The thread seems to have been resolved by updating cudnn. However, my cudnn is v4 and is compatible with caffe. Thoughts?

[Przemek D]

Official docs page about installation says:

cuDNN Caffe: for fastest operation Caffe is accelerated by drop-in integration of NVIDIA cuDNN. To speed up your Caffe models, install cuDNN then uncomment the USE_CUDNN := 1 flag in Makefile.config when installing Caffe. Acceleration is automatic. The current version is cuDNN v5; older versions are supported in older Caffe.

This would mean that you should upgrade to cuDNN v5 (v5.1 is currently the latest). If I remember correctly, this will require upgrading CUDA to version 8, but I advise double checking on that.
Additionally, in the thread I linked it is mentioned that error like yours can be caused by having multiple installations of the cuDNN library - might be worth checking that your system is clean.

[Mythic Cocoa]

I don't thnk that I have a new version of Caffe. I haven't updated it for a year, back when I had originally installed CUDA 7.5 and cuDNN 4. I'll try to update them, but I think I'll have to reinstall everything for that.

Also, I doubt that I have multiple cuDNN installations (because I've installed it only once). I'll check nevertheless (after I figure out how to check if my system is clean :P).

Thanks

[Przemek D]

Oh by "clean system" I meant free from multiple cuDNN/CUDA installations. Sorry for the confusion.
Also, it's good to always have the newest version of caffe. Some errors you would encounter in previous versions might get fixed in a new release, additionally if someone decides to help you with some issue they will first try to reproduce it on their own installation (which will most likely be the fresh version). So if your version and theirs don't match, it will be difficult/impossible for them to help you.