Cuda libs not bundled with Caffe

[Daniel Wexler]

Why aren't the Cuda libs bundled in the .jar for Caffe?

On OSX, I have to install the CUDA libs, libcudart.7.5.dylib, libcublas.7.5.dylib, and libcurand.7.5.dylib into /usr/local/lib to use Caffe via JavaCPP. This is even true if I force Caffe to CPU-only mode. 

I tried putting them into another directory and setting java.library.path to point at the new dir, but that fails as well. I assume that's due to how the loading works in JavaCPP?

This is not an issue for development, but for distribution, it is a PITA, since I don't want to have to install libs into /usr/local/lib for my customers.

Thanks,

Dan

[Samuel Audet]

That's something Cyprien also mentioned was problematic. There doesn't
appear to be any problems with the license of CUDA itself that would
prevent us to distribute the binaries, actually. Cyprien knows who we
could contact at NVIDIA to iron out the details.

However, there is one technical aspect that worries me and that still
needs to be investigated. At load time, CUDA needs libcuda.so/dylib
(nvcuda.dll is the equivalent on Windows I believe) and that comes with
the video driver, not CUDA. So, the question is, can we bundle that? Is
it generic enough across versions of the video driver? My guess is that
it isn't, so we'd still be stuck with the same issue... If you find any
answers concerning that, please let us know!

Another option would be to modify Caffe itself so that when it runs in
CPU mode, that it doesn't try to load the CUDA binaries.

Yet another option is to provide a set of two binaries: one compiled
with CUDA, and one without. If the one compiled with CUDA fails to load,
then we try to load the CPU-only version. That's probably easier to get
done, but requires more disk space and more build time, something that
I'm not ready to provide until we get a build server somewhere...

Have you had any more thoughts about this recently, Cyprien?

The path issue on Mac OS X is because it has a weird way of doing rpath,
and the Caffe build would need to be adjusted for that. Check for
example the patch for FFmpeg:
https://github.com/bytedeco/javacpp-presets/blob/master/ffmpeg/ffmpeg-3.0-macosx.patch

Samuel

[Daniel Wexler]

Great info. I can work on the @rpath patch, either during linking or using install_name_tool. I've done that for Caffe before I started using the JavaCPP version.

Thanks, Samuel!

[Cyprien Noel]

I haven't looked recently. I remember NVIDIA was fine with the licensing, at least on Linux. I have tried different versions of CUDA and the driver, and they were working fine. Maybe looking in the CUDA requirements for the supported driver versions would give more info. We could have a hardcoded list in javacpp that would not start if the driver is too old. Let me send you the contact for the NVIDIA guy for the licensing question.

[Samuel Audet]

Thanks for the input, Cyprien!

The problem isn't about the version of the video driver, though, the
problem is what to do when there isn't any driver installed. When CUDA
can't find libcuda.so/dylib or nvcuda.dll, it refuses to load at all,
period. How should we deal with boxes that do not have a GPU from NVIDIA?

Samuel

[Samuel Audet]

That would be awesome! Please send a pull request with your patch when
it's done! Thanks :)

Samuel