Caffe and CUDA: What does the dependency Mean?

[SRQ]

Hello,

I am a CUDA/Caffe virgin trying to get started with these tools and will be working with them for a couple of years hopefully. I will get the TK1 and TX1 in a few days and set up the practical environment. I have read the manuals but there are a few questions I have which are still unresolved. Any help will be appreciated.

1. I understand that Caffe is a framework for implementing Convnets using python or C++. I have also seen that while setting it up, we need to install CUDA as its dependency. Now, CUDA is a parallelization framework that helps us write code in such a way that it can utilize multiple GPU cores to make the program or algorithm run faster. While reading guides, I have found some that explain how to write code for caffe while others explain how to write code for CUDA. Since CUDA is a dependency for Caffe, is any code written in Caffe automatically using CUDA to better its running time?

2. If answer to question 1 is yes, then what does writing code directly in CUDA mean? Or this will only be done only when we are not using Caffe but only CUDA directly with a GPU?

3. If answer to question 1 is no, do we need to follow any specific guidelines while writing code (say C++) in Caffe?

4. Is there a scenario where I use the TK1/TX1 and use CUDA directly without setting up Caffe? Or does the Caffe GPU mode does the same thing?

I know some of these questions may seem less than intelligent, so please don't throw me under the bus directly.

Regards
Q

[Jan]

See interleaved answers.

Am Dienstag, 22. März 2016 09:48:27 UTC+1 schrieb SRQ:

Hello,

I am a CUDA/Caffe virgin trying to get started with these tools and will be working with them for a couple of years hopefully. I will get the TK1 and TX1 in a few days and set up the practical environment. I have read the manuals but there are a few questions I have which are still unresolved. Any help will be appreciated.

1. I understand that Caffe is a framework for implementing Convnets using python or C++. I have also seen that while setting it up, we need to install CUDA as its dependency. Now, CUDA is a parallelization framework that helps us write code in such a way that it can utilize multiple GPU cores to make the program or algorithm run faster. While reading guides, I have found some that explain how to write code for caffe while others explain how to write code for CUDA. Since CUDA is a dependency for Caffe, is any code written in Caffe automatically using CUDA to better its running time?

I'd say Theano for instance is "a framework for implementing Convnets using python", whereas I'd call caffe a "convnet implementation in C++". Just to be very precise: you don't strictly need to write code to use caffe (other than the configuration files to specify your network), but a little extension here and some python code over there is more or less indispensable if your task differs from the classic object recognition examples given. Yes, there is lots of CUDA code in caffe, every layer that makes sense to implement in CUDA _is_ also implemented in CUDA. Memory is stored and transferred between RAM and VRAM transparently to the user, which means you can switch between CPU and GPU operation with only a single config switch, and leave the rest to caffe. The only thing you might need to think about is the size of your VRAM, which can give you problems if your batchsize and/or network is too large.
 

2. If answer to question 1 is yes, then what does writing code directly in CUDA mean? Or this will only be done only when we are not using Caffe but only CUDA directly with a GPU?

You don't have to write CUDA code for using caffe. The only thinkable case is if you need a layer type that is not implemented yet and you need to implement it yourself. CUDA code is similar to C, but incorporating the parallelization, which can be very tricky to do efficiently. If you ever need to write CUDA code for caffe (which is unlikely) take a look at the CUDA implementations of the existing layers (filename follows the layer_name.cu pattern).

3. If answer to question 1 is no, do we need to follow any specific guidelines while writing code (say C++) in Caffe?

 

"Writing C++ code in caffe" is a strange statement. If by that you mean "extend caffe's code with your own code" then the answer is "depends". If you are interfering with the CUDA stuff then of course you need to take special care, if you're not, then you don't.
 

4. Is there a scenario where I use the TK1/TX1 and use CUDA directly without setting up Caffe? Or does the Caffe GPU mode does the same thing?

Similarly: Of course you can use CUDA without using caffe. CUDA is a framework to execute code on the GPU, and caffe just happens to make use of that. So the caffe GPU mode is basically a CUDA implementation of all the available layers. Again, if your goal is just training networks you probably don't ever need to touch the CUDA stuff itself.

Hint: you need the proprietary NVidia driver to run CUDA code, and hence caffe. If you're on a Ubuntu distro, the easiest way is to use the nvidia-provided ubuntu repo to install the cuda framework and driver (go to https://developer.nvidia.com/cuda-downloads and select deb(network) in the last step, follow the instructions).

I know some of these questions may seem less than intelligent, so please don't throw me under the bus directly.

Regards
Q

Jan

[SRQ]

Thanks Jan for your detailed response. Your answer tells me that I only need to implement the structure of convnet in Caffe and not worry about CUDA at all as the code is being taken care of at the backend. Since the algo guys have already done there job, I don't think I will have to do any work related to algo. My network is actually pretrained and I only have to implement it to get familiar with the technology. The final goal is to take this to FPGA for trying to optimize the hardware. In that case, according to what I understand, CUDA will not be useful as it is meant only for Nvidia GPUs and I will need to do everything again in OpenCL. Is the Caffe OpenCL version mature for all regular tasks? I understand that my question is not exactly valid as I am using TK1/TX1 which do not support the OpenCL drivers anyway.

[Jan]

Basically yes.

OpenCL: I have no idea about that, didn't even know there was a caffe version using OpenCL instead of CUDA. But the TK1 and TX1 do support CUDA. And as long as you can use CUDA, I'd never use OpenCL (CUDA is usually faster). So I see no reason for you to look into OpenCL...

Jan