does classification example use gpu?

[Mo Ki]

This might be an stupid question but please bare with me as I am new to Caffe. Anyways In classification example the image is copied in the data blob using WrapInputLayer() function which is calling mutable_cpu_data() does it mean its using CPU instead of GPU? I tried changing it to mutable_gpu_data() but it causes an exception. How can I make sure the GPU is used?

[Przemek D]

You cannot directly copy between CPU and GPU, because GPU memory is not directly accessible by dereferencing a pointer - there are dedicated CUDA functions for copying data between CPU (host) and GPU (device). Fortunately, you don't have to do this manually. Caffe's implementation of a Blob class handles CPU/GPU memory transfers for you. Whenever a layer is executed on a GPU, data is copied from the host memory (unless it was already on the device - Caffe avoids unnecessary copies and allocations); whenever it is needed back on the CPU - a copy is performed.

Caffe will, by default, attempt to run on the GPU (unless you're using Python - then you have to set_device and set_mode_gpu). You can see that your network runs on the GPU by the command nvidia-smi (in terminal) - it should show memory allocated on the GPU and its utilization.

[David Sorber]

I have essentially the same question as the original poster, what modifications are needed to make the classification example run on a GPU?  I have tried setting the mode and device but my GPU remains idle and everything clearly runs on the CPU.  I have CUDA (version 8) installed and Caffe was built with CUDA support.

The WrapInputLayer member function specifically calls mutable_cpu_data(), is this correct or does it need to be changed (see below)?

void Classifier::WrapInputLayer(std::vector<cv::Mat>* input_channels)

{

    Blob<float>* input_layer = net_->input_blobs()[0];

    int width = input_layer->width();

    int height = input_layer->height();

    float* input_data = input_layer->mutable_cpu_data();

    for (int i = 0; i < input_layer->channels(); ++i)

    {

        cv::Mat channel(height, width, CV_32FC1, input_data);

        input_channels->push_back(channel);

        input_data += width * height;

    }

}

There doesn't appear to be any useful debug information explaining why the GPU is not being used, unless I'm missing something.  Seems to me if I explicitly set the mode to GPU it should either run on the GPU or error out with some useful error message.

-Dave

[Przemek D]

As I said above, rereferencing a pointer to GPU memory in CPU code is an error -- you cannot change to mutable_gpu_data.

There shouldn't be any additional actions needed to run classification on the GPU, because if you compiled with GPU support this line should run and make the whole framework use GPU.

What makes you so certain that everything runs on the CPU? How did you check it? Have you tried running caffe time and compare with caffe time -gpu 0?