C++ geting the net backwards gradients

[oranjee...@protonmail.com]

Hello

I just recently started off with Caffe I've trained my first own networks and now I want to play around more deeply with it.

I just was wondering what is the easiest method of the C++ equivalent in python of getting the backward gradients of the whole neural network. I've seen a lot of Python examples on the net using following code:

# Do backpropagation to calculate the gradient for that outcome
gradient = net.backward(prob=probs)

Where gradient I guess is a Blob.

I am not a big Python code and I prefer to use C++ with which I am far more familiar with. however I checked the C++ Api documentation.

But I realized that the Net::backward method is a void function so it doesn't return a Blob<DataType>.

I guess I can probably change the probs as in the python code above if I want by changing the Net::output_blobs() via the Blob<DataType>::set_cpu_data. and then call  (do I need to call Blob<DataType>::Update afterwards?)

and then call Net::backwards afterwards

But how do I get the equivalent gradients as in this python call?

I checked the source code of the python wrapper but this didn't lead to much either as it seems to just call the C++ function which is weird, same for the matlab/mex wrapper so I wonder how this actually works at all?

Do I have to call Net::bottom_vecs() ?

Any hints would be greatily appreciated thank you!

[oranjee...@protonmail.com]

Well this seems to be very puzzling. 

I tried now several things but basically the gradients (or diffs) in all the layer blobs after calling Net::forward() and then Net::backward() are zero. 

This clearly can't be right. I even activated the debug_info to see what is happening in the backward pass and it shows the same. But I have set force_backward: true in my deploy.prototext and when I check all layers except for the first one get the need_backward flag set as well.

So what am I doing wrong?

[oranjee...@protonmail.com]

Just in case someone is interested in this, I finally found out what I was doing wrong. I was really close but made one slight mistake.

So what you need to do is after the forward pass, get the blob of the output layer, assuming you have a fully connected network with a softMax final layer

Blob<float>* output_layer = net_->output_blobs()[0];

float* outData = output_layer->mutable_cpu_diff();

Then you simply need to set gradients / diff of the class you're interested in to 1 and all others to zero but leave the normal data intact.

 for (int i = 0;  i< output_layer->num() * output_layer->channels() * output_layer->height() * output_layer->width(); i++) 

    outData[i] = 0.0f;

outData[your_class] = 1.0f

and then call 

Net::backwards()