How the weight update works when back propagation?

[Tianqi Tang]

Dear All,

    I am reading the code of Backward function for layers. I find that the weight is updated as the following code, just assign the gradients to this->blobs_[0]->mutable_cpu_diff().

Is there something like "weight_new = weight_old + learn_rate*δ"？ Where is it? And during the runtime, do the gradient assignment and the weight update work together?

 // Gradient with respect to weight
    caffe_cpu_gemm<Dtype>(CblasTrans, CblasNoTrans, N_, K_, M_, (Dtype)1.,
        top_diff, bottom_data, (Dtype)1., this->blobs_[0]->mutable_cpu_diff());

Thanks,

Tianqi 

[Dana Shavit]

The update is performed several stages, first in the solver (e.g. SGDSolver.cpp) in SGDSolver<Dtype>::ComputeUpdateValue and then in Blob::Update() in Blob.cpp

D.