How to train a single layer efficiently in C++

[filipe mutz]

Hey guys, I am developing a fine-tunning/transfer learning application by using a pre-trained net, and replacing its last inner product layer by a new layer. As I am only interested in training the last layer, I set the learning rates of all other layers to 0. The point is that even doing this the whole training process (derivatives calculation, updates, etc.) still being performed for all the layers, making the training process unnecessarily slow. How can I manually (in C++) train a single layer of the net? 

I am imagining something like this: 

1. Perform a forward pass in the net.

             

              solver->net()->Forward();

2. Perform a backward pass in the loss layer, and in my inner product layer (solver).

              int num_layers = solver->net()->layers().size();

              solver->net()->BackwardFromTo(num_layers - 1, num_layers - 2);

3. Update the weights

              // ????????????????

              Are there an elegant way of removing the parameters of the pre-trained layers from the learnable_parameters updated in the net::Update() method?

              How can I say to the solver to skip the pre-trained layers in its weight update procedure?

[filipe mutz]

The best solution I achieved so far is to create two networks, one for the pre-trained net, and another for the part of the net that I want to train. I just forward data through the the first net, then I use its output as input for the second net that I already train. This solution is seems inelegant and tricky though (I have to load two models from prototxt files, etc.).