Finetuning specific synapses

Kai

unread,

Feb 4, 2016, 10:21:50 AM2/4/16

to Caffe Users

I am trying to finetune part of a layer. Say, one channel in layer K has 20 synapses connected to layer K-1. I want to finetune just 10 of the synapses. Is this possible or I would have to implement it? I couldn't find it anywhere in the doc. Thanks.

Evan Shelhamer

unread,

Feb 4, 2016, 1:04:26 PM2/4/16

to Kai, Caffe Users

Fine-tuning certain elements of a parameter while holding others fixed isn't currently part of Caffe. Learning rates can only be set at the layer and parameter levels so that for instance two different layers or the weights and biases of the same layer can have different learning rates.

This could be extended to having a learning rate for each element of a parameter (but that would double the parameter memory).

Evan Shelhamer

On Thu, Feb 4, 2016 at 7:21 AM, Kai <hokchh...@g.brown.edu> wrote:

I am trying to finetune part of a layer. Say, one channel in layer K has 20 synapses connected to layer K-1. I want to finetune just 10 of the synapses. Is this possible or I would have to implement it? I couldn't find it anywhere in the doc. Thanks.

--
You received this message because you are subscribed to the Google Groups "Caffe Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to caffe-users...@googlegroups.com.
To post to this group, send email to caffe...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/caffe-users/38f67e0b-04c5-47f8-ae1e-dd124733b331%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Alex Orloff

unread,

Feb 4, 2016, 4:59:09 PM2/4/16

to Caffe Users, hokchh...@g.brown.edu

Hi Evan

Thanks for your comment.

Regarding OP's question:

a) Suppose K-layer is last layer

b) Stop backpropogandition on K-1 layer

c) Train data contains information only regarding his 20 classes.

Is it not the case? What have I missed?

Thank you

Kai

unread,

Feb 4, 2016, 11:57:29 PM2/4/16

to Caffe Users, hokchh...@g.brown.edu

Hi Evan,

Thank you for your reply. If I want to implement this, would you mind pointing me to where I should start?

Thanks again.

-Kai

Kai

unread,

Feb 4, 2016, 11:59:51 PM2/4/16

to Caffe Users, hokchh...@g.brown.edu

Alex, I'm not sure what you meant, but what I'm trying to do is to finetune specific (not all) synapses connected to a neuron.

Reply all

Reply to author

Forward