what happens when one changes 'num_output' on the last 'fc' layer?

[Carlo Alessi]

Hello,

I fine-tuned the Alexnet Caffe model. I did not change the name of the last fc8, and I restarted backpropagation using 3 classes [say Banana, Apple, Orange] (2 of which were already trained, but the last one was new). 

The last 'fc' layer had 1000 neurons but now I set them to be three. 

1) Does it select the first three? If so I might have trained a neuron with Banana images that was originally trained to recognize cars.

2) Can I backpropagate on the original Banana and Apple neurons and add the new Orange neuron?

Otherwise the only way to finetune would be to restart the last layers from scratch.

3) What are the rules of thumb to finetune with just 1300 images? (which layers to finetune, learning rate, learning rate drop, ...)