Finetuning a deep fully convolutional neural network with skip connections

[Jonathan Balloch]

What is the proper procedure for finetuning a deep fully convolutional neural network with skip connections for transfering between different tasks? For a 'typical' (no skip connections, not fully convolutional) neural network, at least the last layer is "fully connected", which is to say the nodes in this layer use every node in the previous layer as input, and the number of outputs is equal to the number of class labels *M*. In [Shelhamer's original FCN paper][1] and the follow-up PAMI paper, there are details about how to implement this type of finetuning (also sometimes referred to as feature extraction); however, there aren't any details as to how it should be done with a network with skip connections like a [ResNet][2] or a [DenseNet][3]. Does anyone have any advice on best practice here?

(x-posted to Reddit and StackOverflow)

  [1]: http://www.cv-foundation.org/openaccess/content_cvpr_2015/html/Long_Fully_Convolutional_Networks_2015_CVPR_paper.html

  [2]: https://arxiv.org/abs/1512.03385

  [3]: https://arxiv.org/abs/1608.06993