Requesting an explanation of Deconvolution Layer

[Prasanna S]

My current understading

AFAIK, The deconvolution layer's forward pass works by considering the convolution to be happening in reverse order i.e. Convolve the top with the filters to produce bottom. I have seen the example gist provided from the work of FCNs for Semantic segmentation but I have the following questions.

Questions

• But how are the top layers initialized?
  
• What happens to the values at the bottom blob ( which was computed by the previous layer ) if the convolution at the deconvolution layer happens from top to bottom?
• Should the top layer be the "label image" layer only?
• What other layers can be used after a deconvolution layer to do meaningful computation?
• Where is the loss computed?
  
• Is the loss computed at the bottom of the deconv layer?
• How to incorporate multiple deconvolution layers? ( so that we can train an autoencoder directly for initializing convolution / subsampling layers )
  
• Is it possible to give a toy example / point me to a resource from which I can understand the deconvolution layer?

[Prasanna S]

A quick update:

I figured out that I need to consider the value at the bottom blob as a gradient to the the top of deconvolution layer.

The value of top is set as if it is done as a backward pass of the convolution layer.

This happens to answer most of my questions.

[Jonathan S]

Can you explain this is in more detail?

[Daniel Moodie]

I second Jonathan's request.  I'm trying to use deconv layers for creating an auto encoder, but it isn't playing nicely.

I have a 1D feature vector from my inner product layer that should be the sparse encoding of the image, however no matter the parameters I pass to deconv, I still get a 1D vector out.

[Prasanna S]

Disclaimer - This understanding is totally based on the code written in caffe and not based on any math.

What happens in this layer is the reverse of what happens in the convolution layer.

In the forward pass of the convolution layer you do a convolution operation which converts the bottom layer (b x c x w x h) input to top layer (b x d x W x H) output based on the parameters of the conv layer (say d filters of kernel size c x k x k - padding and stride as you wish ) and in the backward pass you take the gradients from top layer and pass it to the bottom layer.

In the deconvolution layer you do it in the opposite sense. The forward pass of deconv layer is more like a backward pass of the conv layer. The forward pass of the deconv layer takes the bottom layer as input and calculates the top layer just like how the gradient is backpropagated in the backward pass of the conv layer and vice versa for the backward pass of the deconv layer.

To calculate what parameters to use, consider this : Take the feature vector - say 1 x d x 1 x 1 is your output of your inner product layer. If your deconvolution layer has parameters like d filters of kernels size 3, the parameters will be of the form d x 3 x 3 x 3. So the output of the deconv layer should be 1 x 3 x 3 x 3.