Way to Convolutional Autoencoder

[Jiaxu Zhu]

I find there is only one step (unpooling layer) to use caffe for a convolutional autoencoder , since we already have deconv layer.

And I find that there is already an implementation of unpooling layer on this modified caffe https://github.com/HyeonwooNoh/caffe. the only problem is the version of this caffe is really old.

So what I need to do is move the unpooling layer implementation to the latest caffe.

Is there any better way to use caffe for a convolutional autoencoder?

Thanks!

[taras sereda]

Hi, I always was looking for convolutional autoencoder in caffe, but also I've found only deconv layer.

Take a look at this repo and blog post. There is conv autoencoder implemented in theano.

https://swarbrickjones.wordpress.com/2015/04/29/convolutional-autoencoders-in-pythontheanolasagne/

https://github.com/mikesj-public/convolutional_autoencoder/tree/master

[Mohammad Babaeizadeh]

There is also another implementation here:
https://github.com/HyeonwooNoh/caffe/tree/master/src/caffe

[Oluwatobi Olabiyi]

Hi, You can use deconvolution layer for your unpooling ... e.g. 

Assuming your pooling layer is below:

layer {

  name: "pool1"

  type: "Pooling"

  bottom: "conv1"

  top: "pool1"

  pooling_param {

    pool: MAX

    kernel_size: 5

    stride: 2

  }

}

You can unpool as follows:

layer {

  name: "unpool1" 

  type: "Deconvolution"

  bottom: "pool1" 

  top: "unpool1"

  convolution_param {

    kernel_size: 5

    stride: 2

    num_output: 3 

    group: 3

    # pad: 0

    weight_filler: { type: "bilinear" } 

    bias_term: false

  }

  param { 

    lr_mult: 0 

    decay_mult: 0 

  }

}

[Oluwatobi Olabiyi]

I edited a bit to ensure downsampling and upsampling match is ensured. Note that you might have to play with the padding in the deconvolution to match upsampling and downsampling rate.

On Tuesday, November 10, 2015 at 10:46:39 AM UTC-8, Oluwatobi Olabiyi wrote:

Hi, You can use deconvolution layer for your unpooling ... e.g. 

Assuming your pooling layer is below:

layer {

  name: "pool1"

  type: "Pooling"

  bottom: "conv1"

  top: "pool1"

  pooling_param {

    pool: MAX

    kernel_size: 5

    stride: 3

  }

}

You can unpool as follows:

layer {

  name: "unpool1" 

  type: "Deconvolution"

  bottom: "pool1" 

  top: "unpool1"

  convolution_param {

    kernel_size: 5

    stride: 3

    num_output: 3 

    group: 3

    # pad: 0/1

    weight_filler: { type: "bilinear" } 

    bias_term: false

  }

  param { 

    lr_mult: 0 

    decay_mult: 0 

  }

}

[Jiaxu Zhu]

Could you please explain the meaning of

num_output: 3 

group: 3

?

What's more, what unpooling doing is give the max value to all in its kernel right?

Then what if there's overlap between kernel?

[JN]

Hi all,

I'm also interested in unpooling. Your example is excellent, but (correct me if I am wrong) I think it differs from the implementation of Noh et al (https://github.com/HyeonwooNoh/caffe) in that this example is just upsampling by interpolation, they actually store the location of the max from the pool layer and re-use when unpooling.

Any news whether "advanced" un-pooling is making its way into mainstream Caffe?

Thanks

Jonas

[Volodymyr Turchenko]

Hi guys,

I have published our solution of CAE, but still without pooling/unpooling layers.

https://groups.google.com/forum/#!topic/caffe-users/GhrCtONcRxY

I would like to say that the architecture without pooling/unpooling is definitely working architecture.
I would like to ask if someone can comment the approach we have used to calculate the number of trainable parameters (see details in Table 3 in the paper) in encoder and decoder parts.
Thanks!

Cheers,
Vlad

середа, 25 листопада 2015 р. 02:56:44 UTC-7 користувач JN написав:

[Philipp H]

Hey,

do you know how to use the Deconvolution-Layer for N-D blob unpooling?

Usually one has dimensions of: channels x width x height, but what if I have channels x depth x height x width?