How to specify locally-connected layer with unshared weights for a convolutional layer.

[Zizhou Liu]

Hi guys,

I'm new to Torch.

I want to create locally-connected layer with unshared weights for a convolutional layer.

This kind of layer is just like a convolutional layer, but without any weight-sharing. That is to say, a different set of filters is applied at every (x, y) location in the input image. Aside from that, it behaves exactly as a convolutional layer.

Thanks,

Zizhou

[soumith]

Then, create a layer with the size of the convolution filter equal to the size of the input.

--
You received this message because you are subscribed to the Google Groups "torch7" group.
To unsubscribe from this group and stop receiving emails from it, send an email to torch7+un...@googlegroups.com.
To post to this group, send email to tor...@googlegroups.com.
Visit this group at http://groups.google.com/group/torch7.
For more options, visit https://groups.google.com/groups/opt_out.

[Zizhou Liu]

I don't think this is how convolutional network works.

On Wednesday, February 5, 2014 4:22:09 PM UTC, smth chntla wrote:

Then, create a layer with the size of the convolution filter equal to the size of the input.

[soumith]

are you sure? do the operation on paper and check.

[Zizhou Liu]

Thanks for the reply. 

In my understanding, the unshared convolutional layer is: (let's say there are 1 feature map (5x5) as input, 10 filters with size 3x3)

For each pixel location on the feature map, there's a 3x3 filter. And the weight of the filter is unshared means different filter would be applied on different pixel location.

In conventional convolutional neural network, it would be 1 shared filter for the feature map. And 10 filters would produce 10 output feature maps.

And in unshared network, it would provide (5-2) x (5-2) x 10 filters with size 3x3 in total.

Correct me if I'm wrong.

Thanks,

Zizhou

On Thursday, February 6, 2014 3:15:09 PM UTC, smth chntla wrote:

are you sure? do the operation on paper and check.

[soumith]

Okay, I was looking at the special case stride==size. 

There is no module in torch that currently does what you want.

--
S

On Thu, Feb 6, 2014 at 10:15 AM, Zizhou Liu via torch7 <torch7+noreply-APn2wQdEDl7Cq1gVZ...@googlegroups.com> wrote:

Thanks for the reply. 

In my understanding, the unshared convolutional layer is: (let's say there are 1 feature map (5x5) as input, 10 filters with size 3x3)

For each pixel location on the feature map, there's a 3x3 filter. And the weight of the filter is unshared means different filter would be applied on different pixel location.

In conventional convolutional neural network, it would be 1 shared filter for the feature map. And 10 filters would produce 10 output feature maps.

And in unshared network, it would provide (5-2) x (5-2) x 10 filters with size 3x3 in total.

Correct me if I'm wrong.

Thanks,

Zizhou

On Wednesday, February 5, 2014 4:22:09 PM UTC, smth chntla wrote:

[Zizhou Liu]

Thanks.

I will do more reading on Torch and see if it's easy to implement this.

Zizhou

On Thursday, February 6, 2014 5:19:53 PM UTC, smth chntla wrote:

Okay, I was looking at the special case stride==size. 

There is no module in torch that currently does what you want.

--
S

[Zizhou Liu]

Is there are way to specify customised connections between layers or restrict weights for a multilayer perceptron network. I can see I can use it as a unshared convolutional layer. 

Cheers, 

Zizhou

On Thursday, February 6, 2014 5:19:53 PM UTC, smth chntla wrote:

Okay, I was looking at the special case stride==size. 

There is no module in torch that currently does what you want.

--
S

[soumith]

customised connections between layers: SpatialConvolutionMap

[Zizhou Liu]

Can you elaborate how to config it for a unshared convolutional layer?  I don't know it is capable of this.

From this: https://groups.google.com/forum/#!topic/torch7/7nYl4yL_Ta4

I can see, Torch don't have this function. 

On Friday, February 7, 2014 3:59:27 PM UTC, smth chntla wrote:

customised connections between layers: SpatialConvolutionMap

[soumith]

unshared convolutional layer: mo module can do that right now in torch, already mentioned that to you.

[Oliver Mattos]

To update this old thread with current state:

Other frameworks have this feature:  
https://code.google.com/p/cuda-convnet/wiki/LayerParams#Locally-connected_layer_with_unshared_weights

It is used by the facebook deepface paper: http://www.cs.toronto.edu/~ranzato/publications/taigman_cvpr14.pdf

Implementation isnt trivial, and as far as I can see currently isn't possible in torch7 without extra work.

On Friday, February 7, 2014 at 4:22:04 PM UTC, smth chntla wrote:

unshared convolutional layer: mo module can do that right now in torch, already mentioned that to you.

[soumith]

Hey Oliver,

This is provided in my port of cuda-convnet2 for torch. https://github.com/soumith/cuda-convnet2.torch

It is very easy to use within torch, as all the ccn2 layers are wrapped as nn modules.

For more options, visit https://groups.google.com/d/optout.

[jie zhang]

Hi Zizhou,

How are you?

Do you eventually find the way to implement this?

I currently face the same problem.

Please let me know if you have any insights.

Thanks

在 2014年2月5日星期三 UTC-5上午6:05:01，Zizhou Liu写道：

[Francisco Vitor Suzano Massa]

You can use nn.SpatialConvolutionLocal, which implements exactly what you are looking for

[jie zhang]

Hi Fmassa, 

I updated my torch through update.sh. However I still could not find the nn.SpatialConvolutionLocal in my nn package.

Do I need to reinstall the torch? 

Thanks

在 2016年2月28日星期日 UTC-5下午6:32:15，Francisco Vitor Suzano Massa写道：

[soumith]

./clean.sh

./update.sh

./install.sh

--

You received this message because you are subscribed to the Google Groups "torch7" group.
To unsubscribe from this group and stop receiving emails from it, send an email to torch7+un...@googlegroups.com.
To post to this group, send email to tor...@googlegroups.com.

Visit this group at https://groups.google.com/group/torch7.

[jie zhang]

Thank you very much.

It works now!

在 2016年3月30日星期三 UTC-4上午12:13:08，smth chntla写道：

./clean.sh

./update.sh

./install.sh

[Chen-Ping Yu]

Hi Jie,

nn.SpatialConvolutionLocal does work, but is extremely slow (slower than CPU by far), is that happening to you too? If so, how do you get around that problem?