CNN with variable length sequences

[alois....@niland.io]

Hi,

I'm trying to find a solution to handle variable length sequences in a convolutionnal neural network. I do some temporal convolutions and use Global Pooling to pool all the outputs. 

I can't think of an easy solution since the theano tensor has a fixed shape. The first idea would be to zero-pad for the shorter sequences, but the zero would be taken as input and their feature maps would be pooled in the Global Pooling.

Do you have any suggestions ?

Best,

Aloïs

[Sander Dieleman]

You could pad shorter sequences and also include a mask variable which indicates which positions in the tensor are valid, and which should be ignored. Then you could use a slightly modified global pooling layer that takes into account the mask when pooling. In practice you would probably need to have a second InputLayer for the mask variable, and a custom GlobalPoolLayer that takes two inputs (the regular input + the mask).

Sander

[alois....@niland.io]

Hi Sander,

Thank you for your answer. I tried to do it, but i'm having trouble since it is the first time i work with multiple inputs. 

Do you have any example ?

Also, would I need to create another custom layer to propagate the mask through the conv or poolings layers (the mask changes when you go through these layers) ? Or can i use the "get_output_shape" of these layers to get the new mask ?

Thank you,

Aloïs

[Sander Dieleman]

Hi Sander,

Thank you for your answer. I tried to do it, but i'm having trouble since it is the first time i work with multiple inputs. 

Do you have any example ?

A layer that takes multiple input layers should inherit from MergeLayer. You can find a bunch of examples in the Lasagne source code on GitHub by looking for 'MergeLayer'.

 

Also, would I need to create another custom layer to propagate the mask through the conv or poolings layers (the mask changes when you go through these layers) ? Or can i use the "get_output_shape" of these layers to get the new mask ?

It might be easiest to just provide the different versions of the masks that you need as separate inputs (i.e. create a separate InputLayer for each). Alternatively you could indeed write custom layers that correctly manipulate the masks. I guess the main thing you would need is something to deal with pooling (convolution shouldn't affect the mask I think?), and a regular max-pooling layer may actually be sufficient for that.

[goo...@jan-schlueter.de]

It might be easiest to just provide the different versions of the masks that you need as separate inputs (i.e. create a separate InputLayer for each).

Specifically, if you only need to mask out values at the final GlobalPoolLayer (and not at different stages of the network), it may be easier to compute the correct mask to use for that directly in numpy rather than writing layers that can propagate the mask through all the network. It should be easy to figure out the valid output length from the valid input sequence length and the sequence of convolutions and poolings, with less code than writing new layer classes.

Best, Jan

[alois....@niland.io]

Thank you both for your answers !

Stride in convolutions would also affect my mask i think ? I'll try to do it in numpy arrays as Jan suggested !

Best,

Aloïs

[alois....@niland.io]

Here are the layers :

https://gist.github.com/aloisg/ab7cf3ae77c56fa030df

with really simple tests i'd like to extend ....

https://gist.github.com/aloisg/f96761b34247b7ed1f54

Thank you for your answers !

[liang16...@gmail.com]

i also think   the stride in convolutions would affect the mask if i will execute multiply convolutions and the final pooling

在 2016年3月11日星期五 UTC+8下午4:48:40，alois...@niland.io写道：