Error in LSTM layer caused by batch size reshape in Python

[Moritz]

Hey,

I'm currently working on RNN with LSTM layers. I implemented a simple RNN with a single LSTM layer and adjacent fully connected layers etc. Training and testing works fine.

However, I would like to vary the batch size for testing purposes directly in Python. After initializing the net and loading the caffemodel file with the trained weights, I reshape the input blobs 'data' and 'clip' (my sequence marker).

deploy_net.blobs['data'].reshape(batch_size, stream, channels, height, width)
deploy_net.blobs['label'].reshape(batch_size, label_dim)

Afterwards, an error occurs when I want to reshape the whole net or compute the forward pass (in this example from 100 to 80).

F1010 16:08:49.571087  5596 recurrent_layer.cpp:187] Check failed: T_ == bottom[0]->shape(0) (100 vs. 80) input number of timesteps changed

I assume that is due to the unrolled implementation of recurrent layers in Caffe. Is there a way to newly setup the LSTM layer in Python after changing the batch size of its input blobs?

Thanks in advance,

Moritz

[auro tripathy]

You'll have to look at documentation that goes along with the LSTM PR, https://github.com/BVLC/caffe/pull/2033

The shape is time-step x batch-size x (channel x width x height)

You can vary the time-step or the batch-size but specify it in a 'counter-intuitive' way

-Auro

[Moritz]

Hey Auro,

Thanks for your reply. You are right, I stated it wrong in the post above. What I did was:

deploy_net.blobs['data'].reshape(timesteps, streams, channels, height, width)
deploy_net.blobs['label'].reshape(timesteps, streams, label_dim)

Loading and training the net works fine. Anyway, it is not possible for me to change the dimensions and reshape the net in Python after I have loaded the net from a prototxt.

I think it is because during net setup Caffe unrolls the net in order to handle the recurrent layer for the given number of timesteps. Is there a way to repeat this setup process manually in Python after changing the dimensions?

Best regards,

Moritz

[Biswa G Singh]

Hi,

I also get the same issue in reshape for testing

  vector<int> shape(2, 1);

  test_data_blob->Reshape(shape);

  test_clip_blob->Reshape(shape);

  test_net->Reshape();

The timesteps is actually 320. With this I get the dim mismatch error  

Check failed: T_ == bottom[0]->shape(0) (320 vs. 2) input number of timesteps changed

Do I have to use a reshape layer here?

Thanks,

Biswa

[Kevin Wang]

Hey Auro,

Could you please clarify a little bit how to vary time-step in a "counter-intuitive" way? I am having the same problem right now.

Thanks!

[Kevin Wang]

I actually fix this by adding another prototxt, i.e. one for training and one for testing. You can have different time-step in testing even if you are loading from the same caffemodel from training.

Maybe Moritz is right. I can not change the length of time-step in python interface. Just to clarify, I am also including my toy example for training and testing (train.prototxt and deploy.prototxt). Please correct me if I am wrong.

Thanks,

Kevin