Multi-dimensional input in LSTM

[caotia...@gmail.com]

Hello, I am new to Keras. Today I am trying to use LSTM.

I have an input array at each time step. Let's say it has a size of 2*3 at each time step. But the argument of RNN layer reads that

• input_dim: dimensionality of the input (integer). This argument (or alternatively, the keyword argument input_shape) is required when using this layer as the first layer in a model.

It seems only a vector can be used as an input at every time step, since input_dim must be an integer. Besides flattening it into 6*1, are there any other ways to deal with it? Actually I am working on images and I hope I could preserve the structure information by keeping the dimension.

Thank you!

[blackhol...@gmail.com]

Rather than specifying input_dim, you could pass input_shape=(2,3).

[caotia...@gmail.com]

Thank you very much.

But according to the documentation, it seems when you try to specify input_shape with 2D tensor, it would be (input_length, input_dim). And that parameter seems more closely related to the length of your overall sequence rather than the shape of your input at each time step. Here is an example in the document:

model = Sequential()
model.add(LSTM(32, input_shape=(10, 64)))
# now model.output_shape == (None, 32)
# note: `None` is the batch dimension.
# the following is identical:
model = Sequential() model.add(LSTM(32, input_dim=64, input_length=10))

• input_length: Length of input sequences, to be specified when it is constant. This argument is required if you are going to connect Flatten then Dense layers upstream (without it, the shape of the dense outputs cannot be computed). Note that if the recurrent layer is not the first layer in your model, you would need to specify the input length at the level of the first layer (e.g. via the input_shape argument)

在 2017年7月25日星期二 UTC-7下午6:47:03，blackhol...@gmail.com写道：

[blackhol...@gmail.com]

The input_shape parameter passed in the first layer will be the shape of the individual samples that your net is training on, not the shape/dimension of the entire sequence.

So if my entire training set was something like [ [[1,1,1], [2,2,2]], [[1,1,1],[2,2,2]], [[1,1,1],[2,2,2]] ], then the shape of my entire training set would be (3,2,3), but the shape of my individual samples would be (2,3), and so input_shape (specified in the first layer) would be (2,3).

[caotia...@gmail.com]

Thank you very much. I write a simple demo to verify our solutions.

import numpy as np
from keras.models import Sequential
from keras.layers import LSTM
from keras import optimizers


model = Sequential()
model.add(LSTM(1, input_shape=(2,3), return_sequences=True))
# data_input has a size of (2,3,2,3), i.e. samples = 2, time length = 3, shape of individual input = (2,3)
data_input = np.array([[[[1,1,1], [2,2,2]], [[1,1,1],[2,2,2]], [[1,1,1],[2,2,2]]],[[[1,1,1], [2,2,2]], [[1,1,1],[2,2,2]], [[1,1,1],[2,2,2]]]])
# data_output has a size of (2,1,2,1), i.e. samples = 2, only shows the final output, and the shape of individual output= (2,1)
data_output = np.array([[[[1],[2]],[[1],[2]],[[1],[2]]],[[[1],[2]],[[1],[2]],[[1],[2]]]])

model.compile(optimizer = 'Adam',loss='mean_squared_error')
model.fit(data_input,data_output)

And here is the error code:

Error when checking input: expected lstm_12_input to have 3 dimensions, but got array with shape (2, 3, 2, 3)

Actually, if input_shape refers to the shape of individual input, say (2,3), the total dimension should be 4, if taking samples and time steps into consideration. Other evidence would be that if you specify input_shape, you wouldn't be able to specify input_length, which may indicate their overlapping. 

But your example with input size of (3,2,3) did work. And I think this would be explained if Keras used the first 3 as the sample size, 2 as the time length, and the last 3 as the input shape.  

I am not sure if my verification was right. I would appreciate it if you could point out my mistakes. Indeed, I really hope I am wrong and Keras could well handle this. 


在 2017年7月25日星期二 UTC-7下午7:27:41，blackhol...@gmail.com写道：