LSTM model does not accept data

[trumee]

Hello,

I am trying to setup an LSTM model but having trouble to get it working. I guess something is incorrect with the data reshape or the LSTM inputs. Any idea what could be wrong?

import numpy as np
from keras.models import Sequential
from keras.layers.core import Dense, Activation
from keras.layers import LSTM
from keras.utils import plot_model
import random

def create_model_lstm(X_train, Y_train):
    batch_size = 2
    model  =  Sequential()
    model.add(LSTM(5, batch_input_shape = (batch_size, X_train.shape[1], X_train.shape[2] )))
    model.add(Dense(1))
    model.compile(loss = 'mean_squared_error', optimizer = 'adam')
    return(model)


X_train_m = np.random.rand(4,8)
Y_train_m = np.random.rand(4,1)

print('Original data shape:')
print(X_train_m.shape,Y_train_m.shape)


X_train_m = np.reshape(X_train_m, ( X_train_m.shape[0],1, X_train_m.shape[1]))
Y_train_m = np.reshape(Y_train_m, ( Y_train_m.shape[0],1, Y_train_m.shape[1]))

print('After reshape:')
print(X_train_m.shape,Y_train_m.shape)


model = create_model_lstm(X_train_m, Y_train_m)
print(model.summary())

model.fit(X_train_m, Y_train_m, epochs = 100, batch_size = 1, verbose = 3)Enter code here...

Running the code gives me this:

Using TensorFlow backend.
Original data shape:
(4, 8) (4, 1)
After reshape:
(4, 1, 8) (4, 1, 1)
_________________________________________________________________
Layer (type)                 Output Shape              Param #  
=================================================================
lstm_1 (LSTM)                (2, 5)                    280      
_________________________________________________________________
dense_1 (Dense)              (2, 1)                    6        
=================================================================
Total params: 286
Trainable params: 286
Non-trainable params: 0
_________________________________________________________________
None
Traceback (most recent call last):
  File "lstm4.py", line 34, in <module>
    model.fit(X_train_m, Y_train_m, epochs = 100, batch_size = 1, verbose = 3)
  File "tensorflow/lib/python3.4/site-packages/keras/models.py", line 856, in fit
    initial_epoch=initial_epoch)
  File "tensorflow/lib/python3.4/site-packages/keras/engine/training.py", line 1429, in fit
    batch_size=batch_size)
  File "tensorflow/lib/python3.4/site-packages/keras/engine/training.py", line 1309, in _standardize_user_data
    exception_prefix='target')
  File "tensorflow/lib/python3.4/site-packages/keras/engine/training.py", line 127, in _standardize_input_data
    str(array.shape))
ValueError: Error when checking target: expected dense_1 to have 2 dimensions, but got array with shape (4, 1, 1)

[김태영]

I think that batch_size is problem. How about changing batch_size from 1 to 2 in fit(), because you set batch_size as 2 in create_model_lstm()?

2017년 6월 19일 월요일 오후 12시 10분 45초 UTC+9, trumee 님의 말:

[trumee]

On Monday, 19 June 2017 12:31:44 UTC-5, 김태영 wrote:

I think that batch_size is problem. How about changing batch_size from 1 to 2 in fit(), because you set batch_size as 2 in create_model_lstm()?

Unfortunately, the error still comes up after this change.

model.fit(X_train_m, Y_train_m, epochs = 100, batch_size = 2, verbose = 3)

Original data shape:
(4, 8) (4, 1)
After reshape:
(4, 1, 8) (4, 1, 1)
_________________________________________________________________
Layer (type)                 Output Shape              Param #  
=================================================================
lstm_1 (LSTM)                (2, 5)                    280      
_________________________________________________________________
dense_1 (Dense)              (2, 1)                    6        
=================================================================
Total params: 286
Trainable params: 286
Non-trainable params: 0
_________________________________________________________________
None
Traceback (most recent call last):
  File "lstm4.py", line 34, in <module>

    model.fit(X_train_m, Y_train_m, epochs = 100, batch_size = 2, verbose = 3)

[김태영]

How about changing first dimension value of X_train_m and Y_train_m f from 4 to 2?

2017년 6월 19일 월요일 오후 12시 10분 45초 UTC+9, trumee 님의 말:

Hello,

[trumee]

Here is the output

Original data shape:
(2, 8) (2, 1)
After reshape:
(2, 1, 8) (2, 1, 1)

Layer (type)                 Output Shape              Param #  
=================================================================
lstm_1 (LSTM)                (2, 5)                    280      
_________________________________________________________________
dense_1 (Dense)              (2, 1)                    6        
=================================================================
Total params: 286
Trainable params: 286
Non-trainable params: 0
_________________________________________________________________
None
Traceback (most recent call last):
  File "lstm4.py", line 34, in <module>

    model.fit(X_train_m, Y_train_m, epochs = 100, batch_size = 2, verbose = 3)

  File "tensorflow/lib/python3.4/site-packages/keras/models.py", line 856, in fit
    initial_epoch=initial_epoch)
  File "tensorflow/lib/python3.4/site-packages/keras/engine/training.py", line 1429, in fit
    batch_size=batch_size)
  File "tensorflow/lib/python3.4/site-packages/keras/engine/training.py", line 1309, in _standardize_user_data
    exception_prefix='target')
  File "tensorflow/lib/python3.4/site-packages/keras/engine/training.py", line 127, in _standardize_input_data
    str(array.shape))

ValueError: Error when checking target: expected dense_1 to have 2 dimensions, but got array with shape (2, 1, 1)

[김태영]

I changed the code as the following :

import numpy as np

from keras.models import Sequential

from keras.layers.core import Dense, Activation

from keras.layers import LSTM

from keras.utils import plot_model

import random

# samples : 4

# time steps : 1

# features : 8

def create_model_lstm(X_train, Y_train):

    batch_size = 2

    model  =  Sequential()

    model.add(LSTM(5, batch_input_shape = (batch_size, X_train.shape[1], X_train.shape[2] )))

    model.add(Dense(1))

    model.compile(loss = 'mean_squared_error', optimizer = 'adam')

    return(model)

X_train_m = np.random.rand(4, 8)

Y_train_m = np.random.rand(4, 1)

print('Original data shape:') 

print(X_train_m.shape,Y_train_m.shape)

X_train_m = np.reshape(X_train_m, (4, 1, 8))

Y_train_m = np.reshape(Y_train_m, (4, 1))

print('After reshape:') 

print(X_train_m.shape,Y_train_m.shape)

model = create_model_lstm(X_train_m, Y_train_m)

print(model.summary())

model.fit(X_train_m, Y_train_m, epochs = 100, batch_size = 2, verbose = 3)

How about this?

2017년 6월 20일 화요일 오전 9시 49분 16초 UTC+9, trumee 님의 말:

[trumee]

Thanks your code worked. The change which made it work was

Y_train_m = np.reshape(Y_train_m, (4, 1))

Any idea why this a 2D tuple is needed rather than 3D?  I thought LSTM needed an (samples, timesteps, features) as an input.

[adam....@gmail.com]

It sounds good. X_train is data and Y_train is label. So, if you want to resolve many to one problem, Y_train doesn't need 'timesteps'. If you want to resolve many to many problem, you have to use return_sequence option in LSTM layer.

[lalp...@gmail.com]

Wow this worked. Thanks!!