LSTM model

[Denis Dal Soler]

Hi 

can someone please tell me  where I can find a graphical expletation of the architetcture build by under code sentences?

My dubts are:

1) What does "128" mean? Does It mean 128 lstm cells (forget state+input state+output state) or somethings else?

2) Area these 128 cells  in series and every one of these have a Xn input extract from the time serie??

3) But if the sentence before is true why I have 128 cells (therefore 128 Xn extract from timeserie) and I set the input_shape=(Look_back,1)??Shouldn't they be the same value??

4) Finally I have a very big lake!

I can't figurate how this serie of lstm cells update the weight with backpropagation.
I have clear how the backpropagation operates in a simple neural network (Es. n inputs with n weights,  only one neuron with a loss function) but I can't figurate how it oparate in a lstm cell.

 Thanks for you help.

Denis

model = Sequential()
model.add(LSTM(128, return_sequences=True, input_shape=(look_back, 1)))
model.add(Dropout(0.2)) 
model.add(Dense(25))
model.add(Dense(1))

[Denis Dal Soler]

Hi, 

for better explain my request I attached a picture.

Can someone please explain me what mean (what does it correspond) in the picture attacched  "128" and "input_shape" from the instructions below ?

model = Sequential()
model.add(LSTM(128, return_sequences=True, input_shape=(look_back, 1)))

Thanks very much.

Denis

[Samer Attrah]

Hello,
according to the image you have uploaded 128 is the number of the green and yellow pairs in one of the layers since in code you are showing one layer and in the image there are two.
and the input shape is the number of horizontal blue arrows on the left of the image.

I hope that helps
Kind regards
Samer

[Denis Dal Soler]

Dear Samir

thankyou for your reply.

But if it's as you tell me why the number of the green and yellow pairs (128) is different from input shape??

If is there  one input (blu arrow)  for each cell  (green+yellow pairs) why the number is different?

This is my dubt.

Thanks

Denis

[Samer Attrah]

you are welcome,
as I see it the number of blue arrows is not different, since in layer one each cell is getting one blue horizontal arrow, 

and there is of course the purple arrows, which represent the time dependence the layer is also considering.
the vertical blue arrows as I see them represent the continuation from h2 to ht which is the total number of cells.

and besides that if you are discussing more generally, the number of cells is an integer that you can choose and change according to quality design standards.
while the number of blue arrows is fixed by the dataset you are using and sometimes you have no control on it.
I have included below a drawing I made to explain my idea.

I hope that helps

Kind regards
Samer

[Praveen Hosdrug]

Hi Denis

I wanted to explain the LSTM architecture and its backward propagation mechanism based on the diagram shared.

The LSTM architecture consists of two essential memory components:

Long-term memory (cell state c_t)
Short-term memory (hidden state h_t)
The diagram shows:

Time steps T-2, T-1, and T
Forward pass (black arrows) propagating information forward
Backward pass (red arrows) computing gradients backwards

The Gradient Path is different as compared to a simple neural network, as the gradient flow occurs through the LSTM can be divided into two paths one for the long term memory unit and the other short term memory unit.

 Long-term path: Linear path through cell states and the Short-term: Branching path through multiple gates of the hidden states.

Have a look at Figure 10.9 Going from a SimpleRNN to an LSTM: adding a carry track from chapter 10 on deep learning with Python second edition where it is explained in further detail. Also if this helps!

Warm Regards,

Praveen Hosdrug

--
You received this message because you are subscribed to the Google Groups "Keras-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to keras-users...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/keras-users/daeaace7-c459-4276-91aa-65fc1b9b0257n%40googlegroups.com.

[Denis Dal Soler]

Dear Praveen 

thanks for your reply.

I've clear enought the forward e back propagation.

What isn't clear for me is how the sequence extract from a time serie is processed in the lstm architecture.

Thanks

D

[Praveen Hosdrug]

Ahh I see, My understanding is that there are two ways of sending in a sequence from a time series. The LSTM layer where you send the 3D tensor(batch,timesteps,features) and if you want more flexibility you could call the LSTMCell which takes in batch and features and you process the LSTMCells required through your custom sequence length, Here i think the batch_size if not explicitly stated will be managed internally. So the lookback = time steps which are used internally to process the sequence.

[Syed Abdul Qadir Gilani]

Respected Teacher,

I hope you are doing well. My name is Syed, and I'm reaching out to offer my services as a data analyst and academic support provider. With expertise in Python programming and English, I can help with creative writing, editing, proofreading, tests, quizzes, any related typing assignment, script writing, data science, machine learning, etc.
Thank you very much to all of you, Stay happy and healthy always.
Please let me know if there is anything I can help with.
Long live.

Kind regards,
Syed
https://linktr.ee/syedabdulqadir
https://cutt.ly/4eaaZYzI
https://cutt.ly/eeaaB4rk

--
You received this message because you are subscribed to the Google Groups "Keras-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to keras-users...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/keras-users/cf062564-f5d4-46a3-8496-e41bb5a4817dn%40googlegroups.com.