Does recompiling a model re-initialize the weights?

[matthew...@gmail.com]

Hi Keras,

I have some code that I'm running in a leave one out style and I notice that the loss is behaving like the weights arent being reset when I recompile. Is this just good luck on my part? Or has this behavior changed? Is there a method for resetting the weights and optimizer? Right now I just have the model.compile() in a for loop for each left out class. 

Thanks

[Qixianbiao Qixianbiao]

If you want to initiate the network with the same weights every time to repeat some experiments, you should set the random, or the network will initiate it randomly...... 

Pls refer to the following code....

from keras.models import Sequential

from keras.layers import Dense, Activation

import numpy as np

np.random.seed(2016)  

#if you set here, then every time, you can initiate the model with the same weights

model = Sequential([

    Dense(3, input_dim=784),

    Activation('relu'),

    Dense(10),

    Activation('softmax'),

])

model.compile(optimizer='rmsprop',

              loss='categorical_crossentropy',

              metrics=['accuracy'])

for layer in model.layers:

    weights = layer.get_weights()

    print(weights)

在 2016年9月13日星期二 UTC+8下午9:38:21，matthew...@gmail.com写道：

[matthew...@gmail.com]

Thanks for the reply, but I'm afraid I wasnt clear. Briefly I have code like this:

model = get_model() #calls a function that builds a model
for left_out_class in list_of_left_out_classes:

model.compile(loss='mse',optimizer='adagrad') #does this reset the weights to random every time?
model.fit(left_out_data)
model.save_weights()# or whatever.

Will compiling inside a loop really reset the weights to random?

[François Chollet]

Compiling a model does not modify its state. Weights after compilation are the same as before compilation.

[matthew...@gmail.com]

Alright thanks. I thought this used to be the behavior, but anyway, is there a better way to reset the weights? or just remake the object (move the get_model() into the loop)?

[Tomasz Melcer]

On 09/14/2016 07:42 PM, matthew...@gmail.com wrote:
> Alright thanks. I thought this used to be the behavior, but anyway, is
> there a better way to reset the weights? or just remake the object (move
> the get_model() into the loop)?

I am doing the latter and it's pretty much fine for me.


--
Tomasz Melcer

[di....@live.com]

Does repeatedly compiling the model in a for loop cause the training to slow down as it progresses? How are you dealing with that?

[Devansh Shah]

repeatedly compiling the model in a for loop cause the training to slow down as it progresses.

How are you dealing with that?

[Lance Norskog]

This case looks di.ney@ is attempting to do "curriculum learning", where you change the training dataset on the fly. If that is the case, do not call model.compile() for each epoch. Only call model.compile at the beginning.

Lance

--
You received this message because you are subscribed to the Google Groups "Keras-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to keras-users...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/keras-users/91752d41-e4c1-4103-abb9-34c7be3674b4n%40googlegroups.com.

--

Lance Norskog
lance....@gmail.com
Redwood City, CA