Is there anyway to freeze/unfreeze layers without recompiling the model?

[geek_kid]

I am under the impression that recompiling restarts optimizer state and other things that should ideally not be restarted. I have some GAN code that freezes discriminator layers to train the generator, then unfreezes the discriminator to train the discriminator and to apply the (un)freezing I believe have to recompile. Is there any way around this apparent catch-22?

[Lance Norskog]

I believe that freezing a layer requires setting a boolean flag, and this flag is checked on each epoch. You should not need to recompile the model.

On Fri, Mar 29, 2019 at 1:01 PM geek_kid <roars...@gmail.com> wrote:

I am under the impression that recompiling restarts optimizer state and other things that should ideally not be restarted. I have some GAN code that freezes discriminator layers to train the generator, then unfreezes the discriminator to train the discriminator and to apply the (un)freezing I believe have to recompile. Is there any way around this apparent catch-22?

--
You received this message because you are subscribed to the Google Groups "Keras-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to keras-users...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/keras-users/f29d14be-d2b4-4f7c-8a22-c9242ab2e554%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Lance Norskog
lance....@gmail.com
Redwood City, CA

[geek_kid]

The docs apear to say I do need to recompile the model

Here is a code snippet of what I am doing:

    #unfreezes the discriminator layer as we are about the train it

    discriminator.trainable=True  

    #keras requires recompialton after layers are frozen or unfrozen

    discriminator.compile(loss='binary_crossentropy', optimizer=discrim_optimizer)

    #trains the discrimanator

    #sepearte batches to help batch norm

    disc_loss_1=discriminator.train_on_batch([images,labels], valid)

    disc_loss_2=discriminator.train_on_batch([gen_images,gen_labels], fake)

    disc_loss_1=discriminator.train_on_batch([images,labels], valid)

    disc_loss_2=discriminator.train_on_batch([gen_images,gen_labels], fake)

    #takes the mean of the two discrimantor losses

    disc_loss=(disc_loss_1+disc_loss_2)/2

    #freezes the discrnaor layers as we want to train the geneatator layers only

    discriminator.trainable=False

    #keras requires recompialton after layers are frozen or unfrozen

    discriminator.compile(loss='binary_crossentropy', optimizer=discrim_optimizer)

    combined.compile(loss='binary_crossentropy', optimizer=gen_optimizer)

    #trains the combined model (which only has generator layers trainable so we are only really training the generator) to make the discrimantor output 1 when passed generator images

    gen_loss=combined.train_on_batch([noise,gen_labels], np.zeros((batch_size,1)))

On Saturday, 30 March 2019 02:06:46 UTC, Lance N. wrote:

I believe that freezing a layer requires setting a boolean flag, and this flag is checked on each epoch. You should not need to recompile the model.

On Fri, Mar 29, 2019 at 1:01 PM geek_kid <roars...@gmail.com> wrote:

I am under the impression that recompiling restarts optimizer state and other things that should ideally not be restarted. I have some GAN code that freezes discriminator layers to train the generator, then unfreezes the discriminator to train the discriminator and to apply the (un)freezing I believe have to recompile. Is there any way around this apparent catch-22?

--
You received this message because you are subscribed to the Google Groups "Keras-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to keras...@googlegroups.com.

To view this discussion on the web, visit https://groups.google.com/d/msgid/keras-users/f29d14be-d2b4-4f7c-8a22-c9242ab2e554%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Lance Norskog]

Ok, at least someone reads the manual :)

If this is just about a consistent optimizer, you can run model.fit() with the starting epoch. as 'initial_epoch'. I assume the optimizers know how to fast-forward to this step.

fit(
    x=None,
    y=None,
    batch_size=None,
    epochs=1,
...
    initial_epoch=0,
 ...
)

To unsubscribe from this group and stop receiving emails from it, send an email to keras-users...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/keras-users/f0ad3036-4e54-4599-a5ac-b438b75994e8%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.