Dropout Proportion vs Dropout Schedule

[ma...@nyu.edu]

I do not understand the difference between dropout_proportion and dropout_schedule in the chain model. Can some explain this? (yes, I realize the schedule is time varying, but then why do you need dropout_proportion?).

Thanks

Michael

[Daniel Povey]

If you are not using a dropout schedule, dropout_proportion controls the dropout.  Otherwise, I think the only thing that matters is, is -1.0 (--> no dropout component is created), or has it been set?  

--
Go to http://kaldi-asr.org/forums.html to find out how to join the kaldi-help group
---
You received this message because you are subscribed to the Google Groups "kaldi-help" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kaldi-help+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kaldi-help/8e8d3253-291a-4981-ae44-6fa570e353fdn%40googlegroups.com.

[ma...@nyu.edu]

It looks like the schedule is a global setting (it is a parameter passed  to steps/chain/train.py)  but the proportion can be set for each layer separately so I thought they interacted in some fashion or other... we certainly get different results depending on the combination of proportion and schedule so there seems to be some connection...

Michael

[Daniel Povey]

I'm pretty sure dropout_schedule just sets dropout-proportion, search for

edit_config_lines

in

egs/wsj/s5/steps/libs/nnet3/train/dropout_schedule.py

To view this discussion on the web visit https://groups.google.com/d/msgid/kaldi-help/0568ad05-443a-4442-ad96-6f2424def117n%40googlegroups.com.