Retraining an already trained DNN (nnet1)

[Anis Chihi]

Hello,

I hope you can help me with these related issues issues:

1) I trained a DNN (nnet 1) using some corpus, and now the Idea is to retrain this same DNN again with some adaptation data. In steps/nnet/train.sh, there is an option called "nnet_init=" where it seems possible to use a pre initialized NN. Would it make sense then to put  "nnet_init= final.nnet", with final.nnet being the model file of the already trained DNN

2) I am interested in training only some components (for example only Input Layer, or output Layer) of the hidden layers. I read in an existing thread, that this is possible to do by putting <Learning_rate> to zero in the model files. Only problem is, I can't seem to find the model file. Is the nnet.proto file meant with model file?

And thanks a lot for your help :) 

[Karel Veselý]

Hi,
yes '1)' makes a lot of sense.
You may need to bypass the learning-rate-scheduling in the 1st epoch on new data by:
steps/nnet/train.sh --scheduler-opts "--keep-lr-iters 1" ...

For '2)' I am currently working on it, there will be a tool which sets the '<LearnRateCoef>' inside the models.
The model is the 'exp/.../final.nnet'.

Meanwhile it can be resaved as ascii by:
nnet-copy --binary=false nn.in nn.out
and modified manually by changing values of '<LearnRateCoef>' and '<BiasLearnRateCoef>'.

All the best,
Karel.

[Anis Chihi]

Thanks a lot Karel,

I came to this solution also. I am going to try using it the next days to retrain some portions of my network. And it will be really great to implement such a thing where the LearnRateCoef and BiasRateCoef could be set via an option.

Anis

[aaron.n...@griffithuni.edu.au]

Hello Anis, I am also interested in doing something like this, I would be very interested in knowing the way you solve this problem. 

[Anis Chihi]

Huy Aaron,

Like I said in my post, I will be trying the proposed solution this week. I'll let you know as soon as I have a working framework :) .

Anis

[Shubham Khandelwal]

Hello,

I tried to do Re-training using nnet3. So firstly, I trained a lstm nnet3 model on a corpus then I extracted <Nnet3> matrix from final.mdl into a raw file (say 0.raw) then I trained the same corpus again but this time, I used  this extracted 0.raw while training again using lstm nnet3 (instead from make_configs.py). But after decoding, I am getting around 95% WER which is a bad result.
Can you please help me in this regard.

Thanking you.

[Vijayaditya Peddinti]

This I think would be a problem with a mismatch in the ivector extractor.  When retraining the model extract ivectors with the same ivector extractor that was used when training the initial model. 

--Vijay

--
You received this message because you are subscribed to the Google Groups "kaldi-help" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kaldi-help+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Shubham Khandelwal]

Thanks Vijay for your reply.
But I did not use ivector for the training of initial model.
Btw now when I doubled the values of --extra-left-context & --extra-right-context while decoding, then I got good results.
Thanks.

[star633669]

Hello.

I want to know how to "extracted <Nnet3> matrix from final.mdl into a raw file (say 0.raw)"

I will appreciate any help,

Thanks 

Shubham Khandelwal於 2016年4月25日星期一 UTC+8下午7時51分07秒寫道：

[Daniel Povey]

You can use nnet3-am-copy with --raw=true.

But since recently, any program that expects a .raw nnet will happily read a .mdl file and just discard the transition-model as it reads, so format convertion is not even necessary.

Dan

--
Go to http://kaldi-asr.org/forums.html find out how to join
---

You received this message because you are subscribed to the Google Groups "kaldi-help" group.

To unsubscribe from this group and stop receiving emails from it, send an email to kaldi-help+unsubscribe@googlegroups.com.

[王星月]

thank you for your reply

You received this message because you are subscribed to a topic in the Google Groups "kaldi-help" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/kaldi-help/C3R7I9CmYLQ/unsubscribe.
To unsubscribe from this group and all its topics, send an email to kaldi-help+unsubscribe@googlegroups.com.