Multi-task learning

[jacka...@gmail.com]

Hi, I'm interested in Multi-task learning in ASR. Now, I have trained a DNN using nnet1/train.sh. Which recipe could I follow to train Multitask DNN? Thanks!

[Daniel Povey]

It's not very easy.  I'm not sure if nnet1 really supports it.  In nnet2 we sometimes do it at the script level by replacing parts of neural nets (e.g. there is a train_multilang.sh example somewhere in the wsj example scripts, or similar-- search for multilang).  In nnet3 it's more naturally supported at the code level, as the nnets can have multiple outputs, and some people are working on this type of thing (e.g. Pegah, cc'd), but we don't yet have any example scripts for this.  It might be better for you to wait a while till we check something in.  If not, you could ask Pegah and see if she's willing to give you some advice.

Dan

On Fri, Dec 25, 2015 at 11:54 PM, <jacka...@gmail.com> wrote:

Hi, I'm interested in Multi-task learning in ASR. Now, I have trained a DNN using nnet1/train.sh. Which recipe could I follow to train Multitask DNN? Thanks!

--
You received this message because you are subscribed to the Google Groups "kaldi-help" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kaldi-help+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[jacka...@gmail.com]

Thanks Dan.

If I follow the train_multilang.sh, should I firstly convert the nnet1 into nnet2? 

BTW, I saw kaldi doc from http://kaldi.sourceforge.net/dnn1.html, run_blocksoftmax.sh in rm may support it, am I right?

[Daniel Povey]

Thanks Dan.

If I follow the train_multilang.sh, should I firstly convert the nnet1 into nnet2? 

No, the existing trained neural net wouldn't be any use.

 

BTW, I saw kaldi doc from http://kaldi.sourceforge.net/dnn1.html, run_blocksoftmax.sh in rm may support it, am I right?

Possibly but I'm not that familiar with the nnet1 tools.
Dan

[xff...@gmail.com]

Hi Dan and Pegah,

currently I also would like to try multitask learning using nnet3.
have you already updated some example scripts how to launch such experiments?
it will be great to have some, even piece of scripts/functions because I am a bit confused how to give two output data/labels for nnet3 training.

Many thanks and Kind regards,
Feifei

[feifei xiong]

Hi Pegah,

thanks a lot!

I checked your github and found
https://github.com/pegahgh/kaldi/blob/multilingual-modified-config/egs/babel_multilang/s5/local/nnet3/run_tdnn_multilingual.sh

hopefully this is the example I can refer to for multitask learning with tdnn as
"For all tasks, we share all except last hidden layer and there is separate final layer per task"

Thanks and Kind regards,

Feifei

On Tue, Apr 11, 2017 at 2:18 PM, Pegah Ghahremani <peg...@gmail.com> wrote:

You can check multilingual-modified-config branch in my github. We willhopefully merge it next week.

--

Kind regards,
Xiong Feifei