Run nnet3 (run_tdnn.sh) without GPU in recipe gale_arabic

[Alaa Jaber]

Hello everybody,

I'm use recipe (gale_arabic/s5b) to build ASR system. but I face some problem when I try run nnet3 (local/chain/run_tdnn.sh).

there statement check if CUDA is installed or not.

if ! cuda-compiled; then

cat <<EOF && exit 1

This script is intended to be used with GPUs but you have not compiled Kaldi with CUDA

If you want to use GPUs (and have them), go to src/, and configure and make on a machine

where "nvcc" is installed.

EOF

fi

So, I want to ask "Is it correct if I disable (comment) this sentence?", I don't have Nvidia Card Graphic on my PC

[Itai Peer]

NNET3 ( and chain)  is not supported without CUDA  , so get a GPU , or use paid server like google or amazon  provide .  

If you really insist to train with CPU ( i faced similar issue last year)  , you can use the NNET2  , which can be trained with and without GPU.

note that NNET2 is pretty much out of support , I'm not sure if its recipes are maintained anymore ,   and of course ,you will get higher WER with larger training time 

בתאריך יום חמישי, 5 בספטמבר 2019 בשעה 09:06:45 UTC+3, מאת Alaa Jaber:

[Alaa Jaber]

@Itai Peer

Thank you. but I don't understand what are you mean in "I  will get higher WER with larger training time! " this not logic, I was think if I have larger hour of training data then I will get better WER. So please if you can explain for me How I can get best WER?

[P-E Honnet]

Better WER = lower WER. So higher WER = worse WER. Word Error Rate.

If you want to get the best WER, you want to get the lowest possible WER.

[Itai Peer]

WER  , word error rate is function of several things , training data size is one of them , but the ASR itself is also important factor.

for the same training&testing sets, if you train 2 ASR models , one is based on NNET2 ( regular DNN ) and one with chain (time delayed NN )   , you will get 

Look for RESULTS file , like https://github.com/kaldi-asr/kaldi/blob/master/egs/wsj/s5/RESULTS  

On Thu, Sep 5, 2019 at 11:47 AM Alaa Jaber <alaa.s....@gmail.com> wrote:

@Itai Peer

Thank you. but I don't understand what are you mean in "I  will get higher WER with larger training time! " this not logic, I was think if I have larger hour of training data then I will get better WER. So please if you can explain for me How I can get best WER?

--
Go to http://kaldi-asr.org/forums.html find out how to join
---
You received this message because you are subscribed to a topic in the Google Groups "kaldi-help" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/kaldi-help/9oRWi7QDckA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to kaldi-help+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kaldi-help/08b28fdd-c5c3-401b-926f-80a2988db33a%40googlegroups.com.

--

Itai Peer

[Itai Peer]

yep  , higher WER = worst  , 

in mst cases you will get 

WER(chain)<WER(nnet3)<WER(nnet2)<WER(SGMM)<WER(tri+fmllr)  and so on 

בתאריך יום חמישי, 5 בספטמבר 2019 בשעה 11:57:31 UTC+3, מאת P-E Honnet:

[Alaa Jaber]

@P-E Honnet

@Itai Peer

Thanks for all

[Jan Trmal]

I think misunderstood -- what Itai was pointing out that nnet2 will (most probably) not give you the same performance as nnet3 (but lower) and the training time would be longer (even on GPU -- on CPU, it will be much longer, my guess is 20-80? x times slower, i..e. not very useful for anything except toy exercises.

y.

On Thu, Sep 5, 2019 at 10:47 AM Alaa Jaber <alaa.s....@gmail.com> wrote:

@Itai Peer

Thank you. but I don't understand what are you mean in "I  will get higher WER with larger training time! " this not logic, I was think if I have larger hour of training data then I will get better WER. So please if you can explain for me How I can get best WER?

--

Go to http://kaldi-asr.org/forums.html find out how to join
---

You received this message because you are subscribed to the Google Groups "kaldi-help" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kaldi-help+...@googlegroups.com.

[Itai Peer]

regarding the WER ( accuracy )  , you can check several RESULTS file in different recipes and see the differences.    ,usually it is between 5% and 10% relative improvement  ( in NNET2 you get WER=20% , so in chain you can expect 18%-19% )   

It difference might increases as Kaldi research now focuses on nnet3+chain so every new improvement is done there and not  in NNET2 

Regarding training speed , i can actually give you some reference :

I trained same NNET2  ,with same data  , once with GPU ( :GeForce RTX 2080 )   , and it took 5 hours , and once with CPU ,32 cores (actually 16 cores with 2 threads per core) , and it took 25 hours .   

off course the WER was the same  , since both were NNET2 , trained on same system. 

So the bottom line is the NNET2 is not too bad if you only have CPUs, although slower, and less accurate.    It all depending on what you are trying to achieve 

You received this message because you are subscribed to a topic in the Google Groups "kaldi-help" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/kaldi-help/9oRWi7QDckA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to kaldi-help+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kaldi-help/CAFReZQaJWZ5%3DYdupzGKZFkZxE5Uo1fWbR0-Kd3_0FNzt_X5atw%40mail.gmail.com.

--

Itai Peer