Phone error rate

[Dem]

Hi everyone,

I am a newbie of Speech Recognition field.

I read some ASR papers which used Phone Error Rate (PER) to evaluate their work.

However, in Kaldi, I only find the way to get Word Error Rate (WER).

So, how can I get PER in Kaldi?

Thank you very much.

[Vijayaditya Peddinti]

Only TIMIT recipe (https://github.com/kaldi-asr/kaldi/blob/master/egs/timit/s5/RESULTS) reports phone error rates. Even in this recipe the scripts report it as WER as the phones are treated as words.

--Vijay

--
You received this message because you are subscribed to the Google Groups "kaldi-help" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kaldi-help+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[lede...@gmail.com]

 So, you mean

- To compute WER:    use  the command: "for x in exp/*/decode*; do [ -d $x ] && [[ $x =~ "$1" ]] && grep WER $x/wer_* | utils/best_wer.sh; done "    ( taken from WSJ)

                

- To compute PER:  use the command : "for x in exp/{mono,tri,sgmm,dnn,combine}*/decode*; do [ -d $x ] && echo $x | grep "${1:-.*}" >/dev/null && grep WER $x/wer_* 2>/dev/null | utils/best_wer.sh; done"

                                and   "for x in exp/{mono,tri,sgmm,dnn,combine}*/decode*; do [ -d $x ] && echo $x | grep "${1:-.*}" >/dev/null && grep Sum $x/score_*/*.sys 2>/dev/null | utils/best_wer.sh; done"

    (taken from TIMIT)

Thanks.

[Jan Trmal]

No, he does not mean that.

What Vijay is saying is that only TIMIT will give you PER.

There is no way for an inexperienced user how to get PER on a system that was set up to give WER. 

y.

--

[Vijayaditya Peddinti]

Yes, but those commands are not computing the WER/PER they are just printing out the WER/PER from the log files.

--Vijay

On Mon, May 2, 2016 at 11:07 PM, <lede...@gmail.com> wrote:

--

[Jesus_Is_Lord]

I'm done with monophone training and /home/user/1111/kaldi/exp/System1/mono/decode/wer_11_0.0 gives me WER and SER, but I want to measure MER (morpheme error rate). Do you've any idea? Thanks in advance!

[an...@sayint.ai]

You would need to use morphemes rather than words as the units in both the LM and the dictionary. You would also have to convert the transcriptions accordingly.

[Jesus_Is_Lord]

thanks for your quick reply. I'm reusing an existing prepared corpus, it seems everything is in morpheme levels. However, what I couldn't do is, in the recipe that I use, recommended to adjust the beam size i.e beam size from --beam=$[$beam] to --beam=$[$beam4] and retry-beam size from --retry-beam=$[$beam4] to --retry-beam=$[$beam*22]. I tried to do that in the steps/train_mono_train.sh, but it keeps failing even with smaller beam size than recommended, it only works with the default settings. Do you have any idea how to change a beam size?