Decoder Error with JSGF Grammar

[xinq...@gmail.com]

Hi all,

     I wanna build a JSGF Grammar based decoder with TDNN acoustic model. 

   

     After converted the jsgf grammar file to HCLG.fst, I found the result is good for positive samples, but for negative samples which is not the words in th grammar, it can still be actived.

    How can I fix this issue?

Many thanks

Xin.q.

[Daniel Povey]

You need to include an alternative path or paths in your grammar that will match arbitrary speech, so that you can reject it.  Of course this will activate the grammar, but you'll be able to work out that there are "bad paths" in it, and not do anything with those utterances.

What I suggest is to use the <unk> symbol (the symbol doesn't matter) in the LM, in various places, to accept arbitrary "unk" speech, and then use the script that inserts a phone-level n-gram language model in place of <unk>.  There is an example of this in the tedlium s5_r2 recipe, search for 'unk' in run.sh.

Make sure that each instance of <unk> is followed by the option to end the grammar, because if you have <unk> followed by some specific word, the decoder may get "stuck" there until it sees that word, so you'll never reach the end-state of the grammar.

Dan

--
You received this message because you are subscribed to the Google Groups "kaldi-help" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kaldi-help+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[rrb]

Hi Dan,

Your suggestion of inserting a phone-level n-gram language model will model the OOV's. But won't that miss out OOV recognition. i.e we won't be knowing that there are bad paths. Is is possible to insert a special symbol that will consume no frames (like eps) but will help us to identify that a OOV path is taken ?. For example, a __oov_start to indicate that graph search entered phone-level path and __oov_end to indicate it exited from the path ? The wfst optimization otherwise merge all the paths and make it difficult to identify which is bad path and which one is good path.

Thanks,

Rag

[xinq...@gmail.com]

Hi Dan

My JSGF file is as following for an example:

grammar hello;
public <greet> = (hi | hello) ( bhiksha | evandro | paul | philip );

as you said, my fsm is:

1  2   hi  hi

1  2   hello  hello

1  2   UNK  UNK

2  3   bhiksha bhiksha
2 3 evandro evandro

2 3 paul  paul

2 3 philip philip
2 3 UNK UNK

3

 

the alternative path is added as the red front.

The test result in negative samples is much better than before. But still some false alarm.

1. Is there any path I missed?

2. How can I improve the performance in this case. 

3. if the jsgf just one word only for Wake Up Word Recognition, how can I do?.

 

Many tanks

Xin.q.

在 2016年11月5日星期六 UTC+8上午6:19:56，Dan Povey写道：

To unsubscribe from this group and stop receiving emails from it, send an email to kaldi-help+...@googlegroups.com.

[Daniel Povey]

Your suggestion of inserting a phone-level n-gram language model will model the OOV's. But won't that miss out OOV recognition. i.e we won't be knowing that there are bad paths. Is is possible to insert a special symbol that will consume no frames (like eps) but will help us to identify that a OOV path is taken ?. For example, a __oov_start to indicate that graph search entered phone-level path and __oov_end to indicate it exited from the path ? The wfst optimization otherwise merge all the paths and make it difficult to identify which is bad path and which one is good path.

There is still a word-level symbol e.g. <unk> that you can use to identify that the path was "bad'.

Dan

 

On Friday, 4 November 2016 15:23:50 UTC+5:30, xinq...@gmail.com wrote:

Hi all,

     I wanna build a JSGF Grammar based decoder with TDNN acoustic model. 

   

     After converted the jsgf grammar file to HCLG.fst, I found the result is good for positive samples, but for negative samples which is not the words in th grammar, it can still be actived.

    How can I fix this issue?

Many thanks

Xin.q.

--

You received this message because you are subscribed to the Google Groups "kaldi-help" group.

To unsubscribe from this group and stop receiving emails from it, send an email to kaldi-help+unsubscribe@googlegroups.com.

[Yvonne Hong]

Hi Xin,

How did you convert the jsgf grammar file to HCLG.fst?

I created jsgf grammar for phone-based language model with TIMIT corpus.

Then I converted it into G.fst (.jsgf -> .fsm using 'sphinx_jsgf2fsg' -> .fst using 'fstcompile').

Now I'm trying to compose HCLG.fst with mkgraph.sh, and I get an error due to the failure of composing L_disambig.fst and G.fst. The error message reads olabel of G.fst is not sorted. So I tried to olabel sort using fstarcsort --sort-type=olabel, but it failed:( I also made sure that the word(=phone) id used for L and G are equivalent.

Have you worked on a similar problem or can you guess ?

My text version of G.fst is as follows:

0 1 <s>

1 2 sil

2 3 aa

2 3 ae

2 3 ah

2 3 ao

2 3 aw

2 3 ax

2 3 ay

2 3 b

2 3 ch

2 3 cl

2 3 d

2 3 dh

2 3 dx

2 3 eh

2 3 el

2 3 en

2 3 epi

2 3 er

2 3 ey

2 3 f

2 3 g

2 3 hh

2 3 ih

2 3 ix

2 3 iy

2 3 jh

2 3 k

2 3 l

2 3 m

2 3 n

2 3 ng

2 3 ow

2 3 oy

2 3 p

2 3 r

2 3 s

2 3 sh

2 3 t

2 3 th

2 3 uh

2 3 uw

2 3 v

2 3 vcl

2 3 w

2 3 y

2 3 z

2 3 zh

2 3 #0

2 4 aa

2 4 ae

2 4 ah

2 4 ao

2 4 aw

2 4 ax

2 4 ay

2 4 b

2 4 ch

2 4 cl

2 4 d

2 4 dh

2 4 dx

2 4 eh

2 4 el

2 4 en

2 4 epi

2 4 er

2 4 ey

2 4 f

2 4 g

2 4 hh

2 4 ih

2 4 ix

2 4 iy

2 4 jh

2 4 k

2 4 l

2 4 m

2 4 n

2 4 ng

2 4 ow

2 4 oy

2 4 p

2 4 r

2 4 s

2 4 sh

2 4 t

2 4 th

2 4 uh

2 4 uw

2 4 v

2 4 vcl

2 4 w

2 4 y

2 4 z

2 4 zh

2 4 #0

3 5 sil

4 2 <eps>

5 6 </s>

6 

and the graphical representation of G.fst is as follows:

It seems that I'm missing some basic points, but can' t figure it out.

I'd be glad if you have any suggestion on this! 

Thank you!

2016년 11월 4일 금요일 오후 6시 53분 50초 UTC+9, xinq...@gmail.com 님의 말:

[Daniel Povey]

fstarcsort can never fail.  You should be more specific about what happened.

--

[Yvonne Hong]

You're right. The error wrt fstarcsort was due to my careless spelling mistake (misspelled "--sort_type" as "--sort-type").

Still, I get an empty LG.fst.

Here's my jsgf grammar for TIMIT language model. I wanted uniformly distributed phone-based LM.

#JSGF V1.0;

grammar languageModel ;

public <ngram> = <START> <SIL> <PHONE>+ <SIL> <END>;

<PHONE> =  aa | ae | ah | ao | aw | ax | ay | b | ch | cl | d | dh | dx | eh | el | en | epi | er | ey | f | g | hh | ih | ix | iy | jh | k | l | m | n | ng | ow | oy | p | r | s | sh | t | th | uh | uw | v | vcl | w | y | z | zh | #0 ;

<START> = "<s>" ;

<END> = "</s>" ;

<SIL> = sil;

(I deleted double-quotation after the processing.)

After converting the jsgf grammar to G.fst (now olabel sorted), mkgraph.sh cannot successully compose L_disambig.fst and G.fst. (LG.fst is 66bytes). 

Below is the exact error message from mkgraph.sh:

tree-info test_created/am/tree

tree-info test_created/am/tree

make-h-transducer --disambig-syms-out=test_created/graph/disambig_tid.int --transition-scale=1.0 test_created/lang/tmp/ilabels_3_1 test_created/am/tree test_created/am/final.mdl

fsttablecompose test_created/graph/Ha.fst test_created/lang/tmp/CLG_3_1.fst

fstrmepslocal

fstdeterminizestar --use-log=true

fstminimizeencoded

fstrmsymbols test_created/graph/disambig_tid.int

fstisstochastic test_created/graph/HCLGa.fst

0 0

add-self-loops --self-loop-scale=0.1 --reorder=true test_created/am/final.mdl

ASSERTION_FAILED (add-self-loops:AddSelfLoops():hmm-utils.cc:554) : 'fst->Start() != fst::kNoStateId'

[ Stack-Trace: ]

add-self-loops() [0x64ff88]

kaldi::MessageLogger::HandleMessage(kaldi::LogMessageEnvelope const&, char const*)

kaldi::MessageLogger::~MessageLogger()

kaldi::KaldiAssertFailure_(char const*, char const*, int, char const*)

kaldi::AddSelfLoops(kaldi::TransitionModel const&, std::vector<int, std::allocator<int> > const&, float, bool, fst::VectorFst<fst::ArcTpl<fst::TropicalWeightTpl<float> > >*)

main

__libc_start_main

_start

utils/mkgraph.sh: line 137: 25552 Aborted                 (core dumped) add-self-loops --self-loop-scale=$loopscale --reorder=true $model < $dir/HCLGa.fst > $dir/HCLG.fst

2017년 1월 24일 화요일 오전 11시 52분 28초 UTC+9, Dan Povey 님의 말:

To unsubscribe from this group and stop receiving emails from it, send an email to kaldi-help+...@googlegroups.com.

[Daniel Povey]

You shouldn't have `#0` in there, that is not a phone, it's a  disambiguation symbol.

And the graph (G.fst) should not start with `<s>` and end with `</s>`.  Those shouldn't appear in the graph, they won't have any pronunciations.

Dan

[Yvonne Hong]

Thank you!!

I misunderstood that the disambiguation symbol in G.fst should be applied even in the jsgf grammar modelling level. 

Thank you so much!

2017년 1월 24일 화요일 오후 2시 25분 35초 UTC+9, Dan Povey 님의 말: