Why is there a disambiguation symbol in L_disambig.fst after optional silence?

[Peter Smit]

Hi

The lexicon fst that is used in decoding (L_disambig.fst) has, besides the G.fst disambiguation #0 on self-loops, also a disambiguation symbol after the optional in-between-word-silence (added by make_lexicon_fst.pl). I do understand the reasons for the disambiguation symbol in G.fst and the other disambiguation symbols in the lexicon, but this one eludes me.

It seems it was already introduced in 2011 in the first version of the scripts ( https://github.com/kaldi-asr/kaldi/commit/7f5790f0fd7d285aaea0223cb0d1236a93ef8431 ), except that it might be related to lattice-determination crashing. The documentation (http://kaldi-asr.org/doc/graph_recipe_test.html) is not helping, and in the code examples there it is actually not used.

Is there still use for this particular disambiguation symbol? If yes, what is exactly the use? If I make my own L.fst for decoding with not only optional silence, but more options, e.g. SIL, SIL NOISE, NOISE SIL, should I use a lot of different disambiguation symbols? Or one per unique phone there, and straight after each phone?

Thanks in advance,

Peter

[Daniel Povey]

I believe in the current scripts we add a disambiguation symbol when
you don't consume the optional silence, simply because that was easier
to implement (you have an epsilon arc in that case, so there is
somewhere to put the disambig symbol).
The reason we need a disambiguation symbol to say whether
optional-silence was consumed, is to handle the case where the lexicon
has silence as a word (and where you are not using
word-position-dependent phones). There are certain cases where
determinization can fail otherwise. I forget the details-- it was a
very complicated thing.
You are safe if those phones never appear in your dictionary. If they
do appear, it would be safer to add a separate disambiguation symbol
for each sequence of phones added by the lexicon.

Dan

> --
> You received this message because you are subscribed to the Google Groups
> "kaldi-help" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to kaldi-help+...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Peter Smit]

It is actually put after every sil, a specific disambig-state is made (https://github.com/kaldi-asr/kaldi/blob/master/egs/wsj/s5/utils/make_lexicon_fst.pl#L112-L113).

If I reason well, I can also just add disambiguation symbols to those silence words in the lexicon (and if there area actual multiple words mapping to silence they would have disambig symbols already). 

[Daniel Povey]

> It is actually put after every sil, a specific disambig-state is made
> (https://github.com/kaldi-asr/kaldi/blob/master/egs/wsj/s5/utils/make_lexicon_fst.pl#L112-L113).

Oh, yes you are right.

> If I reason well, I can also just add disambiguation symbols to those
> silence words in the lexicon (and if there area actual multiple words
> mapping to silence they would have disambig symbols already).

It might not be that simple if silence can appear inside words in the lexicon.


Dan