Calculation of a phoneme level probabilities for a GMM model

[Rustam Guliev]

Hi,

First of all, thank you for the Kaldi tool and all work you do.

As for my question, I want to get phoneme level probabilities and use them to calculate phoneme level errors (like gop, for exampe) for a mono-phone GMM model. I saw other questions on this topic, but I did get the answer. I wrote my own python code but I am not sure that it calculates exactly what I suppose it to do. Could you please just verify that I'm doing everything right?

[Rustam Guliev]

So, here is my algorithm. Let's assume I have trained a mono-phone GMM model and got forced alignment. Then for each utterance:

1. Estimate 'real' phonemes for each frame by alignment:

    ali-to-phones --per-frame=true <model> <ali_rspec> ark,t:-

    Output: alis - Vector of length M, M - number of frames

2. Compute log-likelihoods of pdf_ids for each frame:

    gmm-compute-likes {model} {feats_rspec} ark,t:-

    Output: likes - Matrix MxN, M - number of frames, N - number of pdf_ids

3. Convert log-likelihoods to probabilities by softmax transformation:

    probs[i,:] = softmax(likes[i,:])

    Output: probs - Matrix MxN

[Rustam Guliev]

4. Map pdf_id to phonemes:

    show-transitions {model_path}/phones.txt {model_path}/final.mdl {model_path}/final.occs | grep Transition-state
    # The output looks like:
    # Transition-state 134: phone = ws hmm-state = 0 pdf = 43
    # So I map pdf_id=43 to the 'ws' phoneme

    Output: mapping dict pdf_id->phoneme

[Rustam Guliev]

5. From pdf_id probabilities to phoneme probabilities: sum pdf_id-probabilities by phoneme:

    # For j-th phoneme
    probs_new[i,j] = sum(probs[i, <all pdf_ids of the phoneme j>])

    Output: probs_new - Matrix MxK, K - number of phonemes

Here I assume that for each frame (i):

 - predicted probability of the 'real' phoneme is probs_new[i,j] where   phoneme j == phoneme of alis[i]

 - predicted phoneme is argmax(probs_new[i,:]), i.e. phoneme with the highest probability

[Rustam Guliev]

P.S. I had to split it into many messages because I could not send it otherwise.

[Nickolay Shmyrev]

There are https://github.com/tbright17/kaldi-dnn-ali-gop and https://github.com/jimbozhang/kaldi-gop between

[Rustam Guliev]

Thanks, Nickolay. I'm aware of them but I was not able to implement any of them. It might be my bad because I'm new to Kaldi and their docs are not well detailed.

Anyway, even if there is a good working tool for gop. I would like get better understanding of what is going on inside. So it would be very helpful if someone validated my implementation.

четверг, 4 апреля 2019 г., 7:46:58 UTC+3 пользователь Nickolay Shmyrev написал:

[Daniel Povey]

I'm not aware of any length limit on posts, and would be very surprised if it existed, unless it relates to some kind of Russian firewall.

Most people post by email, FYI.

Your workflow looks like it would give you something reasonable.

It might be possible to implement some parts of it by using binaries `prob-to-post` and `get-post-on-ali`.

--
Go to http://kaldi-asr.org/forums.html find out how to join
---
You received this message because you are subscribed to the Google Groups "kaldi-help" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kaldi-help+...@googlegroups.com.
To post to this group, send email to kaldi...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kaldi-help/0b395a00-effc-42ec-85a7-eae185b31ce0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Yondu Tsai]

HI Rustam, did you get your system to work? 

[Rustam Guliev]

Hi,

Currently, I created a helper-script in python. It is available on GitHub: https://github.com/it-muslim/kaldi/blob/quran-academy/egs/hqa/helpers/phone_probs.py. Maybe later I will put it in a separate repo.

Rustam

среда, 28 августа 2019 г., 12:19:03 UTC+3 пользователь Yondu Tsai написал:

[Yondu Tsai]

Thank you, I tried it but it seems gmm-compute-likes has an error
Do you have the full example of that, here is what I get when i print gmm_likes_command
```
gmm-compute-likes model/final.mdl ark,s,cs:apply-cmvn --utt2spk=ark:data/utt2spk scp:data/cmvn.scp scp:data/feats.scp ark:- 
```

can you provide what you have when you print gmm_likes_command

Thank you

[Yondu Tsai]

Ok I got it working, but there is another issue

If the phoneme from forced alignment in the alignment file is not in the gmm likes file, then there is a lookup error

'ali_prob': prob.lookup(np.arange(prob.shape[0]), ali),

 

[Rustam Guliev]

Hi Yondu,

I moved it to a separate repo: https://github.com/it-muslim/kaldi-helpers

I created an issue for your question: https://github.com/it-muslim/kaldi-helpers/issues/1 . Let's proceed with the discussion there since it is a bit irrelevant here.

четверг, 29 августа 2019 г., 13:00:08 UTC+3 пользователь Yondu Tsai написал:

[Muhammad Talha Umair Alim]

Hi Rustam,
I hope you are doing great. I need your help regarding lexicon file of Quran Arabic. Actually I am working on Quran recitation correction project but facing different issues regarding Kaldi. I have seen your project Github repo. Did your project working fine. If it so then can you please share your lexicon or phonemes file with me. BTW I have done with voxforge as well as kaldi for dummies.
Waiting for your reply.

[Rustam Guliev]

Assalamu alaikum Muhammad,

I am sorry for the late reply. Unfortunately, I don't have access to those files now. As far as I recall, we had them only for al-Fatiha, so I am not sure if it is helpful or not.

Wish you all the best,

Rustam Guliev

--
Go to http://kaldi-asr.org/forums.html to find out how to join the kaldi-help group
---
You received this message because you are subscribed to a topic in the Google Groups "kaldi-help" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/kaldi-help/Yci5W0l-lPw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to kaldi-help+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kaldi-help/3ac2dd69-9f15-403d-bb53-459999e01ed9n%40googlegroups.com.