Higher log-likelihood score when give more training data in GMM-UBM speaker verification system

[Nguyễn Cẩn]

Dear all,

I'm in trouble with log-likehihood score in GMM-UBM speaker verification system when apply my recorded audio to test.

When I'm give more data for training speaker model, log-likelihood score when testing this model increase. Even if it is impostor, the score increase and higher than my threshold 

(I have this threshold when make experiment with TIMIT dataset, accuracy is 93%). Hence, my system is going to be fail with more training data.

I dont know why it happen.

Have you have any idea for this problem? Hope to have your advice.

Thanks you.

P/s: This is code summary of my speaker verification system with bob. I'm using MFCCs features.

 #train UBM

  ubm_model = algorithm.GMM(1024)

  ubm_model.train_projector(my_features, 'save_name.hdf')

 #Enroll speaker model

   speaker_model = ubm_model.enroll(speaker_features)

 #Test

   probe = ubm_model.project(speaker_testing_features)

   score = ubm_model.score(speaker_model, probe)

[Tiago Freitas Pereira]

Hi Nguyễn,

I'm not a speaker recognition guy, but I think I can give you some hints on this one.

First, have you systematically analysed this behaviour?

For instance, would be nice to see the score distributions (of the genuine and impostors) as a function of number of samples used to enroll a client.

The base question here is; are they both (genuine and impostors score distributions) shifted or they are more intersected?

Depending on the result of this analysis, you may need to fine tune your decision threshold.

Second; I know nothing about your dataset.

Assuming that you have a substantial amount (whatever substantial means) of data to train a speaker model, you can try to play with the MAP adaptation factor parameter (https://www.idiap.ch/software/bob/docs/bob/bob.bio.gmm/stable/implemented.html#bob.bio.gmm.algorithm.GMM).

Hope it helps

Cheers

--
-- You received this message because you are subscribed to the Google Groups bob-devel group. To post to this group, send email to bob-...@googlegroups.com. To unsubscribe from this group, send email to bob-devel+unsubscribe@googlegroups.com. For more options, visit this group at https://groups.google.com/d/forum/bob-devel or directly the project website at http://idiap.github.com/bob/
---
You received this message because you are subscribed to the Google Groups "bob-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bob-devel+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Tiago

[Nguyễn Cẩn]

Dear Tiago,

Thanks for your advice. 

When give more data for enrollment, I've already used MAP adaption. Score of genuine and impostors they both increase (testing with 15 speaker). But, as your advice, I think I will systematically analysed this behaviour.

I recorded myself this testing data. Keep the same sampling frequency with UBM training data in TIMIT dataset.

I've suspected that it's UBM model problem. Then, I recorded some of my voices and retrain UBM model. But the scores still increase.

What do you think about this step?

By the way, can you have a look at my code summary? Is it in right steps? I can't run bob's experiment. So I read bob' documents and make this code.

Thank you very much for helping.

Best,

[Tiago Freitas Pereira]

Hi Nguyễn,

Sorry for the delay (I forgot about this message).

The code summary you printed seems alright. Just for curiosity, you should analyse this behaviour with the `GMMRegular` class.
This one uses the classic scoring function from Reynolds (not the linear scoring implemented in the GMM class).

About original your issue, it's difficult to tell by e-mail.

Your insight about the UBM seems reasonable; you suspect that there's a channel offset between TIMIT and your data and this offset this biasing your scoring.

One easy thing that you can do to tackle this issue is to use the data you collected to fine tune the decision threshold.
Another thing that you can experiment (this one a bit more complex) is the ISV.

Such approach try to suppress possible channel variability in the GMM supervector space.

You can use your data+TIMIT to train the ISV background models (UBM+U matrix).

Hope it helps

Cheers

--
-- You received this message because you are subscribed to the Google Groups bob-devel group. To post to this group, send email to bob-...@googlegroups.com. To unsubscribe from this group, send email to bob-devel+unsubscribe@googlegroups.com. For more options, visit this group at https://groups.google.com/d/forum/bob-devel or directly the project website at http://idiap.github.com/bob/
---
You received this message because you are subscribed to the Google Groups "bob-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bob-devel+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Tiago

[Nguyễn Cẩn]

Dear Tiago,

It's my pleases to have your reply.

I'll have a look with GMMRegular as soon as possible.

About the issues, after analysed this behaviour. I found that it just happen with my own data. My system works well with TIMIT when i give more TIMIT data for speaker modeling.

Then I think my system is overfitting with TIMIT dataset. 

Hence, I make some pre-processing, try to make spectrogram of my own data look like with spectrogram of TIMIT data. Those steps make the score closer to the threshold (not too much higher than threshold like before). It did not solve problem at all, but i can use it to fine tune a new threshold as your advice.

I'll try with ISV and i-vector (in kaldi framework) too.

Again, thanks so much. As a new man in this field, your answer is so useful with me.

Best,

On Monday, March 5, 2018 at 3:22:32 PM UTC+7, Nguyễn Cẩn wrote:

[Amir Mohammadi]

Hi Nguyễn,

Why don't you create a file-list based interface for your dataset and run it against the bob.bio.base pipeline?
https://www.idiap.ch/software/bob/docs/bob/bob.bio.base/stable/experiments.html
https://www.idiap.ch/software/bob/docs/bob/bob.bio.base/stable/filelist-guide.html
https://www.idiap.ch/software/bob/docs/bob/bob.bio.spear/stable/baselines.html

The docs on filelist-based database interfaces is a bit outdated but you can see several emails in this mailing list that

we explain on how to create a file-list database interface.

This way you can make sure you are running all the steps properly.

Thanks,

Amir

--
-- You received this message because you are subscribed to the Google Groups bob-devel group. To post to this group, send email to bob-...@googlegroups.com. To unsubscribe from this group, send email to bob-devel+...@googlegroups.com. For more options, visit this group at https://groups.google.com/d/forum/bob-devel or directly the project website at http://idiap.github.com/bob/

---
You received this message because you are subscribed to the Google Groups "bob-devel" group.

To unsubscribe from this group and stop receiving emails from it, send an email to bob-devel+...@googlegroups.com.

[Nguyễn Cẩn]

Dear Amir,

Thanks for suggestion,

I'm going to do it.

Best,

On Monday, March 5, 2018 at 3:22:32 PM UTC+7, Nguyễn Cẩn wrote: