training ivector extractor

[Peng Jerry]

Hi everyone~

I am trying to develop a standard i-vector extractor and I refer to the relevant codes in kaldi(povery and synder's) and idiap's kaldi-ivector https://github.com/idiap/kaldi-ivector.

By simply updating projection matrix in Maximization phase, my current result is worse than the ivector based on SGMM. And what makes me confused is that, by increasing the number of training iteration from 6 to 15, my extractor goes worse which seems impossible in theory. I also tested idiap's kaldi-ivector https://github.com/idiap/kaldi-ivector, it suffers the same problem. While for synder's ivector, the performance gets a little enhanced... (So sad ...)

All these above use ubm model to generate posteriors and initialize extractors. They are evaluated by simply cosine scoring and the performance is measured by EER. I also testify them with and without normalization and centering of i-vectors. But it doesnt give much difference in EER.

Also I look into the auxiliary objective function in func UpdateProjection() of class IvectorExtractor file ivector/ivector-extractor.cc:

  In my log file, It converges and the impr value in each iteration gets smaller.

I also tried to evaluate the model parameters' difference by subtracting the projection matrix between each iteration and then applying Frobenius norm. It also indicates the convergence of training ...

I know it may be hopeless to solve this problem as I cannot show the details of both my codes and idiap's but I have tried to debug and get nothing. Wish someone met this problem before!

Best Regards,

Jerry

[Daniel Povey]

Cc'ing  Petr Motlicek in case he has any ideas.  I wonder if there could be a mismatch between the data used for training the UBM, vs. the data used for training the i-vector extractor, or maybe you are using a very small amount of data (normally it should be hundreds or thousands of hours).  There might be min-counts or priors in our code that are making this less bad.

Even if there is a difference, the difference should be very tiny, like 1% to 5% relative difference, I would guess.  The size of the difference in error rates makes me things something might be wrong in your setup.

Dan

--
Go to http://kaldi-asr.org/forums.html find out how to join
---
You received this message because you are subscribed to the Google Groups "kaldi-help" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kaldi-help+unsubscribe@googlegroups.com.
To post to this group, send email to kaldi...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kaldi-help/a03599e5-14f1-4bc4-b7b4-25dfab0d13b9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Peng Jerry]

Many thanks for your advice. I tried debug the codes and derived the formulations again again and again, but got no progress.

However, I find a discussion https://groups.google.com/forum/#!msg/bob-devel/2lmIXZJvmAo/6qdCC3Vka3sJ 

that talks the same problem: more iterations do not always lead to better performance.

And they solved this problem by pre-processing i-vector before cosing scoring, say, whitening. And I didn't expect the whitening would have such a great impact.

I will try to justify this conclusion later. And I guess maybe there is a mismatch between the training objective function and the final performance because T, the projection matrix actually tries to capture all the variance of data not just the variance associated to speakers.

But still, I don't know why your extractor works so well even without pre-processing. I will check the codes later...

On Tuesday, August 7, 2018 at 1:42:57 PM UTC+8, Dan Povey wrote:

Cc'ing  Petr Motlicek in case he has any ideas.  I wonder if there could be a mismatch between the data used for training the UBM, vs. the data used for training the i-vector extractor, or maybe you are using a very small amount of data (normally it should be hundreds or thousands of hours).  There might be min-counts or priors in our code that are making this less bad.

Even if there is a difference, the difference should be very tiny, like 1% to 5% relative difference, I would guess.  The size of the difference in error rates makes me things something might be wrong in your setup.

Dan

On Mon, Aug 6, 2018 at 9:26 PM, Peng Jerry <jerryp...@gmail.com> wrote:

Hi everyone~

I am trying to develop a standard i-vector extractor and I refer to the relevant codes in kaldi(povery and synder's) and idiap's kaldi-ivector https://github.com/idiap/kaldi-ivector.

By simply updating projection matrix in Maximization phase, my current result is worse than the ivector based on SGMM. And what makes me confused is that, by increasing the number of training iteration from 6 to 15, my extractor goes worse which seems impossible in theory. I also tested idiap's kaldi-ivector https://github.com/idiap/kaldi-ivector, it suffers the same problem. While for synder's ivector, the performance gets a little enhanced... (So sad ...)

All these above use ubm model to generate posteriors and initialize extractors. They are evaluated by simply cosine scoring and the performance is measured by EER. I also testify them with and without normalization and centering of i-vectors. But it doesnt give much difference in EER.

Also I look into the auxiliary objective function in func UpdateProjection() of class IvectorExtractor file ivector/ivector-extractor.cc:

  In my log file, It converges and the impr value in each iteration gets smaller.

I also tried to evaluate the model parameters' difference by subtracting the projection matrix between each iteration and then applying Frobenius norm. It also indicates the convergence of training ...

I know it may be hopeless to solve this problem as I cannot show the details of both my codes and idiap's but I have tried to debug and get nothing. Wish someone met this problem before!

Best Regards,

Jerry

--
Go to http://kaldi-asr.org/forums.html find out how to join
---
You received this message because you are subscribed to the Google Groups "kaldi-help" group.

To unsubscribe from this group and stop receiving emails from it, send an email to kaldi-help+...@googlegroups.com.

[Daniel Povey]

In our formulation we essentially re-whiten on each iteration.  Perhaps the other formulations don't do that because they assume it will be done as a post-processing step.

Dan

To unsubscribe from this group and stop receiving emails from it, send an email to kaldi-help+unsubscribe@googlegroups.com.

To post to this group, send email to kaldi...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/kaldi-help/b4544bb5-17ce-4ad9-9985-e169b163def5%40googlegroups.com.

[Peng Jerry]

 Thank you for your help!!!

 Problem solved when adding whitening phase before cosine score. Now the performance is nearly the same as yours.