Scores in score-dev file

[Prasanna Kothalkar]

Hello,

 I am trying to understand how the scores (last column corresponding to each utterance file) present in scores-dev, scores-eval etc. are computed. For GMMs I looked into GMM.py and found the following score function:

def score(self, model, probe):
	"""Computes the score for the given model and the given probe.
	The score are Log-Likelihood.
	Therefore, the log of the likelihood ratio is obtained by computing the following difference."""
	assert isinstance(model, bob.learn.em.GMMMachine)
	self._check_feature(probe)
	score = sum(model.log_likelihood(probe[i,:]) - self.ubm.log_likelihood(probe[i,:]) for i in range(probe.shape[0]))
	return score/probe.shape[0]

 Is this the function used to compute each score? If so why are you calculating the likelihood ratio of learned model with UBM for each probe? 

 Further the computed score is compared with a threshold (EER) to get correctly classified positives or correctly classified negatives.

foreach (k in positives) if positives[k] >= threshold: classified[k] = true else: classified[k] = false

  Is there a reference for this method of classification? 

 Say if there are 2 speakers to be identified and we learn 2 models (A,B) and then adapt based on UBM. My understanding is that an utterance is classified to belong to a model (A) if the likelihood of the utterance for the given model (A) is greater than the likelihood of the utterance for the other model (B). Is your method of classification a variation of the same technique?

Thanks,

Prasanna

[André Anjos]

Hello Prasanna,

The use of an universal background model (UBM) model is handy when you have not just A and B models, but a lot of models, such as in an realistic authentication system.

If you compare each individual to every other, then the number of comparisons scale exponentially with the number of models in the dataset. With an UBM you're trying assess the likelihood individual A looks more like individual A than mr. nobody (represented by the UBM).

As you'll see in many cases and dataset evaluations that this is enough to distinguish individuals uniquely while still normalising the scores a bit (by the UBM). The UBM-GMM system have less scaling issues than the other solution comparing each model to every other in the dataset. I hope that is clear.

If you have a problem in which you only have 2 classes, using an UBM makes no sense and the approach you suggested would be, therefore, a better solution.

As for the >= operation on the comparison of thresholds, it is an arbitrary decision in the software framework. It could be done in the other way as well. I'm not sure you'll find discussions or justifications for this.

Best, Andre

--
-- You received this message because you are subscribed to the Google Groups bob-devel group. To post to this group, send email to bob-...@googlegroups.com. To unsubscribe from this group, send email to bob-devel+unsubscribe@googlegroups.com. For more options, visit this group at https://groups.google.com/d/forum/bob-devel or directly the project website at http://idiap.github.com/bob/
---
You received this message because you are subscribed to the Google Groups "bob-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bob-devel+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Dr. André Anjos
Idiap Research Institute
Centre du Parc - rue Marconi 19
CH-1920 Martigny, Suisse
Phone: +41 27 721 7763
Fax: +41 27 721 7712
http://andreanjos.org