Re: [bob-devel] bob.learn.em.ML_GMMTrainer ERROR

[Tiago Freitas Pereira]

Hey Ziv,

First thing, check your input data.

Are you sure that you don't have any `NaN`,  of 'inf'?

Cheers

On Thu, Oct 6, 2016 at 4:54 PM, ziv rader <zivr...@gmail.com> wrote:

Hi,

I am trying to run a verification test with my own datasets (both for UBM, T-matrix training). I have made a protocol similar to the timit protocol with the following command line:

bin/verify.py -vv -d 'swb' -p 'energy-2gauss' -e 'mfcc-60' -a 'ivec-cosine-swb' -s 'ivec-cosine-swb' -v  --skip-preprocessing --skip-extraction

(I am using the skip options as my extracted features are already exist)

while preprocessing and feature extraction works OK, I get the following Error in the UBM training:

ERROR: During the execution, an exception was raised: bob.learn.em.ML_GMMTrainer - cannot perform the e_step method: C++ exception caught: 'logadd: minusdif (-nan) log_b (-nan) or log_a (-84.420370) is nan'

full log in  tha attached file.

can someone assist with the above or can help me get to the (probably) cpp file that generate the error message?

Thanks,

Ziv

--
-- You received this message because you are subscribed to the Google Groups bob-devel group. To post to this group, send email to bob-...@googlegroups.com. To unsubscribe from this group, send email to bob-devel+unsubscribe@googlegroups.com. For more options, visit this group at https://groups.google.com/d/forum/bob-devel or directly the project website at http://idiap.github.com/bob/
---
You received this message because you are subscribed to the Google Groups "bob-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bob-devel+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Tiago

[Manuel Günther]

After seeing repeating this type of discussion for the 1000th time now, we should perform some actions:

1. Provide a better exception message that people can understand, and that contains some actions, such as: "GMM training failed due to NaN values: check your inputs!" or alike

2. Check that you do not have NaN values after KMeans, i.e., to avoid that this error comes up in the first place, see: https://gitlab.idiap.ch/bob/bob.learn.em/issues/3

3. Do not produce NaN values in KMeans. AFAIK, most of the times, NaN values are produced, when the ``RANDOM`` initializer http://pythonhosted.org/bob.learn.em/py_api.html#bob.learn.em.KMeansTrainer is used (which is the **undocumented** default: https://gitlab.idiap.ch/bob/bob.learn.em/blob/master/bob/learn/em/include/bob.learn.em/KMeansTrainer.h#L43).

   To avoid this problem to happen that often, we should:

  3.1 Change the default to  ``RANDOM_NO_DUPLICATES``

  3.2 Document the default

[Tiago Freitas Pereira]

Good Manuel, lead us :-P 

Let's do that.
The point where the exception is raised is not good at all.
We can do some input tests and raise some exceptions before hand.

I will open the issues.

Thanks

--

-- You received this message because you are subscribed to the Google Groups bob-devel group. To post to this group, send email to bob-...@googlegroups.com. To unsubscribe from this group, send email to bob-devel+unsubscribe@googlegroups.com. For more options, visit this group at https://groups.google.com/d/forum/bob-devel or directly the project website at http://idiap.github.com/bob/
---
You received this message because you are subscribed to the Google Groups "bob-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bob-devel+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Tiago