Training iVector using UBM

[Abishek S]

I am trying to train an iVector  for Speaker Recognition via the bob wrapper for Kaldi. Upon passing the fubm as a parameter for iVEctor Training, I encounter the below error:

IndexError   Traceback (most recent call last)
<ipython-input-17-b9a9b37ef8f0> in <module>()
   num_gselect=20, ivector_dim=600,
   use_weights=False, 
   num_iters=5, min_post=0.025,
   num_samples_for_weights=3, posterior_scale=1.0)

~/miniconda3/envs/bob_py3/lib/python3.6/site- 
packages/bob/kaldi/ivector.py in ivector_train(feats, fubm, 
                                             ivector_extractor, num_gselect, 
                                             ivector_dim, use_weights, 
                                             num_iters, 
                                             min_post, 
                                             num_samples_for_weights, 
                                             posterior_scale)
    for i, utt in enumerate(feats):
     uttid = 'utt' + str(i)
     io.write_mat(f, utt, key=uttid.encode('utf-8'))

    # Initialize the i-vector extractor using the FGMM input

~/miniconda3/envs/bob_py3/lib/python3.6/site-packages/bob/kaldi/io.py                 
    in write_mat(file_or_fd, m, key)
    fd.write(struct.pack('I', m.shape[0]))  # rows
    fd.write(b'\04')
    fd.write(struct.pack('I', m.shape[1]))  # cols
    # Data,
    # m.tofile(fd, sep=b"") # binary

IndexError: tuple index out of range


The Python Code is shown below:
# Python bob.kaldi iVector 
import os
import numpy as np
import glob
import io
from subprocess import PIPE, Popen
from os.path import isfile
import tempfile
import logging
logger = logging.getLogger(__name__)

import bob
from bob.kaldi.ivector import ivector_train
from bob.kaldi.ivector import ivector_extract

#bob kaldi MFCCs
filename = "/home/abishek/Downloads/basic-speaker-impl/development_set/anthonyschaller-20071221-/wav/a0491.wav"
feats = bob.kaldi.mfcc_from_path(filename, channel=0, preemphasis_coefficient=0.97,
                                raw_energy=True, frame_length=25, frame_shift=10,
                                num_ceps=13, num_mel_bins=23, cepstral_lifter=22,
                                low_freq=20, high_freq=0, dither=1.0, snip_edges=True)

# ubm train 
ubmname = "/home/abishek/Downloads/iVector - Speaker Recog/ubm.model"
# bob.kaldi.ubm_train
dubm = bob.kaldi.ubm_train(feats, ubmname, num_threads=4, num_frames=200, min_gaussian_weight=0.0001, 
                num_gauss=32, num_gauss_init=0, num_gselect=30, num_iters_init=20, num_iters=4, 
                remove_low_count_gaussians=True)

fubmfile = "/home/abishek/Downloads/iVector - Speaker Recog/fubm.model"
# Diagonal UBM
fubm = bob.kaldi.ubm_full_train(feats, dubm, fubmfile, 
                                num_gselect=20, num_iters=4, 
                                min_gaussian_weight=0.0001)

#iVector Train
ivector_extractor = "/home/abishek/Downloads/iVector - Speaker Recog/iVecExt.model"
iVec_Model = bob.kaldi.ivector_train(feats, fubm, ivector_extractor, num_gselect=20, ivector_dim=600, use_weights=False, num_iters=5, min_post=0.025, 
                                     num_samples_for_weights=3, posterior_scale=1.0)

[Abishek S]

[Pavel Korshunov]

Hi,

It seems you are trying to build a UBM model using features extracted from a single audio file. It does not make much sense, as you need more than one audio file to build a UBM. 

I hope it helps.

Best,

Pavel

[Abishek S]

Hi

I have attached the python code for enrolling multiple wav files from a directory. I am getting the same "IndexError : tuple out of range" error

[Manuel Günther]

Dear Abishek,

I am afraid that you have to be more specific in what you try to achieve. Please tell us more about your problem, so that we have a chance to help you.

As Pavel pointed out, you read a single audio file (which is on your local disk and we do not have access to it), and try to build a UBM out of it. This won't be sufficient. Re-think your approach.

Manuel

[Abishek S]

Hi Manuel,

Thank you for your response. I apologise for not explaining my problem sufficiently.

I am attempting to train an iVector model for the task of Speaker Verification on a set of 50 speakers, all of whom who've had their 10 speech utterances recorded in English. 

- Based on my limited understanding of the bob.kaldi python wrapper, I extracted the MFCCs for 1 wav file and trained a UBM on it. I realise that a UBM should trained on a set of large speakers to capture speaker independent distribution of features.

If I need to train a UBM on multiple speakers, should I just concatenate the MFCCs extracted on multiple wav files in the "feats" parameter of the 

"bob.kaldi.ubm_train" function ?

Sincerely,

Abishek

[Manuel Günther]

Alright, I hope that my former colleagues from Idiap (who have implemented the bob.kaldi interface) now can help you better. I have absolutely no experience with bob.kaldi, so I cannot help you with this.

Manuel

[Amir Mohammadi]

I don't know much about Kaldi either. 

Would you consider using bob.ap, bob.learn.em, and bob.bio.spear instead of Kaldi? We have a lot of experience in there which we could help you with.

Best,

Amir

On Fri, Apr 27, 2018, 6:07 PM 'Manuel Günther' via bob-devel <bob-...@googlegroups.com> wrote:

Alright, I hope that my former colleagues from Idiap (who have implemented the bob.kaldi interface) now can help you better. I have absolutely no experience with bob.kaldi, so I cannot help you with this.

Manuel

--
-- You received this message because you are subscribed to the Google Groups bob-devel group. To post to this group, send email to bob-...@googlegroups.com. To unsubscribe from this group, send email to bob-devel+...@googlegroups.com. For more options, visit this group at https://groups.google.com/d/forum/bob-devel or directly the project website at http://idiap.github.com/bob/
---
You received this message because you are subscribed to the Google Groups "bob-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bob-devel+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[captain...@gmail.com]

Hi Abishek,

     I am doing speaker recognition by using bob.kaldi as well. I think you can extract MFCC feature for each .wav file, then concatenate each MFCC array to a big 

one, after that. put it as feats parameter to the bob.kaldi.ubm_train function.

     Have you made any progress on ivector training? I am a little confused with that, Can you share your steps, maybe code, about how to train ivector?

     Thank you.

[Abishek S]

Hi

I have switched to bob.bio.spear. I had no luck with the iVector training in bob.kaldi.

Sincerely,

Abishekl

[Abishek S]

Hi Amir,

Sorry for the late reply. 

Based on your recommendation I have switched to bob.bio.spear. 

I am getting the following error upon using the train_projector function under 

bob.bio.gmm.algorithm.IVector

<<<<Code>>>>>>>

######################################## i-Vector ##########################################################

# __init___

subspace_dimension_of_t = 3

iVector = bob.bio.gmm.algorithm.IVector(subspace_dimension_of_t, 

                                        tv_training_iterations=25, 

                                        update_sigma=True, 

                                        use_whitening=True, 

                                        use_lda=False, 

                                        use_wccn=False, 

                                        use_plda=False, 

                                        lda_dim=50, 

                                        plda_dim_F=50, 

                                        plda_dim_G=50, 

                                        plda_training_iterations=50,

                                       number_of_gaussians=2)

#------------------------------------------------------------------------------------------------------------

train_features = anthony_feats[0]

for i in range(1,10):

    train_features = np.vstack((train_features,anthony_feats[i]))

train_features = np.float64(train_features)

print(train_features.shape) #  Output : (1975, 39)

# Train_Projector and Enroller

spearUBM_iVec_path = "/home/abishek/Downloads/iVector-SpeakerRecog/spearUBM_iVec.model"

projector_file = bob.io.base.HDF5File(spearUBM_iVec_path,'w')

iVec_train_proj = iVector.train_projector(train_features, projector_file)

#----------------------------------------------------------------------------------------------------------

#**********************************************************************************************************

Output:

if not isinstance(feature, numpy.ndarray) or feature.ndim != 2 or feature.dtype != numpy.float64:

 raise ValueError("The given feature is not appropriate")

         ValueError: The given feature is not appropriate

#**********************************************************************************************************

>>> print(not isinstance(train_features, np.ndarray) or train_features.ndim != 2 or train_features.dtype != np.float64)

>>> False

#***********************************************************************************************************

Can you please expalin why this error is being shown even though the condition inside the if statement is false??

Moreover, the train_projector(train_features, projector_file)  function seems to be inconsistent with the kind of numpy array being used for

the train_features in bob.bio.gmm.algorithm.GMM  and bob.bio.gmm.algorithm.IVector

For bob.bio.gmm.algorithm.GMM it accepts a 3-D numpy array and for bob.bio.gmm.algorithm.IVector does not seem to work at all.

Sincerely,

Abishek

[Amir Mohammadi]

Hi,

Unless you attach some code and example data that I can run to reproduce your problem, I cannot tell you what is wrong exactly.

My suggestion is that you use ipdb https://github.com/gotcha/ipdb

(conda install ipdb) to debug your code. You can see what data is being entered in that if statement and why is it failing.

You can also look into this guide:
https://www.idiap.ch/software/bob/docs/bob/docs/stable/bob.learn.em/doc/guide.html

for a simple guide on how to train GMM, ivector, ...

Also, look into this file:
https://gitlab.idiap.ch/bob/bob.bio.base/blob/v3.3.0/bob/bio/base/tools/algorithm.py

to see how bob.bio.base trains algorithms.

Best,

Amir

[Abishek S]

Hi

Thank you for the reply. 

I managed to figure out how to go about it. 

I have attached the python code with this response. 

Kindly go through it.

I constantly get the message Aborted (core dumped) when I use the function train_projector(train_features,projector_file) 

Kindly look at the lines 122 and 123 in the attachment.

subspace_dimension_of_t = 39

iVector = bob.bio.gmm.algorithm.IVector(subspace_dimension_of_t, 

                                        tv_training_iterations=25, 

                                        update_sigma=True, 

                                        use_whitening=False, 

                                        use_lda=False, 

                                        use_wccn=False, 

                                        use_plda=True, 

                                        lda_dim=50, 

                                        plda_dim_F=2, 

                                        plda_dim_G=50, 

                                        plda_training_iterations=50,

                                       number_of_gaussians=2)

# Train_Projector and Enroller

spearUBM_iVec_path = "/home/ubuntu/abishek/UBM-train-iVector/2spk_iVec_PLDA.model"

# projector_file = bob.io.base.HDF5File(spearUBM_iVec_path,'w')

projector_file = spearUBM_iVec_path

iVec_train_proj = iVector.train_projector(train_features, projector_file)

When use_plda= True , PLDA set to true I  get the message Aborted (core dumped)

I realise it is a memory usage issue. 

Should I execute the training in parallel? 

Please tell me how to go about this.

Sincerely,

Abishek

[Tiago Freitas Pereira]

Hey Abishek,

The PLDA training is not distributed, so, aside from buying memories, there's nothing that you can do about it.

Sorry for that.

--
-- You received this message because you are subscribed to the Google Groups bob-devel group. To post to this group, send email to bob-...@googlegroups.com. To unsubscribe from this group, send email to bob-devel+unsubscribe@googlegroups.com. For more options, visit this group at https://groups.google.com/d/forum/bob-devel or directly the project website at http://idiap.github.com/bob/

---
You received this message because you are subscribed to the Google Groups "bob-devel" group.

To unsubscribe from this group and stop receiving emails from it, send an email to bob-devel+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Tiago

[Abishek S]

Hi

How much memory would you say is sufficient?

Sincerely,

Abishek

--
-- You received this message because you are subscribed to the Google Groups bob-devel group. To post to this group, send email to bob-...@googlegroups.com. To unsubscribe from this group, send email to bob-devel+...@googlegroups.com. For more options, visit this group at https://groups.google.com/d/forum/bob-devel or directly the project website at http://idiap.github.com/bob/

---
You received this message because you are subscribed to the Google Groups "bob-devel" group.

To unsubscribe from this group and stop receiving emails from it, send an email to bob-devel+...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Tiago