Re: [kaldi-help] KALDI USING TIMIT (LOCALLY)

[Vijayaditya Peddinti]

Could you just try running the below command and report the result

'/home/kaldi/kaldi-trunk/egs/timit/s5/../../../tools/sph2pipe_v2.5/sph2pipe -f wav /home/kaldi/kaldi-trunk/egs/timit/data/TIMIT/TRAIN/DR5/MJRG0/SX16.wav

--VIjay

On Sun, Jan 10, 2016 at 1:27 PM, Senjam Shantirani <senj...@gmail.com> wrote:

Deal All,

I am new to Kaldi, and I wish to create acoustic models using Kaldi and TIMIT data.
Using google search I came to know that we can run it locally too, which I prefer.

1-  I edited the cmd.sh  and uncomment the following while commenting others:

#c) run it locally...
export train_cmd=run.pl
export decode_cmd=run.pl
export cuda_cmd=run.pl
export mkgraph_cmd=run.pl

2- in the run.sh file, my TIMIT data path is

timit=/home/kaldi/kaldi-trunk/egs/timit/data/TIMIT

3- Then I execute inside S5,  ./run.sh which gives the below:

Succeeded in formatting data.
============================================================================
         MFCC Feature Extration & CMVN for Training and Test set          
============================================================================
steps/make_mfcc.sh --cmd run.pl --nj 30 data/train exp/make_mfcc/train mfcc
steps/make_mfcc.sh: [info]: no segments file exists: assuming wav.scp indexed by utterance.
run.pl: 30 / 30 failed, log is in exp/make_mfcc/train/make_mfcc_train.*.log
root@shanti-Satellite-C650:/home/kaldi/kaldi-trunk/egs/timit/s5#


4- The log files says the following ERRORS:

ERROR (compute-mfcc-feats:Read4ByteTag():wave-reader.cc:74) WaveData: expected 4-byte chunk-name, got read errror
WARNING (compute-mfcc-feats:Read():feat/wave-reader.h:148) Exception caught in WaveHolder object (reading).
WARNING (compute-mfcc-feats:LoadCurrent():util/kaldi-table-inl.h:232) TableReader: failed to load object from '/home/kaldi/kaldi-trunk/egs/timit/s5/../../../tools/sph2pipe_v2.5/sph2pipe -f wav /home/kaldi/kaldi-trunk/egs/timit/data/TIMIT/TRAIN/DR5/MJRG0/SX16.wav |'
ERROR (compute-mfcc-feats:Value():util/kaldi-table-inl.h:142) TableReader: failed to load object from '/home/kaldi/kaldi-trunk/egs/timit/s5/../../../tools/sph2pipe_v2.5/sph2pipe -f wav /home/kaldi/kaldi-trunk/egs/timit/data/TIMIT/TRAIN/DR5/MJRG0/SX16.wav |' (to suppress this error, add the permissive (p, ) option to the rspecifier.
WARNING (compute-mfcc-feats:Close():kaldi-io.cc:446) Pipe /home/kaldi/kaldi-trunk/egs/timit/s5/../../../tools/sph2pipe_v2.5/sph2pipe -f wav /home/kaldi/kaldi-trunk/egs/timit/data/TIMIT/TRAIN/DR5/MJRG0/SX16.wav | had nonzero return status 256
ERROR (compute-mfcc-feats:Value():util/kaldi-table-inl.h:142) TableReader: failed to load object from '/home/kaldi/kaldi-trunk/egs/timit/s5/../../../tools/sph2pipe_v2.5/sph2pipe -f wav /home/kaldi/kaldi-trunk/egs/timit/data/TIMIT/TRAIN/DR5/MJRG0/SX16.wav |' (to suppress this error, add the permissive (p, ) option to the rspecifier.



5- in one of the discussions I saw the following comment by Povey.

This is because you don't have GridEngine installed locally.
You could try setting in the environment
train_cmd=run.pl
decode_cmd=run.pl
but be careful, as it may cause too many jobs to run on your machine.
Dan


I tried the setting of environment. But same error.

Please Help Shanti

--
You received this message because you are subscribed to the Google Groups "kaldi-help" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kaldi-help+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Senjam Shantirani]

Hi Vijayaditya,

I ran the command and got the below output:

Input file /home/kaldi/kaldi-trunk/egs/timit/data/TIMIT/TRAIN/DR5/MJRG0/SX16.wav is not a valid SPHERE file

I search about SPHERE file earlier but could not get clear information.

Thanking you,
Shanti

[Vijayaditya Peddinti]

Could you verify that the file home/kaldi/kaldi-trunk/egs/timit/data/TIMIT/TRAIN/DR5/MJRG0/SX16.wav is not corrupted. You could run the soxi command to do that. 

The output should look something like

soxi /export/corpora5/LDC/LDC93S1/timit/TIMIT/TRAIN/DR5/MJRG0/SX16.WAV

Input File     : '/export/corpora5/LDC/LDC93S1/timit/TIMIT/TRAIN/DR5/MJRG0/SX16.WAV' (sph)

Channels       : 1

Sample Rate    : 16000

Precision      : 16-bit

Duration       : 00:00:01.94 = 31028 samples ~ 145.444 CDDA sectors

File Size      : 63.1k

Bit Rate       : 260k

Sample Encoding: 16-bit Signed Integer PCM

--Vijay

--

[Senjam Shantirani]

Thanking for your kind and prompt reply, I ran the code and got the output below:

root@shanti-Satellite-C650:/home/kaldi/kaldi-trunk/egs/timit/s5# soxi  /home/kaldi/kaldi-trunk/egs/timit/data/TIMIT/TRAIN/DR5/MJRG0/SX16.wav

Input File     : '/home/kaldi/kaldi-trunk/egs/timit/data/TIMIT/TRAIN/DR5/MJRG0/SX16.wav'
Channels       : 1
Sample Rate    : 8000
Precision      : 16-bit
Duration       : 00:00:01.94 = 15514 samples ~ 145.444 CDDA sectors
File Size      : 31.1k
Bit Rate       : 128k

Sample Encoding: 16-bit Signed Integer PCM

Also in one of the comment given by Dan, he mentioned the following

http://sourceforge.net/p/kaldi/discussion/1355348/thread/f59290d9/?limit=25

Probably that error message is coming from the program sph2pipe, because
your data-preparation has been set up to expect sphere files not wav files.
Basically, your wav.scp is probably invalid, and should have the form
<key> <wav-filename>
instead of
<key> sph2pipe <ops> <wav-filename> |
Dan

I tried editing the timit_data_prep.sh at line number 99

 awk '{printf("%s '$sph2pipe' -f wav %s |\n", $1, $2);}' < ${x}_sph.scp > ${x}_wav.scp

as

  awk '{printf("%s %s |\n", $1, $2);}' < ${x}_sph.scp > ${x}_wav.scp

IT created lots of strange looking file in the S5 folder and still do not solve.

please help,

[Vijayaditya Peddinti]

The soxi output seems to show that you have wav files rather than sph files, as you already identified. Another important difference is that you are using 8Khz files rather than 16Khz files. IIRC results on TIMIT are all reported on 16KHz data. If you want to continue with 8KHz data you might not be able to replicate the results reported in RESULTS file.

Now coming back to the main issue, you would have check that wav.scp file has the 

<key> <wav-filename>

format. The change you made to the awk command seems sufficient.  Issues like  

"IT created lots of strange looking file in the S5 folder and still do not solve."

 would require some bash script debugging, which I can't help with remotely.

--Vijay

--

[Vijayaditya Peddinti]

On Sun, Jan 10, 2016 at 2:43 PM, Senjam Shantirani <senj...@gmail.com> wrote:

To run with sph, will sox be able to convert the wav to sph and 16KHz?

Increasing the sampling rate to 16KHz does not solve the problem as the higher harmonics would still be missing.

I would not recommend doing this. Also as you already have wav files it is not necessary to have sph files.

Just modify the wav.scp file to have the format specified in Dan's mail and you should be at least able to run the experiment. 

 

If so, will the output of sox be okay to run in Kaldi.

As long as the out is 16 bit PCM you are fine. Your output is already in this format so you don't even need to use sox. 

I will make sure that wav.scp is in the
<key> <wav-filename> format.

--
You received this message because you are subscribed to a topic in the Google Groups "kaldi-help" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/kaldi-help/q6ydwWJiNcA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to kaldi-help+...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

warm Regards,

Senjam Shantirani

============================

SpeecHWareNet (I) Pvt. Ltd

Technology Incubation Center,
IIT Guwahati, India
www.speechwarenet.com

============================

[Daniel Povey]

He would probably also have to edit conf/mfcc.conf by adding --sample-rate=8000 or something like that.

This is an example of where people try to 'improve' the LDC databases in-place and just end up confusing things.  I.e. your data is obviously not the original data released by LDC.

Dan

[Senjam Shantirani]

Thank you Vijayaditya. Thank you Dan.

I tried what is suggested. But still the error comes:
while it says that     " Data & Lexicon & Language Preparation   " is succeeded.
arpa2fst -
\data\
Processing 1-grams
Processing 2-grams
Connected 0 states without outgoing arcs.
fstisstochastic data/lang_test_bg/G.fst
0.000307754 -0.0435805

Succeeded in formatting data.


============================================================================
         MFCC Feature Extration & CMVN for Training and Test set          
============================================================================
steps/make_mfcc.sh --cmd run.pl --nj 30 data/train exp/make_mfcc/train mfcc
steps/make_mfcc.sh: [info]: no segments file exists: assuming wav.scp indexed by utterance.
run.pl: 30 / 30 failed, log is in exp/make_mfcc/train/make_mfcc_train.*.log
root@shanti-Satellite-C650:/home/kaldi/kaldi-trunk/egs/timit/s5#

Is there any way this can be solved?

Shanti

[Daniel Povey]

Please don't ask the list before investigating yourself for a while.  It shows a lack of respect for other peoples' time.

It looks like you didn't even look in the log files  exp/make_mfcc/train/make_mfcc_train.*.log.  

Most likely the answer is something that I already told you, which is to set --sample-rate=8000 in conf/mfcc.conf (create it if necessary).

--

[Senjam Shantirani]

I did added the sample-rate you mentioned and ran it.But the error keep coming.

I will spend some more time and try to solve it.

Thank you for your help.

Shanti

--
You received this message because you are subscribed to a topic in the Google Groups "kaldi-help" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/kaldi-help/q6ydwWJiNcA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to kaldi-help+...@googlegroups.com.

[Senjam Shantirani]

Thank you for the hint, it worked.

I wish to have the output of Phoneloop or Force Alignment as we have in Sphinx or HTK like the following:

HTK:
#!MLF!#
"*/TEST_DR1_FAKS0_SA1.rec"
0 5900000 SIL -4509.977539
5900000 8200000 SHE -1763.600830
8200000 10600000 HED -2109.827881
10600000 11700000 YEAH -968.743225
11700000 15000000 DARK -2760.698730
15000000 17900000 SOON -2342.449219
17900000 19800000 DIDNT -1655.165283
19800000 24000000 GREASY -3564.877930
24000000 28100000 WASH -3158.285889
28100000 32300000 WATER -3260.373047
32300000 34300000 ALL -1649.975586
34300000 38100000 YEAR -2663.035889
38100000 39500000 SIL -1040.035645

or Sphinx:

     SFrm  EFrm    SegAScr Word
        0     2     -27648 <s>
        3     9     -70172 <s>
       10    28    -408546 ONLY
       29    43    -546434 LAWYERS
       44    67    -468845 LOVE
       68    99    -776008 MILLIONAIRES
      100   103     -84155 </s>
      104   107     -37355 </s>
 Total score:    -2419163

Thanks in advance,

Shanti

[Daniel Povey]

The script steps/get_train_ctm.sh may help here.  It needs a directory with alignments- you can give it any directory where you trained a model.

[Senjam Shantirani]

I ran the script, but it seeks word-boundary.int, which I could not found in lang/phones. I tried creating them by setting position_dependent_phones = false , in prepare_lang.sh as in one of your post it was mentioned TIMIT was prepared without position dependent phones. Could you please suggest how to get this word_boundary.into and get this output format.

Thanking you,
Shanti

[Daniel Povey]

Oh-- you are using TIMIT.  TIMIT is weird, and I refuse to deal with any TIMIT-specific questions because I think it's not a useful corpus.

[Senjam Shantirani]

Thank you Dan, I will dig more and see. If possible, I will create own database and do it.

Thank you for your help.

Shanti.