Error getting feature dimension

[Jehad Yaqout]

Dear all ,

I have this error, any ideas ?

===== MONO TRAINING =====

steps/train_mono.sh --nj 1 --cmd run.pl data/train data/lang exp/mono

steps/train_mono.sh: Initializing monophone system.

feat-to-dim 'ark,s,cs:apply-cmvn --utt2spk=ark:data/train/split1/1/utt2spk scp:data/train/split1/1/cmvn.scp scp:data/train/split1/1/feats.scp ark:- | add-deltas ark:- ark:- |' - 

apply-cmvn --utt2spk=ark:data/train/split1/1/utt2spk scp:data/train/split1/1/cmvn.scp scp:data/train/split1/1/feats.scp ark:- 

WARNING (apply-cmvn[5.4.10~1-f629b]:Open():util/kaldi-table-inl.h:106) Failed to open script file data/train/split1/1/feats.scp

add-deltas ark:- ark:- 

ERROR (apply-cmvn[5.4.10~1-f629b]:SequentialTableReader():util/kaldi-table-inl.h:860) Error constructing TableReader: rspecifier is scp:data/train/split1/1/feats.scp

[ Stack-Trace: ]

apply-cmvn() [0x59fbca]

kaldi::MessageLogger::HandleMessage(kaldi::LogMessageEnvelope const&, char const*)

kaldi::MessageLogger::~MessageLogger()

kaldi::SequentialTableReader<kaldi::KaldiObjectHolder<kaldi::Matrix<float> > >::SequentialTableReader(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)

main

__libc_start_main

_start

ERROR (feat-to-dim[5.4.10~1-f629b]:main():feat-to-dim.cc:58) Could not read any features (empty archive?)

[ Stack-Trace: ]

feat-to-dim() [0x5511a4]

kaldi::MessageLogger::HandleMessage(kaldi::LogMessageEnvelope const&, char const*)

kaldi::MessageLogger::~MessageLogger()

main

__libc_start_main

_start

error getting feature dimension

[Daniel Povey]

it looks like he's running the Kaldi for Dummies tutorial and something went wrong earlier in the make_mfcc.sh stage.

On Tue, Feb 27, 2018 at 10:56 AM, Zoltán Somogyi <zsomo...@gmail.com> wrote:

This could be caused by the '--nj 1' parameter. Try to increase it to at least 2 and if that works then this is the problem. 

--
Go to http://kaldi-asr.org/forums.html find out how to join
---
You received this message because you are subscribed to the Google Groups "kaldi-help" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kaldi-help+unsubscribe@googlegroups.com.
To post to this group, send email to kaldi...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/kaldi-help/a7c88e94-8f1c-487d-b6cf-f16a6947c279%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Jehad Yaqout]

where i can find  '--nj 1' parameter ? 

[SaiKrishna Rallabandi]

Hi,

The warning might indicate the issue:

Failed to open script file data/train/split1/1/feats.scp

Check if data/train/split1/1/feats.scp exists. If it doesnt, there was an issue with feature extraction as Dan pointed out. If so, try running just the feature extraction step separately. 

Thanks,
Sai Krishna

On Wed, Feb 28, 2018 at 5:24 AM Zoltán Somogyi <zsomo...@gmail.com> wrote:

Since you do not know what nj is you most probably did not change it, therefore the problem will be probably somewhere else. Try to use the digits projects from the eggs folder instead of creating it yourself. You can then compare your files to the original digits files and find the problem.

--
Go to http://kaldi-asr.org/forums.html find out how to join
---
You received this message because you are subscribed to the Google Groups "kaldi-help" group.

To unsubscribe from this group and stop receiving emails from it, send an email to kaldi-help+...@googlegroups.com.

To post to this group, send email to kaldi...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/kaldi-help/0686faba-7184-4236-adee-08305dd9bb04%40googlegroups.com.

[Daniel Povey]

For everyone on the list, just a reminder: make note of who is replying as well as the content of the response.

Generally responses from me will be more accurate than those of others.

See my earlier response:

"it looks like he's running the Kaldi for Dummies tutorial and something went wrong earlier in the make_mfcc.sh stage."

On Wed, Feb 28, 2018 at 5:50 AM, SaiKrishna Rallabandi <saikrishnaral...@gmail.com> wrote:

Hi,

The warning might indicate the issue:

Failed to open script file data/train/split1/1/feats.scp

Check if data/train/split1/1/feats.scp exists. If it doesnt, there was an issue with feature extraction as Dan pointed out. If so, try running just the feature extraction step separately. 

Thanks,
Sai Krishna

On Wed, Feb 28, 2018 at 5:24 AM Zoltán Somogyi <zsomo...@gmail.com> wrote:

Since you do not know what nj is you most probably did not change it, therefore the problem will be probably somewhere else. Try to use the digits projects from the eggs folder instead of creating it yourself. You can then compare your files to the original digits files and find the problem.

On Wednesday, February 28, 2018 at 10:05:51 AM UTC+1, Jehad Yaqout wrote:

where i can find  '--nj 1' parameter ? 

On Tuesday, February 27, 2018 at 5:56:30 PM UTC+2, Zoltán Somogyi wrote:

This could be caused by the '--nj 1' parameter. Try to increase it to at least 2 and if that works then this is the problem. 

--
Go to http://kaldi-asr.org/forums.html find out how to join
---
You received this message because you are subscribed to the Google Groups "kaldi-help" group.

To unsubscribe from this group and stop receiving emails from it, send an email to kaldi-help+unsubscribe@googlegroups.com.

To post to this group, send email to kaldi...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kaldi-help/0686faba-7184-4236-adee-08305dd9bb04%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Go to http://kaldi-asr.org/forums.html find out how to join
---
You received this message because you are subscribed to the Google Groups "kaldi-help" group.

To unsubscribe from this group and stop receiving emails from it, send an email to kaldi-help+unsubscribe@googlegroups.com.

To post to this group, send email to kaldi...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/kaldi-help/CABN9n5PO8CbZxXtUTNq9E3Si_f_ug_2z8QkEWHDPvYcq6bpWQw%40mail.gmail.com.

[Zoltán Somogyi]

I of course agree that you will be much more helpful than anybody else! Just for your information: I had this the same issue with nj=1 before, that the feats.scp files were not created. This was the reason for my immediate response and of course the aim to help. From the further responses of the user I see now that most probably there must be an other reason why those files were not created in the former step, I will remove my answer above to not confuse others.

On Wednesday, February 28, 2018 at 7:37:28 PM UTC+1, Dan Povey wrote:

For everyone on the list, just a reminder: make note of who is replying as well as the content of the response.

Generally responses from me will be more accurate than those of others.

See my earlier response:

"it looks like he's running the Kaldi for Dummies tutorial and something went wrong earlier in the make_mfcc.sh stage."

On Wed, Feb 28, 2018 at 5:50 AM, SaiKrishna Rallabandi <saikrishnaral...@gmail.com> wrote:

Hi,

The warning might indicate the issue:

Failed to open script file data/train/split1/1/feats.scp

Check if data/train/split1/1/feats.scp exists. If it doesnt, there was an issue with feature extraction as Dan pointed out. If so, try running just the feature extraction step separately. 

Thanks,
Sai Krishna

On Wed, Feb 28, 2018 at 5:24 AM Zoltán Somogyi <zsomo...@gmail.com> wrote:

Since you do not know what nj is you most probably did not change it, therefore the problem will be probably somewhere else. Try to use the digits projects from the eggs folder instead of creating it yourself. You can then compare your files to the original digits files and find the problem.

On Wednesday, February 28, 2018 at 10:05:51 AM UTC+1, Jehad Yaqout wrote:

where i can find  '--nj 1' parameter ? 

On Tuesday, February 27, 2018 at 5:56:30 PM UTC+2, Zoltán Somogyi wrote:

This could be caused by the '--nj 1' parameter. Try to increase it to at least 2 and if that works then this is the problem. 

--
Go to http://kaldi-asr.org/forums.html find out how to join
---
You received this message because you are subscribed to the Google Groups "kaldi-help" group.

To unsubscribe from this group and stop receiving emails from it, send an email to kaldi-help+...@googlegroups.com.

To post to this group, send email to kaldi...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kaldi-help/0686faba-7184-4236-adee-08305dd9bb04%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Go to http://kaldi-asr.org/forums.html find out how to join
---
You received this message because you are subscribed to the Google Groups "kaldi-help" group.

To unsubscribe from this group and stop receiving emails from it, send an email to kaldi-help+...@googlegroups.com.

To post to this group, send email to kaldi...@googlegroups.com.

[Jehad Yaqout]

I'm not  running the Kaldi for Dummies tutorial  i'm running my model ... It's an Arabic language model. Could this affect in the feature extraction ?
c

[SaiKrishna Rallabandi]

Can you paste the exact command you are using and terminal output you are getting for the feature extraction phase? 

Feature extraction might fail if the data folder is not as expected. You can check this by running:

./utils/fix_data_dir.sh data/train      # (assuming utils directory is properly linked)

Thanks,

SaiKrishna Rallabandi

--
Go to http://kaldi-asr.org/forums.html find out how to join
---
You received this message because you are subscribed to the Google Groups "kaldi-help" group.

To unsubscribe from this group and stop receiving emails from it, send an email to kaldi-help+...@googlegroups.com.

To post to this group, send email to kaldi...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/kaldi-help/06b3cefa-8d2d-4d35-8b0e-0ae4eb133a66%40googlegroups.com.

[Jehad Yaqout]

first i converte the  wav fies from 8khz to 16khz and bit sample rate =16 using SOX

2- using some python scripts i write kaldi files(wav.scp,text,corups.txt,util2spk)

3- I made the dict files using arabic phoentizer (Buck walter phoentizer)

4- I copied steps and utils from wsj/s5 model

5- I copied the scripts path.sh, run.sh, cmd.sh from kaldi dummy tutiorals 

in link you will find a copy of my model  

I want to thank you all for your help and support

https://drive.google.com/open?id=15MXGb6n_cOTaXdz6u1OUTCTHfgozWolU

[SaiKrishna Rallabandi]

Hi,

I have looked at your folders. As Dan pointed out,  feature extraction was the one failing. It was because data/train/utt2spk was not in the expected format. 

Running utils/fix_data_dir.sh data/train echos the exact filaname(utt2spk in this case) and the issue. 

I couldnt see the python scripts(that you used to generate scp files) so I have regenerated the scp files using the commands below. I was able to obtain the MFCCs after  this. 

# Generate utt2spk

cut -d ' ' -f 1 data/train/text > utterances.train

cut -d ' ' -f 1 data/train/text > speakers.train

paste utterances.train speakers.train > data/train/utt2spk

# Generate spk2utt

./utils/utt2spk_to_spk2utt.pl data/train/utt2spk > data/train/spk2utt 

./utils/fix_data_dir.sh data/train  

Doing this way, you should be able to inspect the files  utterances.train and speakers.train at a later time. I have removed the spk2gender for simplicity. Once certain that the data folder is as expected, feature extraction should be easy. 

# Extract feats

steps/make_mfcc.sh --cmd "run.pl" --nj 20 data/train/ exp/make_mfcc mfcc     # nj is the number of jobs. u can make them 1 as in our original implementation. 

Modified version of your directory ( with mfccs ) is here:

https://drive.google.com/file/d/1NsvSEOyFb2qPyQFrQk9sz-hDHwgXlwgM/view?usp=sharing

Thanks,

SaiKrishna Rallabandi

--
Go to http://kaldi-asr.org/forums.html find out how to join
---
You received this message because you are subscribed to the Google Groups "kaldi-help" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kaldi-help+...@googlegroups.com.
To post to this group, send email to kaldi...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/kaldi-help/edfbf471-3cee-4249-b709-803a745eb7d3%40googlegroups.com.

[Jehad Yaqout]

Thank you very much for your help 

 but I wanted to ask why the utt2spk file format is different from the kaldi for dummies tutorial like in the first picture and I also run the model but I had an error " SRILM toolkit is probably not installed.

 Instructions: tools/install_srilm.sh " but i have installed srilm and run some other models . 

[SaiKrishna Rallabandi]

Hi,

On Sun, Mar 4, 2018 at 4:07 AM Jehad Yaqout <jehad....@gmail.com> wrote:

Thank you very much for your help 

 but I wanted to ask why the utt2spk file format is different from the kaldi for dummies tutorial like in the first picture

I dont understand your question enough to correctly answer this. May be someone else....

 

and I also run the model but I had an error " SRILM toolkit is probably not installed.

 Instructions: tools/install_srilm.sh " but i have installed srilm and run some other models . 

Are you sure SRILM is installed? Its not installed by just running the script. The tar file needs to be manually downloaded. 

 

To view this discussion on the web visit https://groups.google.com/d/msgid/kaldi-help/a456e2ef-fd97-44a6-8371-2f602890e1b4%40googlegroups.com.

[Jehad Yaqout]

Hi 

[Jehad Yaqout]

I've solved the approve error but i still don't understand why we change the format of utt2spk file and i have a new error  "decode.sh: no such file data/test/split1/1/feats.scp" , Any ideas ? 

and thank you 

[SaiKrishna Rallabandi]

I suggest you to go through either dummies tutorial or the yesno example first.

--

Go to http://kaldi-asr.org/forums.html find out how to join
---
You received this message because you are subscribed to the Google Groups "kaldi-help" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kaldi-help+...@googlegroups.com.
To post to this group, send email to kaldi...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/kaldi-help/df87126c-fdff-4dad-b9f8-7a3d248effbf%40googlegroups.com.

[Jehad Yaqout]

Ok