Tutorial for dummies. No segments file exists.

[Kostas Stogiannidis]

I am trying to run the tutorial with my own data and i get the following message. At feature extraction third row it says no segments file exists. Can anybody help me? Thak you in advance for your  time.

===== PREPARING ACOUSTIC DATA =====


===== FEATURES EXTRACTION =====

steps/make_mfcc.sh --nj 1 --cmd run.pl data/train exp/make_mfcc/train mfcc
utils/validate_data_dir.sh: Successfully validated data-directory data/train
steps/make_mfcc.sh: [info]: no segments file exists: assuming wav.scp indexed by utterance.
run.pl: job failed, log is in exp/make_mfcc/train/make_mfcc_train.1.log
steps/make_mfcc.sh --nj 1 --cmd run.pl data/test exp/make_mfcc/test mfcc
utils/validate_data_dir.sh: Successfully validated data-directory data/test
steps/make_mfcc.sh: [info]: no segments file exists: assuming wav.scp indexed by utterance.
run.pl: job failed, log is in exp/make_mfcc/test/make_mfcc_test.1.log
steps/compute_cmvn_stats.sh data/train exp/make_mfcc/train mfcc
make_cmvn.sh: no such file data/train/feats.scp
steps/compute_cmvn_stats.sh data/test exp/make_mfcc/test mfcc
make_cmvn.sh: no such file data/test/feats.scp

===== PREPARING LANGUAGE DATA =====

utils/prepare_lang.sh data/local/dict <UNK> data/local/lang data/lang
Checking data/local/dict/silence_phones.txt ...
--> reading data/local/dict/silence_phones.txt
--> data/local/dict/silence_phones.txt is OK

Checking data/local/dict/optional_silence.txt ...
--> reading data/local/dict/optional_silence.txt
--> data/local/dict/optional_silence.txt is OK

Checking data/local/dict/nonsilence_phones.txt ...
--> reading data/local/dict/nonsilence_phones.txt
--> data/local/dict/nonsilence_phones.txt is OK

Checking disjoint: silence_phones.txt, nonsilence_phones.txt
--> disjoint property is OK.

Checking data/local/dict/lexicon.txt
--> reading data/local/dict/lexicon.txt
--> data/local/dict/lexicon.txt is OK

Checking data/local/dict/extra_questions.txt ...
--> data/local/dict/extra_questions.txt is empty (this is OK)
--> SUCCESS [validating dictionary directory data/local/dict]

**Creating data/local/dict/lexiconp.txt from data/local/dict/lexicon.txt
fstaddselfloops data/lang/phones/wdisambig_phones.int data/lang/phones/wdisambig_words.int
prepare_lang.sh: validating output directory
utils/validate_lang.pl data/lang
Checking data/lang/phones.txt ...
--> data/lang/phones.txt is OK

Checking words.txt: #0 ...
--> data/lang/words.txt is OK

Checking disjoint: silence.txt, nonsilence.txt, disambig.txt ...
--> silence.txt and nonsilence.txt are disjoint
--> silence.txt and disambig.txt are disjoint
--> disambig.txt and nonsilence.txt are disjoint
--> disjoint property is OK

Checking sumation: silence.txt, nonsilence.txt, disambig.txt ...
--> summation property is OK

Checking data/lang/phones/context_indep.{txt, int, csl} ...
--> 10 entry/entries in data/lang/phones/context_indep.txt
--> data/lang/phones/context_indep.int corresponds to data/lang/phones/context_indep.txt
--> data/lang/phones/context_indep.csl corresponds to data/lang/phones/context_indep.txt
--> data/lang/phones/context_indep.{txt, int, csl} are OK

Checking data/lang/phones/nonsilence.{txt, int, csl} ...
--> 80 entry/entries in data/lang/phones/nonsilence.txt
--> data/lang/phones/nonsilence.int corresponds to data/lang/phones/nonsilence.txt
--> data/lang/phones/nonsilence.csl corresponds to data/lang/phones/nonsilence.txt
--> data/lang/phones/nonsilence.{txt, int, csl} are OK

Checking data/lang/phones/silence.{txt, int, csl} ...
--> 10 entry/entries in data/lang/phones/silence.txt
--> data/lang/phones/silence.int corresponds to data/lang/phones/silence.txt
--> data/lang/phones/silence.csl corresponds to data/lang/phones/silence.txt
--> data/lang/phones/silence.{txt, int, csl} are OK

Checking data/lang/phones/optional_silence.{txt, int, csl} ...
--> 1 entry/entries in data/lang/phones/optional_silence.txt
--> data/lang/phones/optional_silence.int corresponds to data/lang/phones/optional_silence.txt
--> data/lang/phones/optional_silence.csl corresponds to data/lang/phones/optional_silence.txt
--> data/lang/phones/optional_silence.{txt, int, csl} are OK

Checking data/lang/phones/disambig.{txt, int, csl} ...
--> 2 entry/entries in data/lang/phones/disambig.txt
--> data/lang/phones/disambig.int corresponds to data/lang/phones/disambig.txt
--> data/lang/phones/disambig.csl corresponds to data/lang/phones/disambig.txt
--> data/lang/phones/disambig.{txt, int, csl} are OK

Checking data/lang/phones/roots.{txt, int} ...
--> 22 entry/entries in data/lang/phones/roots.txt
--> data/lang/phones/roots.int corresponds to data/lang/phones/roots.txt
--> data/lang/phones/roots.{txt, int} are OK

Checking data/lang/phones/sets.{txt, int} ...
--> 22 entry/entries in data/lang/phones/sets.txt
--> data/lang/phones/sets.int corresponds to data/lang/phones/sets.txt
--> data/lang/phones/sets.{txt, int} are OK

Checking data/lang/phones/extra_questions.{txt, int} ...
--> 9 entry/entries in data/lang/phones/extra_questions.txt
--> data/lang/phones/extra_questions.int corresponds to data/lang/phones/extra_questions.txt
--> data/lang/phones/extra_questions.{txt, int} are OK

Checking data/lang/phones/word_boundary.{txt, int} ...
--> 90 entry/entries in data/lang/phones/word_boundary.txt
--> data/lang/phones/word_boundary.int corresponds to data/lang/phones/word_boundary.txt
--> data/lang/phones/word_boundary.{txt, int} are OK

Checking optional_silence.txt ...
--> reading data/lang/phones/optional_silence.txt
--> data/lang/phones/optional_silence.txt is OK

Checking disambiguation symbols: #0 and #1
--> data/lang/phones/disambig.txt has "#0" and "#1"
--> data/lang/phones/disambig.txt is OK

Checking topo ...

Checking word_boundary.txt: silence.txt, nonsilence.txt, disambig.txt ...
--> data/lang/phones/word_boundary.txt doesn't include disambiguation symbols
--> data/lang/phones/word_boundary.txt is the union of nonsilence.txt and silence.txt
--> data/lang/phones/word_boundary.txt is OK

Checking word-level disambiguation symbols...
--> data/lang/phones/wdisambig.txt exists (newer prepare_lang.sh)
Checking word_boundary.int and disambig.int
--> generating a 5 word sequence
--> resulting phone sequence from L.fst corresponds to the word sequence
--> L.fst is OK
--> generating a 45 word sequence
--> resulting phone sequence from L_disambig.fst corresponds to the word sequence
--> L_disambig.fst is OK

Checking data/lang/oov.{txt, int} ...
--> 1 entry/entries in data/lang/oov.txt
--> data/lang/oov.int corresponds to data/lang/oov.txt
--> data/lang/oov.{txt, int} are OK

--> data/lang/L.fst is olabel sorted
--> data/lang/L_disambig.fst is olabel sorted
--> SUCCESS [validating lang directory data/lang]

===== LANGUAGE MODEL CREATION =====
===== MAKING lm.arpa =====

Using SRILM language modelling tool from /home/kostas/kaldi/egs/digits/../../tools/srilm/bin/i686

===== MAKING G.fst =====

arpa2fst --disambig-symbol=#0 --read-symbol-table=data/lang/words.txt data/local/tmp/lm.arpa data/lang/G.fst
LOG (arpa2fst[5.1.46~1-0d031]:Read():arpa-file-parser.cc:96) Reading \data\ section.
LOG (arpa2fst[5.1.46~1-0d031]:Read():arpa-file-parser.cc:151) Reading \1-grams: section.
LOG (arpa2fst[5.1.46~1-0d031]:RemoveRedundantStates():arpa-lm-compiler.cc:355) Reduced num-states from 1 to 1

===== MONO TRAINING =====

steps/train_mono.sh --nj 1 --cmd run.pl data/train data/lang exp/mono
steps/train_mono.sh: Initializing monophone system.
feat-to-dim 'ark,s,cs:apply-cmvn --utt2spk=ark:data/train/split1/1/utt2spk scp:data/train/split1/1/cmvn.scp scp:data/train/split1/1/feats.scp ark:- | add-deltas ark:- ark:- |' -
add-deltas ark:- ark:-
apply-cmvn --utt2spk=ark:data/train/split1/1/utt2spk scp:data/train/split1/1/cmvn.scp scp:data/train/split1/1/feats.scp ark:-
WARNING (apply-cmvn[5.1.46~1-0d031]:Open():util/kaldi-table-inl.h:106) Failed to open script file data/train/split1/1/feats.scp
ERROR (apply-cmvn[5.1.46~1-0d031]:SequentialTableReader():util/kaldi-table-inl.h:876) Error constructing TableReader: rspecifier is scp:data/train/split1/1/feats.scp

[ Stack-Trace: ]

kaldi::MessageLogger::HandleMessage(kaldi::LogMessageEnvelope const&, char const*)
kaldi::MessageLogger::~MessageLogger()
kaldi::SequentialTableReader<kaldi::KaldiObjectHolder<kaldi::Matrix<float> > >::SequentialTableReader(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)
main
__libc_start_main
apply-cmvn() [0x8082321]

ERROR (feat-to-dim[5.1.46~1-0d031]:main():feat-to-dim.cc:58) Could not read any features (empty archive?)

[ Stack-Trace: ]

kaldi::MessageLogger::HandleMessage(kaldi::LogMessageEnvelope const&, char const*)
kaldi::MessageLogger::~MessageLogger()
main
__libc_start_main
feat-to-dim() [0x805e9d1]

error getting feature dimension

[nit...@almagu.com]

The "No segments file exists" portion is actually OK - your script can run without it. The problem is somewhere else - if you want more help you should attach exp/make_mfcc/train/make_mfcc_train.1.log.

I don't know what tutorial you're running, but the problem is probably to do with the fact that you gave the program wave files that don't exist. Look at wav.scp and see if the files written in the second column exist.

בתאריך יום רביעי, 7 ביוני 2017 בשעה 16:16:01 UTC+3, מאת Kostas Stogiannidis:

[sra...@simpragma.com]

Is it solved? Please let me know how?

[Peter Paton]

I'm also getting a similar issue, anyone found a solution here?

[Armando]

solution to what?

[Peter Paton]

This line:

WARNING (apply-cmvn[5.1.46~1-0d031]:Open():util/kaldi-table-inl.h:106) Failed to open script file data/train/split1/1/feats.scp

feats.scp isn't being created as it should be and when I check manually it is not found in split/1/

[Armando]

then there was an error before, you should have some log before, during the feature extraction stage that tell you why that file has not been created

[Peter Paton]

Exactly, in the make_mfcc_train.1.log I'm getting:

# compute-mfcc-feats --write-utt2dur=ark,t:exp/make_mfcc/train/utt2dur.1 --verbose=2 --config=conf/mfcc.conf scp,p:exp/make_mfcc/train/wav_train.1.scp ark:- | copy-feats --write-num-frames=ark,t:exp/make_mfcc/train/utt2num_frames.1 --compress=true ark:- ark,scp:/home/peter/Documents/kaldi/egs/digits/mfcc/raw_mfcc_train.1.ark,/home/peter/Documents/kaldi/egs/digits/mfcc/raw_mfcc_train.1.scp 

# Started at Tue Dec  8 13:00:20 GMT 2020

#

copy-feats --write-num-frames=ark,t:exp/make_mfcc/train/utt2num_frames.1 --compress=true ark:- ark,scp:/home/peter/Documents/kaldi/egs/digits/mfcc/raw_mfcc_train.1.ark,/home/peter/Documents/kaldi/egs/digits/mfcc/raw_mfcc_train.1.scp 

compute-mfcc-feats --write-utt2dur=ark,t:exp/make_mfcc/train/utt2dur.1 --verbose=2 --config=conf/mfcc.conf scp,p:exp/make_mfcc/train/wav_train.1.scp ark:- 

ERROR (compute-mfcc-feats[5.5.846~1-a7c42]:ComputeFeatures():feat/feature-common-inl.h:47) Waveform and config sample Frequency mismatch: 8000 .vs 16000 (use --allow-upsample=true option to allow  upsampling the waveform).

And then for each utterance I have for training:

[ Stack-Trace: ]

/home/peter/Documents/kaldi/src/lib/libkaldi-base.so(kaldi::MessageLogger::LogMessage() const+0x793) [0x7f5677e55183]

compute-mfcc-feats(kaldi::MessageLogger::LogAndThrow::operator=(kaldi::MessageLogger const&)+0x25) [0x556e5c6f5bc5]

compute-mfcc-feats(kaldi::OfflineFeatureTpl<kaldi::MfccComputer>::ComputeFeatures(kaldi::VectorBase<float> const&, float, float, kaldi::Matrix<float>*)+0x25c) [0x556e5c702efa]

compute-mfcc-feats(main+0xbc7) [0x556e5c6f3bb0]

/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3) [0x7f5677a4e0b3]

compute-mfcc-feats(_start+0x2e) [0x556e5c6f2f2e]

WARNING (compute-mfcc-feats[5.5.846~1-a7c42]:main():compute-mfcc-feats.cc:151) Failed to compute features for utterance jackson_0_0

ERROR (compute-mfcc-feats[5.5.846~1-a7c42]:ComputeFeatures():feat/feature-common-inl.h:47) Waveform and config sample Frequency mismatch: 8000 .vs 16000 (use --allow-upsample=true option to allow  upsampling the waveform).

I've been playing around seeing where to put that flag --allow-upsample=true flag in the initial call in run.sh, is it supposed to go somewhere else?

[Armando]

kaldi has good logs, they usually tell you what you are doing wrong. It is very explicit, isn"t it?

Waveform and config sample Frequency mismatch: 8000 .vs 16000

--allow-upsample=true

you are giving a file audio with a sampling rate 8KHz, the binary expects 16KHz

put

--allow-upsample=true

in conf/mfcc.conf

or put 

--sample-frequency=8000

or upsample your file yourself with sox

learn reading the logs, and print the binary command line to learn about its options

compute-mfcc-feats

[Peter Paton]

Yep exactly just figured it was in the mfcc.conf file, thanks!