Core dumped when training nnet

[Xuerui Yang]

Hi,

I can not run neural network training this afternoon and the settings are just as before, the error is:

steps/nnet/train_faster_scheduler.sh: line 94: 34708 Aborted                 (core dumped) $train_tool --cross-validate=true --randomize=false --verbose=$verbose $train_tool_opts ${feature_transform:+ --feature-transform=$feature_transform} ${frame_weights:+ "--frame-weights=$frame_weights"} ${utt_weights:+ "--utt-weights=$utt_weights"} "$feats_cv" "$labels_cv" $mlp_best 2>> $log

And in iter00.initial.log:

ASSERTION_FAILED (nnet-train-fsmn-streams[5.4.147~1407-e9491]:ParametricRelu():cu-matrix.cc:1461) : 'src.NumCols() == this->NumCols()'

[ Stack-Trace: ]

nnet-train-fsmn-streams() [0x78c220]

kaldi::MessageLogger::HandleMessage(kaldi::LogMessageEnvelope const&, char const*)

kaldi::MessageLogger::~MessageLogger()

kaldi::KaldiAssertFailure_(char const*, char const*, int, char const*)

kaldi::CuMatrixBase<float>::ParametricRelu(kaldi::CuMatrixBase<float> const&, kaldi::CuVectorBase<float> const&, kaldi::CuVectorBase<float> const&)

kaldi::nnet1::ParametricRelu::PropagateFnc(kaldi::CuMatrixBase<float> const&, kaldi::CuMatrixBase<float>*)

kaldi::nnet1::Component::Propagate(kaldi::CuMatrixBase<float> const&, kaldi::CuMatrix<float>*)

kaldi::nnet1::Nnet::Propagate(kaldi::CuMatrixBase<float> const&, kaldi::CuMatrix<float>*)

main

__libc_start_main

_start

[Daniel Povey]

The code that crashed is not in the Kaldi master so you shouldn't be
asking us for help-- ask whoever created the pull request or branch
that you are working on, e.g. open an issue on their github page.

> --
> Go to http://kaldi-asr.org/forums.html find out how to join
> ---
> You received this message because you are subscribed to the Google Groups "kaldi-help" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to kaldi-help+...@googlegroups.com.
> To post to this group, send email to kaldi...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/kaldi-help/644b460b-86a1-4d59-8de0-df101fe904bd%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Xuerui Yang]

Thank you, I've solved this problem

在 2018年8月29日星期三 UTC+8下午11:42:49，Dan Povey写道：