nnet3 accuracy report

[Sidifen Koe]

Hello,

I'm using nnet3 to train an BLSTM with 3 layers. Previously, I worked with nnet1 BLSTMs. To check how reasonable my hyperparameter settings are, I want to have a look at the loss on the training and CV set and the training and CV accuracy. So far, I've only found the file "accuracy.report" which reports me the following:

%Iter   duration        train_loss      valid_loss      difference
0       177,0   0,000100669     0,000704403     0,000603734
1       99,0    0,376504        0,366491        -0,010013
2       347,0   0,498716        0,472654        -0,026062
3       198,0   0,398601        0,39195 -0,006651
4       481,0   0,531031        0,524226        -0,006805
5       275,0   0,437358        0,411774        -0,025584
6       278,0   0,565058        0,553811        -0,011247
7       277,0   0,599436        0,572931        -0,026505
8       293,0   0,614637        0,599849        -0,014788
9       273,0   0,632607        0,610767        -0,02184
10      273,0   0,646197        0,619673        -0,026524
11      275,0   0,657724        0,62878 -0,028944
12      276,0   0,676096        0,636629        -0,039467
13      271,0   0,684351        0,64483 -0,039521
14      281,0   0,690492        0,64473 -0,045762

Now, I am a bit confused. The training loss and CV loss are both increasing instead of decreasing. In the nnet1 setup, both values were decreasing monotonically. Moreover, the names are confusing: On the one hand, the file is named "accuracy.report", on the other hand, the column headers are "train_loss" and "valid_loss". What are those values? Are they accuracies or loss values?

Where can I find more data useful for debugging?

[Daniel Povey]

Vijay, perhaps you could answer this?

In addition to the objective being confusingly named (or whatever the
issue is), I'd like to understand why the numbers are being printed
using the French locale or whatever it is. We should always use the
"C" locale for stuff like this.

Dan

> --
> Go to http://kaldi-asr.org/forums.html find out how to join
> ---
> You received this message because you are subscribed to the Google Groups
> "kaldi-help" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to kaldi-help+...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Vijayaditya Peddinti]

This seems to be wrong terminology which was not identified before as I never looked at the text files.

I did not set the locale in the script so it was probably picking up from the default environment settings. I will force set it in this particular script.

I will submit PR with these modifications.

--VIjay 

> email to kaldi-help+unsubscribe@googlegroups.com.

[Daniel Povey]

Normally the path.sh does
export LC_ALL=C
and generally speaking the scripts that would call the python script
would source path.sh, so I would have thought that it was not
necessary for python scripts to set the locale themselves. Sidifen,
from where was this called, were you working from a particular example
script?
Dan

>> > email to kaldi-help+...@googlegroups.com.

[Vijayaditya Peddinti]

This python script is directly called from the command line, and not from a wrapper shell script. So if path.sh was not sourced in the particular environment this can happen.

--Vijay

>> > email to kaldi-help+unsubscribe@googlegroups.com.

[Sidifen Koe]

Sorry for the late reply. The local of the script output is actually correct. I just replaced the point by comma using 'tr' to export the data into another program which uses german locale. I just copied here what I had in my clipboard and didn't noticed about that fact. Sorry for that inconvenience.

[Sidifen Koe]

So, the data in the text file are actually accuracy values? Where can I find the training and CV loss of the particular iterations? Are those values logged, too?

> email to kaldi-help+...@googlegroups.com.