BLEU scores per Epoch using multi_bleu.perl

[ksenja...@gmail.com]

Hi Rico,

we are currently trying to run the validate script as suggested in one of your samples. However, we are getting an error regarding the model.npz_best_bleu file (No such file or directory) which then leads to errors seen below:

cat: modelCE.npz_best_bleu: No such file or directory
Use of uninitialized value in division (/) at mosesdecoder/scripts/generic/multi-bleu.perl line 139, <STDIN> line 10002.
Use of uninitialized value in division (/) at mosesdecoder/scripts/generic/multi-bleu.perl line 139, <STDIN> line 10002.

We cannot find where this file is created but believe this might has to do with setting the parameters --validFreq  --saveFreq  --dispFreq  and --sampleFreq.
Our overall goal is to calculate the BLEU score for every epoch over a dev set.

Could you advice us on that?

Our training set has 90000 lines and the dev set 10000 lines.

Best regards,
Ksenja

[Rico Sennrich]

Hello Ksenja,

the first error message is harmless, since the file modelCE.npz_best_bleu will only be created after the first validation run. The other error message happens if your BLEU score is zero, which can happen if your translation is really bad, or if the translation of the dev set crashed for some reason. Check that you've correctly set the variables dev, ref, and prefix in validate.sh, and have a look at the file "$dev.output.postprocessed.dev" that should have been created. You can also execute the validate.sh script directly, or execute the commands in it individually, to get a better idea what is going wrong.

best wishes,
Rico

--
You received this message because you are subscribed to the Google Groups "Nematus Support" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nematus-suppo...@googlegroups.com.
To post to this group, send email to nematus...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/nematus-support/69727e92-6cb7-4468-a97c-8d53165f0873%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.