Validation script not working while training.

[Haruo Con]

Hi Rico,

On running the following training script:

THEANO_FLAGS=mode=FAST_RUN,floatX=float32,device=$device,gpuarray.preallocate=0.8 python $nematus_home/nematus/train.py \

    --model $working_dir/model.npz \

    --datasets $data_dir/train.bpe.$src $data_dir/train.bpe.$trg \

    --valid_datasets $data_dir/dev.bpe.$src $data_dir/dev.bpe.$trg \

    --dictionaries $data_dir/train.bpe.$src.json $data_dir/train.bpe.$trg.json \

    --valid_script $script_dir/validate.sh \

    --dim_word 512 \

    --dim 1024 \

    --lrate 0.0001 \

    --optimizer adam \

    --maxlen 50 \

    --batch_size 80 \

    --valid_batch_size 40 \

    --validFreq 5000 \

    --dispFreq 2000 \

    --saveFreq 10000 \

    --sampleFreq 10000 \

    --tie_decoder_embeddings \

    --layer_normalisation \

    --dec_base_recurrence_transition_depth 8 \

    --enc_recurrence_transition_depth 4

 it keeps on showing following message :

....

INFO: Translated 960 sents

INFO: Translated 1000 sents

INFO: Translated 1000 sents in 63.33332705497742 sec. Speed 15.789475249451769 sents/sec

nematus/scripts/validate.sh: 44: nematus/scripts/validate.sh: cannot open nematus/scripts/../model/dev.bpe.en.output.dev: No such file

cat: nematus/scripts/../model/model.npz_best_bleu: No such file or directory

nematus/scripts/validate.sh: 49: nematus/scripts/validate.sh: cannot open nematus/scripts/../model/dev.bpe.en.output.postprocessed.dev: No such file

nematus/scripts/validate.sh: 1: nematus/scripts/validate.sh: cannot open nematus/scripts/../model/dev.bpe.en.output.postprocessed.dev: No such file

WARNING: Validation script output does not look like a score: Script location: 

BLEU = 

Because of this "model.npz" seem to store the first checkpoint ignoring the later checkpoints with better result.

The translation of valid dataset was successful according to the message shown above.

How can I fix this?

Regards,

Haru