Can't open the joshua.config.final file

[theresa...@gmail.com]

Hi,

I am trying to train a model on my own data. I successfully trained the bengali-english model from the tutorial, and I even finished training a chinese-english model from my own data. However now I have preprocessed my data in a different way and tried to train again, and I'm getting the following error (this specific message appeared after already running the command an earlier time, therefore it was able to skip the earlier steps. The first time yielded the same "No such file or directory" message, though). While there is an alias to the joshua.config.final file in all of my model directories, such as the one I trained on bengali from the tutorial, I can no longer find the original one. Any idea what could have happened to it?

Thanks!

-Theresa

Theresas-MacBook-Pro-2:zh-en theresabreiner$ $JOSHUA/bin/pipeline.pl --source zh --target en --type hiero --no-prepare --aligner berkeley --corpus input/train.zh-en --tune input/tune.zh-en --test input/test.zh-en --lm-gen berkeleylm

[source-numlines] cached, skipping...

[source-numlines] retrieved cached result =>    68351

[berkeley-aligner-chunk-0] cached, skipping...

[aligner-combine] cached, skipping...

[lm-sort-uniq] cached, skipping...

[berkeleylm] cached, skipping...

[compile-kenlm] cached, skipping...

[filter-tune] cached, skipping...

[glue-tune] cached, skipping...

[tune-bundle] cached, skipping...

[mert-1] rebuilding...

  dep=/Users/theresabreiner/Documents/MCIT/Spring2016/MachineTranslation/models/zh-en/input/tune.zh-en.zh 

  dep=/Users/theresabreiner/Documents/MCIT/Spring2016/MachineTranslation/models/zh-en/tune/joshua.config [CHANGED]

  dep=tune/model/grammar.filtered.gz.packed/slice_00000.source 

  dep=/Users/theresabreiner/Documents/MCIT/Spring2016/MachineTranslation/models/zh-en/tune/joshua.config.final [NOT FOUND]

  cmd=/Users/theresabreiner/Documents/MCIT/Spring2016/MachineTranslation/joshua-6.0.5/scripts/training/run_tuner.py /Users/theresabreiner/Documents/MCIT/Spring2016/MachineTranslation/models/zh-en/input/tune.zh-en.zh /Users/theresabreiner/Documents/MCIT/Spring2016/MachineTranslation/models/zh-en/input/tune.zh-en.en --tunedir /Users/theresabreiner/Documents/MCIT/Spring2016/MachineTranslation/models/zh-en/tune --tuner mert --decoder /Users/theresabreiner/Documents/MCIT/Spring2016/MachineTranslation/models/zh-en/tune/decoder_command --decoder-config /Users/theresabreiner/Documents/MCIT/Spring2016/MachineTranslation/models/zh-en/tune/joshua.config --decoder-output-file /Users/theresabreiner/Documents/MCIT/Spring2016/MachineTranslation/models/zh-en/tune/output.nbest --decoder-log-file /Users/theresabreiner/Documents/MCIT/Spring2016/MachineTranslation/models/zh-en/tune/joshua.log --iterations 15 --metric 'BLEU 4 closest'

^[[B  took 5135 seconds (1h25m35s)

[filter-test] cached, skipping...

[test-pack] cached, skipping...

[test-bundle-1] rebuilding...

  dep=/Users/theresabreiner/Documents/MCIT/Spring2016/MachineTranslation/models/zh-en/tune/joshua.config.final [NOT FOUND]

  dep=/Users/theresabreiner/Documents/MCIT/Spring2016/MachineTranslation/models/zh-en/data/test/grammar.packed/slice_00000.source [CHANGED]

  dep=/Users/theresabreiner/Documents/MCIT/Spring2016/MachineTranslation/models/zh-en/test/1/model/joshua.config [NOT FOUND]

  cmd=/Users/theresabreiner/Documents/MCIT/Spring2016/MachineTranslation/joshua-6.0.5/scripts/support/run_bundler.py --force --symlink --absolute --verbose -T /tmp /Users/theresabreiner/Documents/MCIT/Spring2016/MachineTranslation/models/zh-en/tune/joshua.config.final /Users/theresabreiner/Documents/MCIT/Spring2016/MachineTranslation/models/zh-en/test/1/model --copy-config-options '-top-n 300 -pop-limit 5000 -output-format "%i ||| %s ||| %f ||| %c" -mark-oovs false'  --pack-tm /Users/theresabreiner/Documents/MCIT/Spring2016/MachineTranslation/models/zh-en/data/test/grammar.packed --tm /Users/theresabreiner/Documents/MCIT/Spring2016/MachineTranslation/models/zh-en/data/tune/grammar.glue

  JOB FAILED (return code 2)

ERROR:root:ERROR: argument config: can't open '/Users/theresabreiner/Documents/MCIT/Spring2016/MachineTranslation/models/zh-en/tune/joshua.config.final': [Errno 2] No such file or directory: '/Users/theresabreiner/Documents/MCIT/Spring2016/MachineTranslation/models/zh-en/tune/joshua.config.final'

[Matt Post]

Hi Theresa,

It looks like either tuning failed or you deleted it. What does

ls -lR tune

show?

--
You received this message because you are subscribed to the Google Groups "Joshua Technical Support" group.
To unsubscribe from this group and stop receiving emails from it, send an email to joshua_suppor...@googlegroups.com.
To post to this group, send email to joshua_...@googlegroups.com.
Visit this group at https://groups.google.com/group/joshua_support.
For more options, visit https://groups.google.com/d/optout.

[theresa...@gmail.com]

Hi Matt,

I'm not sure what happened exactly but I think that perhaps tuning failed when I first tried to run it, and then when I tried to run it again it thought it had finished properly and so it continued from the following step, therefore it wasn't able to find the config file that should have been created. I tried deleting all of the files and starting fresh and let it run and it just finished a few minutes ago and it seems to be working perfectly now! 

Thanks for your response.

-Theresa

[Matt Post]

Okay, great to hear!

matt