Memory limitation error

Skip to first unread message

Prakash Pimpale

Dec 16, 2015, 7:27:45 AM12/16/15
Trying to run the pipeline using the recent Joshua version 6.0. I am getting following error. I have tried setting the all the Xmx values to 2g by default in joshua-6.0.5/scripts/training/ But that is also not helping.

I am on 4GB RAM ubuntu system. 

 cmd=/home/prakash/smt/joshua/joshua-6.0.5/scripts/training/ /home/prakash/smt/joshua/joshua-6.0.5/examples/data/tune/corpus.hi /home/prakash/smt/joshua/joshua-6.0.5/examples/data/tune/corpus.en --tunedir /home/prakash/smt/joshua/joshua-6.0.5/examples/tune --tuner mert --decoder /home/prakash/smt/joshua/joshua-6.0.5/examples/tune/decoder_command --decoder-config /home/prakash/smt/joshua/joshua-6.0.5/examples/tune/joshua.config --decoder-output-file /home/prakash/smt/joshua/joshua-6.0.5/examples/tune/output.nbest --decoder-log-file /home/prakash/smt/joshua/joshua-6.0.5/examples/tune/joshua.log --iterations 15 --metric 'BLEU 4 closest'
  JOB FAILED (return code 1)
Invalid maximum heap size: -Xmx4g
The specified size exceeds the maximum representable size.
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.
Traceback (most recent call last):
  File "/home/prakash/smt/joshua/joshua-6.0.5/scripts/training/", line 537, in <module>
  File "/home/prakash/smt/joshua/joshua-6.0.5/scripts/training/", line 520, in main
    run_zmert(opts.tunedir, opts.source,, opts.decoder, opts.decoder_config, opts.decoder_output_file, opts)
  File "/home/prakash/smt/joshua/joshua-6.0.5/scripts/training/", line 401, in run_zmert
    opts.metric, opts.iterations or 10)
  File "/home/prakash/smt/joshua/joshua-6.0.5/scripts/training/", line 383, in setup_configs
    for feature,weight in get_features(config):
  File "/home/prakash/smt/joshua/joshua-6.0.5/scripts/training/", line 335, in get_features
    output = check_output("%s/bin/joshua-decoder -c %s -show-weights -v 0" % (JOSHUA, config_file), shell=True)
  File "/usr/lib/python2.7/", line 544, in check_output
    raise CalledProcessError(retcode, cmd, output=output)

I am using this to run the pipeline:
$JOSHUA/bin/ --rundir .  --type hiero --tune input/t --test input/t --corpus input/t --source hi  --target en --lmfile english50k.lm.blm --no-corpus-lm --joshua-mem 2g

Please help.
Thank you.

प्रकाश बा. पिंपळे । Prakash B Pimpale 
मुंबई, महाराष्ट्र, भारत । Mumbai, Maharashtra, India
मला माहित नाही मी जग बदलू शकेल की नाही, पण त्यासाठी प्रयत्न करायचा निश्चय तरी मी केलाय!
I don't know if I can change the world, but I have taken a firm decision to give it a try!

Matt Post

Dec 16, 2015, 9:06:01 AM12/16/15
What version of Joshua do you have installed?

It looks like has hard-coded 4gb for MERT. Change that line in the file and see if that helps.


You received this message because you are subscribed to the Google Groups "Joshua Technical Support" group.
To unsubscribe from this group and stop receiving emails from it, send an email to
To post to this group, send email to
Visit this group at
For more options, visit

Dec 17, 2015, 11:58:47 PM12/17/15
to Joshua Technical Support
I am using joshua 6. I have changed all hardcoded 4g to 2g. But not worked. I think the value is being picked up from tune/joshua.config but there is no such value as 4g. Please suggest.

On Wednesday, December 16, 2015 at 7:36:01 PM UTC+5:30, Matt Post wrote:
What version of Joshua do you have installed?

It looks like has hard-coded 4gb for MERT. Change that line in the file and see if that helps

Prakash Pimpale

Dec 18, 2015, 2:05:13 AM12/18/15
to Joshua Technical Support
I figured out that and it was hard coded in $JOSHUA/bin/joshua-decoder

## memory usage; default is 4 GB

That particular error has went away. Thank you for help.
Reply all
Reply to author
0 new messages