Cannot execute binary file in [compile-kenlm] stage of pipeline

[rich.bra...@me.com]

Hi, 

I'm new to Joshua and so just trying to follow the quick start. The project builds fine with ant and JAVA_HOME and JOSHUA are both set. When running this:

$JOSHUA/bin/pipeline.pl --source bn --target en \

    --no-prepare --aligner berkeley \

    --corpus input/bn-en/tok/training.bn-en \

    --tune input/bn-en/tok/dev.bn-en \

    --test input/bn-en/tok/devtest.bn-en

I get this error:

[source-numlines] rebuilding...

  dep=/opt/joshua/models/bn-en/joshua-decoder-indian-parallel-corpora-1f235c9/bn-en/tok/training.bn-en.bn [CHANGED]

  cmd=cat /opt/joshua/models/bn-en/joshua-decoder-indian-parallel-corpora-1f235c9/bn-en/tok/training.bn-en.bn | wc -l

  took 0 seconds (0s)

[source-numlines] retrieved cached result => 20788

[berkeley-aligner-chunk-0] rebuilding...

  dep=alignments/0/word-align.conf [CHANGED]

  dep=/opt/joshua/models/bn-en/data/train/splits/corpus.bn.0 [CHANGED]

  dep=/opt/joshua/models/bn-en/data/train/splits/corpus.en.0 [CHANGED]

  dep=alignments/0/training.align [NOT FOUND]

  cmd=java -d64 -Xmx10g -jar /opt/joshua/joshua-v6.0.4/lib/berkeleyaligner.jar ++alignments/0/word-align.conf

  took 156 seconds (2m36s)

child finished

[aligner-combine] rebuilding...

  dep=alignments/0/training.align [CHANGED]

  dep=alignments/training.align [NOT FOUND]

  cmd=cat alignments/0/training.align > alignments/training.align

  took 0 seconds (0s)

[thrax-input-file] rebuilding...

  dep=/opt/joshua/models/bn-en/joshua-decoder-indian-parallel-corpora-1f235c9/bn-en/tok/training.bn-en.bn [CHANGED]

  dep=/opt/joshua/models/bn-en/joshua-decoder-indian-parallel-corpora-1f235c9/bn-en/tok/training.bn-en.en [CHANGED]

  dep=alignments/training.align [CHANGED]

  dep=/opt/joshua/models/bn-en/data/train/thrax-input-file [NOT FOUND]

  cmd=paste /opt/joshua/models/bn-en/joshua-decoder-indian-parallel-corpora-1f235c9/bn-en/tok/training.bn-en.bn /opt/joshua/models/bn-en/joshua-decoder-indian-parallel-corpora-1f235c9/bn-en/tok/training.bn-en.en alignments/training.align | perl -pe 's/\t/ ||| /g' | grep -v '()' | grep -v '||| \+$' > /opt/joshua/models/bn-en/data/train/thrax-input-file

  took 0 seconds (0s)

[thrax-run] rebuilding...

  dep=/opt/joshua/models/bn-en/data/train/thrax-input-file [CHANGED]

  dep=thrax-hiero.conf [CHANGED]

  dep=grammar.gz [NOT FOUND]

  cmd=hadoop/bin/hadoop jar /opt/joshua/joshua-v6.0.4/thrax/bin/thrax.jar -D mapred.child.java.opts='-Xmx2g' thrax-hiero.conf thrax > thrax.log 2>&1; rm -f grammar grammar.gz; hadoop/bin/hadoop fs -getmerge thrax/final/ grammar.gz

  took 107 seconds (1m47s)

[lm-sort-uniq] rebuilding...

  dep=/opt/joshua/models/bn-en/joshua-decoder-indian-parallel-corpora-1f235c9/bn-en/tok/training.bn-en.en [CHANGED]

  dep=/opt/joshua/models/bn-en/joshua-decoder-indian-parallel-corpora-1f235c9/bn-en/tok/training.bn-en.en.uniq [NOT FOUND]

  cmd=/opt/joshua/joshua-v6.0.4/scripts/training/scat /opt/joshua/models/bn-en/joshua-decoder-indian-parallel-corpora-1f235c9/bn-en/tok/training.bn-en.en | sort -u -T /tmp -S 2G | gzip -9n > /opt/joshua/models/bn-en/joshua-decoder-indian-parallel-corpora-1f235c9/bn-en/tok/training.bn-en.en.uniq

  took 0 seconds (0s)

[kenlm] rebuilding...

  dep=/opt/joshua/models/bn-en/joshua-decoder-indian-parallel-corpora-1f235c9/bn-en/tok/training.bn-en.en.uniq [CHANGED]

  dep=lm.gz [NOT FOUND]

  cmd=/opt/joshua/joshua-v6.0.4/bin/lmplz -o 5 -T /tmp -S 2G --verbose_header --text /opt/joshua/models/bn-en/joshua-decoder-indian-parallel-corpora-1f235c9/bn-en/tok/training.bn-en.en.uniq  | gzip -9n > lm.gz

  took 0 seconds (0s)

[compile-kenlm] rebuilding...

  dep=lm.gz [CHANGED]

  dep=lm.kenlm [NOT FOUND]

  cmd=/opt/joshua/joshua-v6.0.4/src/joshua/decoder/ff/lm/kenlm/build_binary lm.gz lm.kenlm

  JOB FAILED (return code 126)

/bin/bash: /opt/joshua/joshua-v6.0.4/src/joshua/decoder/ff/lm/kenlm/build_binary: cannot execute binary file

Other stuff that might be relevant:

1. Here's what uname -a returns: 

Linux localhost.localdomain 3.10.0-229.el7.x86_64 #1 SMP Fri Mar 6 11:36:42 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux

2. I installed java open jdk 1.7 via yum.

3. I didn't install hadoop - didn't think it was needed.

4. I'm in /opt/, but have chmod'd and opened up permissions for the user running this command.

5. Cannot execute binary seems to be something often associated with running 32bit on 64bit systems, but I can't see where I might have screwed that up.

Any pointers?

Thanks very much.

[Matt Post]

I would try rebuilding kenlm; it looks like you have an incompatible binary. You can do this with

    cd $JOSHUA/src/joshua/decoder/ff/lm/kenlm

    make clean

    cd $JOSHUA

    ant kenlm

matt (from my phone)

--
You received this message because you are subscribed to the Google Groups "Joshua Technical Support" group.
To unsubscribe from this group and stop receiving emails from it, send an email to joshua_suppor...@googlegroups.com.
To post to this group, send email to joshua_...@googlegroups.com.
Visit this group at http://groups.google.com/group/joshua_support.
For more options, visit https://groups.google.com/d/optout.