Joshua 6.0.5 KenLM Build Error

552 views
Skip to first unread message

lesman...@gmail.com

unread,
Nov 8, 2015, 10:59:54 AM11/8/15
to Joshua Technical Support
Dear Joshua devs,

First, congratulations for the 6.0.5 release. I'm very happy to see that the 6.0.5 release also publish the light runtime version. This is gonna be fun to work with.

So, I try to build the Joshua 6.0.5 release, but I find an error message regarding the query, lmplz, and build_binary unable to be copied because unable to be found.
At first, I ignore it and try to work through the pipeline. But, as I suspected, since I know that lmplz and build_binary is crucial part of one of the pipeline step, 
it stop at the after the [lm-sort-uniq] phase. 

Here is the error message when I build Joshua. Please note the "No such file or directory" messages.
=============================================================
root@tesis:/datadrive/tools/joshua-6.0.5# ant -f $JOSHUA/build.xml kenlm
Buildfile: /datadrive/tools/joshua-6.0.5/build.xml

check-joshua-home:
     [echo] JOSHUA = /datadrive/tools/joshua-6.0.5 basedir = /datadrive/tools/joshua-6.0.5

kenlm:
     [exec] -- Boost version: 1.54.0
     [exec] -- Found the following Boost libraries:
     [exec] --   program_options
     [exec] --   system
     [exec] --   thread
     [exec] --   unit_test_framework
     [exec] -- Configuring done
     [exec] -- Generating done
     [exec] -- Build files have been written to: /datadrive/tools/joshua-6.0.5/src/kenlm/build
     [exec] [ 35%] Built target kenlm_util
     [exec] Linking CXX executable ../bin/bit_packing_test
     [exec] CMakeFiles/kenlm_util.dir/read_compressed.cc.o: In function `util::(anonymous namespace)::StreamCompressed<util::(anonymous namespace)::GZip>::~StreamCompressed()':
     [exec] read_compressed.cc:(.text+0x219): undefined reference to `inflateEnd'
     [exec] CMakeFiles/kenlm_util.dir/read_compressed.cc.o: In function `util::(anonymous namespace)::StreamCompressed<util::(anonymous namespace)::GZip>::~StreamCompressed()':
     [exec] read_compressed.cc:(.text+0x439): undefined reference to `inflateEnd'
     [exec] CMakeFiles/kenlm_util.dir/read_compressed.cc.o: In function `util::(anonymous namespace)::ReadFactory(int, unsigned long&, void const*, unsigned long, bool)':
     [exec] read_compressed.cc:(.text+0xa49): undefined reference to `inflateInit2_'
     [exec] CMakeFiles/kenlm_util.dir/read_compressed.cc.o: In function `util::(anonymous namespace)::StreamCompressed<util::(anonymous namespace)::GZip>::Read(void*, unsigned long, util::ReadCompressed&)':
     [exec] read_compressed.cc:(.text+0xe8f): undefined reference to `inflate'
     [exec] collect2: error: ld returned 1 exit status
     [exec] make[2]: *** [bin/bit_packing_test] Error 1
     [exec] make[1]: *** [util/CMakeFiles/bit_packing_test.dir/all] Error 2
     [exec] make: *** [all] Error 2
     [exec] cp: cannot stat ‘bin/query’: No such file or directory
     [exec] cp: cannot stat ‘bin/lmplz’: No such file or directory
     [exec] cp: cannot stat ‘bin/build_binary’: No such file or directory
     [exec] g++: error: lm/CMakeFiles/kenlm.dir/*.o: No such file or directory
     [exec] Result: 1

BUILD SUCCESSFUL
=================================================

And, here is the error message from when I invoke the pipeline using this command : 

$JOSHUA/bin/pipeline.pl --type hiero --corpus input/train --tune input/tune --test input/test --source en --target id --tmp /datadrive/tmp --joshua-mem 10g --hadoop-mem 42g --threads 8

===========================================================
[lm-sort-uniq] cached, skipping...
* FATAL: /datadrive/tools/joshua-6.0.5/bin/lmplz (for building LMs) does not exist.
  This is often a problem with the boost libraries (particularly threaded
  versus unthreaded).
===========================================================

I try to work my way around it by adding  "--lm-gen berkeleylm --buildlm-mem 20g" to the pipeline command, but as I suspected, stop at the [compile-kenlm] phase with this error message

============================================================
[compile-kenlm] rebuilding...
  dep=lm.gz [CHANGED]
  dep=lm.kenlm [NOT FOUND]
  cmd=/datadrive/tools/joshua-6.0.5/bin/build_binary lm.gz lm.kenlm
  JOB FAILED (return code 127)
/bin/bash: /datadrive/tools/joshua-6.0.5/bin/build_binary: No such file or directory
============================================================


I already try to build Joshua using the latest source code from Github, but it seems the error is still the same, unable to copy those kenlm lib items.

Here is my setup :
  • Ubuntu Server 14.04 LTS
  • OpenJDK 7
  • Installing Boost from apt-get -> apt-get install libboost-all-dev
  • Joshua 6.0.5 (both from the tarball and the source code, same error)

Looking forward for anyone to help me out here. Thanks for your time.

Regards,
Reza Lesmana

Matt Post

unread,
Nov 8, 2015, 9:50:04 PM11/8/15
to joshua_...@googlegroups.com
It looks like cmake is not finding the zlib libraries. Do you have these installed? What system are you on?


--
You received this message because you are subscribed to the Google Groups "Joshua Technical Support" group.
To unsubscribe from this group and stop receiving emails from it, send an email to joshua_suppor...@googlegroups.com.
To post to this group, send email to joshua_...@googlegroups.com.
Visit this group at http://groups.google.com/group/joshua_support.
For more options, visit https://groups.google.com/d/optout.

Reza Lesmana

unread,
Nov 9, 2015, 12:32:50 AM11/9/15
to Joshua Technical Support
Hi, Matt

Yes, I think I have the zlib installed. I'm on Ubuntu 14.04 Server LTS, installed as VM on Azure.

=================================================
root@tesis:/home/reza# apt-get install zlib1g-dev
Reading package lists... Done
Building dependency tree
Reading state information... Done
zlib1g-dev is already the newest version.
0 upgraded, 0 newly installed, 0 to remove and 4 not upgraded.
=================================================

Matt Post

unread,
Nov 9, 2015, 9:39:31 AM11/9/15
to joshua_...@googlegroups.com
Is it in your LD_LIBRARY_PATH? Where is libz.so?

I do not encounter this problem on any of the four systems I have tried building kenlm on, so unfortunately, there's not much I can say. You might try downloading KenLM directly (github.com/kpu/kenlm) and compiling it there (same cmake build system), and seeing if you have the same problem. If you don't, then there's something wrong with Joshua, but if you do, then probably there's a fix needed for their build system. Please do keep us updated.

matt

Reza Lesmana

unread,
Nov 9, 2015, 9:01:26 PM11/9/15
to Joshua Technical Support
Hi, Matt

I don't know about LD_LIBRARY_PATH, and LD_LIBRARY_PATH is not in my environment variables. 

According to this : http://joshua-decoder.org/6.0/faq.html , I should set LD_LIBRARY_PATH to my boost library directory. But, BOOST_ROOT environment variable should be enough, right? And according to the kenlm build log, it's able to find my boost lib directory, so it shouldn't be a problem. 

Anyway, I try to set LD_LIBRARY_PATH to my boost library directory, but kenlm build still has the same error. Please let me know if I did something wrong.

I try to find my libz.so, and the path is : 

/lib/x86_64-linux-gnu/libz.so.1.2.8

What should I do with this libz.so path?

I try to build kenlm in two locations. First, the kenlm source on joshua-6.0.5 ( $JOSHUA/src/kenlm ), and it succeeded to build those kenlm lib items I'm missing in Joshua KenLM build. Second, I clone the kenlm git source, and then build it. The same result, succeeded to build those kenlm lib items. I build both kenlm using ./bjam command (as stated in the BUILDING file). If I succeeded to build the kenlm from the git source, can I just copy those kenlm lib items? 

If you want an access to my server, please let me know, I'll give you the information. 

Regards,
Reza Lesmana

Reza Lesmana

unread,
Nov 9, 2015, 9:09:22 PM11/9/15
to Joshua Technical Support
Hi, Matt

I try to copy the build_binary, lmplz, and query I've succeeded to build from kenlm git source, to $JOSHUA/bin folder. And it seems that everything works OK.

Will let you know more soon.

Regards,
Reza Lesmana

Matt Post

unread,
Nov 10, 2015, 1:04:04 PM11/10/15
to joshua_...@googlegroups.com
Yes, please let me know more. At the very least, the sequence of commands you used to download and compile KenLM from its source would be useful to help us update KenLM inside Joshua.

matt

Reza Lesmana

unread,
Nov 11, 2015, 4:20:17 AM11/11/15
to Joshua Technical Support
Hi, Matt

Sorry for the late reply. 

I'm only using this to download and compile KenLM from its source :

$>clone https://github.com/kpu/kenlm.git
$>cd kenlm
$>./bjam


That's it. I've managed to get the lmplz, build_binary, and query, and then copy it to $JOSHUA/bin folder. 

The pipeline goes through until [mert] phase :

===============================
[mert-1] rebuilding...
  dep=/datadrive/workdir_baseline/data/tune/corpus.en [CHANGED]
  dep=/datadrive/workdir_baseline/tune/joshua.config [CHANGED]
  dep=tune/model/grammar.filtered.gz.packed/slice_00000.source [CHANGED]
  dep=/datadrive/workdir_baseline/tune/joshua.config.final [NOT FOUND]
  cmd=/datadrive/tools/joshua-6.0.5/scripts/training/run_tuner.py /datadrive/workdir_baseline/data/tune/corpus.en /datadrive/workdir_baseline/data/tune/corpus.id --tunedir /datadrive/workdir_baseline/tune --tuner mert --decoder /datadrive/workdir_baseline/tune/decoder_command --decoder-config /datadrive/workdir_baseline/tune/joshua.config --decoder-output-file /datadrive/workdir_baseline/tune/output.nbest --decoder-log-file /datadrive/workdir_baseline/tune/joshua.log --iterations 15 --metric 'BLEU 4 closest'
  JOB FAILED (return code 1)
* FATAL: Can't find libken.so (libken.dylib on OS X) in $JOSHUA/lib
*        This probably means that the KenLM library didn't compile.
*        Make sure that BOOST_ROOT is set to the root of your boost
*        installation (it's not /opt/local/, the default), change to
*        $JOSHUA, and type 'ant kenlm'. If problems persist, see the
*        website (joshua-decoder.org).
Traceback (most recent call last):
  File "/datadrive/tools/joshua-6.0.5/scripts/training/run_tuner.py", line 537, in <module>
    main(sys.argv)
  File "/datadrive/tools/joshua-6.0.5/scripts/training/run_tuner.py", line 520, in main
    run_zmert(opts.tunedir, opts.source, opts.target, opts.decoder, opts.decoder_config, opts.decoder_output_file, opts)
  File "/datadrive/tools/joshua-6.0.5/scripts/training/run_tuner.py", line 401, in run_zmert
    opts.metric, opts.iterations or 10)
  File "/datadrive/tools/joshua-6.0.5/scripts/training/run_tuner.py", line 383, in setup_configs
    for feature,weight in get_features(config):
  File "/datadrive/tools/joshua-6.0.5/scripts/training/run_tuner.py", line 335, in get_features
    output = check_output("%s/bin/joshua-decoder -c %s -show-weights -v 0" % (JOSHUA, config_file), shell=True)
  File "/usr/lib/python2.7/subprocess.py", line 573, in check_output
    raise CalledProcessError(retcode, cmd, output=output)
subprocess.CalledProcessError: Command '/datadrive/tools/joshua-6.0.5/bin/joshua-decoder -c /datadrive/workdir_baseline/tune/joshua.config -show-weights -v 0' returned non-zero exit status 1
==================================================================

Apparently, I should generate the libken.so too. But, using ./bjam is not generating the libken.so file. How should I generate this file when I want to build KenLM separately?

Btw, it seems other user also generate the same problem, but it seems 6.0.5 release should resolve this problem (reference : https://github.com/joshua-decoder/joshua/issues/225 ). But, apparently in my case it's not. My cmake is newest version. I also try on Ubuntu 15.10 server, and it's the same problem. 

Matt, I'm gonna send you an email to access my server and test it out. Is it okay for you?

Regards,
Reza Lesmana

Matt Post

unread,
Nov 11, 2015, 7:16:10 AM11/11/15
to joshua_...@googlegroups.com
Matt, I'm gonna send you an email to access my server and test it out. Is it okay for you?


Sure, that would be a simple way to resolve this. Email me offline and we can set this up.


Matt Post

unread,
Nov 11, 2015, 7:20:38 AM11/11/15
to joshua_...@googlegroups.com
Okay, you are using the bjam build system in KenLM. We can't include that in Joshua because it is not robust enough. You could try using the cmake instructions within KenLM to see if that works.

If you look in jni/build_kenlm.sh, you see there is a command that builds the KenLM library and places it in lib/. You could try running that command manually, but if all the object files have not compiled, it's not going to work properly.

matt

joa.b...@gmail.com

unread,
Nov 19, 2015, 2:42:24 PM11/19/15
to Joshua Technical Support
Hi,

I've been experiencing the same problems as Reza, and having compiled KenLM myself, I'm stuck at the point where Joshua says libken.so is missing in $JOSHUA/lib.
What exactly has been your solution for this?

By the way, part of the reason Joshua fails to build KenLM is that no *.o files are present in $JOSHUA/src/kenlm/build/lm/CMakeFiles/kenlm.dir (that is after manually adding the KenLM binaries to $JOSHUA/lib). These files are required by build_kenlm.sh.

Best wishes,
Joachim

Matt Post

unread,
Nov 19, 2015, 2:58:13 PM11/19/15
to joshua_...@googlegroups.com
Hi,

Those *.o files should be *produced* by build_kenlm.sh (via the call to make).

What OS are you working on? Can you tell me exactly what commands you typed after downloading Joshua?

matt


joa.b...@gmail.com

unread,
Nov 19, 2015, 6:44:36 PM11/19/15
to Joshua Technical Support
Thanks for your quick answer! I've tried it on Ubuntu 14.04 and 15.04. On both systems, I followed the install instructions at http://joshua-decoder.org/6.0/install.html and ran the commands

$ export JAVA_HOME=...
$ wget
-q http://cs.jhu.edu/~post/files/joshua-runtime-6.0.5.tgzwget
$ tar xzf joshua
-runtime-6.0.5.tgz
$ cd joshua
-runtime-6.0.5
$
export JOSHUA=$PWD
$ ant


... and when I realised KenLM did not install properly, I also ran

$ ant kenlm

... wherupon I copied the KenLM executables from a separate KenLM installation into $JOSHUA/bin, but to no avail. No I'm stuck where Joshua wants to see libken.so

Thank you very much for helping out!

Matt Post

unread,
Nov 20, 2015, 10:20:50 AM11/20/15
to joshua_...@googlegroups.com
Can you send me the output of

cd $JOSHUA
bash ./jni/build_kenlm.sh

matt

joa.b...@gmail.com

unread,
Nov 20, 2015, 1:18:50 PM11/20/15
to Joshua Technical Support
The output is

-- Boost version: 1.54.0

-- Found the following Boost libraries:
--   program_options
--   system
--   thread
--   unit_test_framework
-- Configuring done
-- Generating done
-- Build files have been written to: /home/joachim/nlp/joshua-6.0.5/src/kenlm/build
[ 35%] Built target kenlm_util
Linking CXX executable ../bin/bit_packing_test
CMakeFiles/kenlm_util.dir/read_compressed.cc.o: In function `util::(anonymous namespace)::StreamCompressed<util::(anonymous namespace)::GZip>::~StreamCompressed()':

read_compressed.cc:(.text+0x219): undefined reference to `
inflateEnd'

CMakeFiles/kenlm_util.dir/read_compressed.cc.o: In function `util::(anonymous namespace)::StreamCompressed<util::(anonymous namespace)::GZip>::~StreamCompressed()'
:

read_compressed
.cc:(.text+0x439): undefined reference to
`inflateEnd'
CMakeFiles/kenlm_util.dir/read_compressed.cc.o: In function `
util::(anonymous namespace)::ReadFactory(int, unsigned long&, void const*, unsigned long, bool)':

read_compressed.cc:(.text+0xa49): undefined reference to `inflateInit2_'

CMakeFiles/kenlm_util.dir/read_compressed.cc.o: In function `util::(anonymous namespace)::StreamCompressed<util::(anonymous namespace)::GZip>::Read(void*, unsigned long, util::ReadCompressed&)':

read_compressed.cc:(.text+0xe8f): undefined reference to `
inflate'

collect2: error: ld returned 1 exit status
make[2]: *** [bin/bit_packing_test] Error 1
make[1]: *** [util/CMakeFiles/bit_packing_test.dir/all] Error 2
make: *** [all] Error 2
cp: cannot stat ‘bin/query’: No such file or directory
cp: cannot stat ‘bin/lmplz’: No such file or directory
cp: cannot stat ‘bin/build_binary’: No such file or directory
g++: error: lm/CMakeFiles/kenlm.dir/*.o: No such file or directory


After copying the KenLM excutables to $JOSHUA/src/kenlm/build/bin, the 'cp: cannot stat...' errors obviously disappear, but the last line remains.
 

Matt Post

unread,
Nov 20, 2015, 4:16:41 PM11/20/15
to joshua_...@googlegroups.com
What happens if you do this?

cd $JOSHUA/src
rm -rf kenlm
cd $JOSHUA
bash jni/build_kenlm.sh

matt


joa.b...@gmail.com

unread,
Nov 21, 2015, 4:31:48 PM11/21/15
to Joshua Technical Support
Same error as before... Here's the full output. Does this make any sense to you?
Thanks!

-- The C compiler identification is GNU 4.9.2
-- The CXX compiler identification is GNU 4.9.2
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/g++
-- Check for working CXX compiler: /usr/bin/g++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Boost version: 1.55.0

-- Found the following Boost libraries:
--   program_options
--   system
--   thread
--   unit_test_framework
-- Configuring done
-- Generating done
-- Build files have been written to: /home/joachim/tools/joshua-6.0.5/src/kenlm/build
Scanning dependencies of target kenlm_util
[  1%] Building CXX object util/CMakeFiles/kenlm_util.dir/double-conversion/bignum-dtoa.cc.o
[  2%] Building CXX object util/CMakeFiles/kenlm_util.dir/double-conversion/bignum.cc.o
[  3%] Building CXX object util/CMakeFiles/kenlm_util.dir/double-conversion/cached-powers.cc.o
[  5%] Building CXX object util/CMakeFiles/kenlm_util.dir/double-conversion/diy-fp.cc.o
[  6%] Building CXX object util/CMakeFiles/kenlm_util.dir/double-conversion/double-conversion.cc.o
[  7%] Building CXX object util/CMakeFiles/kenlm_util.dir/double-conversion/fast-dtoa.cc.o
[  8%] Building CXX object util/CMakeFiles/kenlm_util.dir/double-conversion/fixed-dtoa.cc.o
[ 10%] Building CXX object util/CMakeFiles/kenlm_util.dir/double-conversion/strtod.cc.o
[ 11%] Building CXX object util/CMakeFiles/kenlm_util.dir/stream/chain.cc.o
[ 12%] Building CXX object util/CMakeFiles/kenlm_util.dir/stream/io.cc.o
[ 14%] Building CXX object util/CMakeFiles/kenlm_util.dir/stream/line_input.cc.o
[ 15%] Building CXX object util/CMakeFiles/kenlm_util.dir/stream/multi_progress.cc.o
[ 16%] Building CXX object util/CMakeFiles/kenlm_util.dir/stream/rewindable_stream.cc.o
[ 17%] Building CXX object util/CMakeFiles/kenlm_util.dir/bit_packing.cc.o
[ 19%] Building CXX object util/CMakeFiles/kenlm_util.dir/ersatz_progress.cc.o
[ 20%] Building CXX object util/CMakeFiles/kenlm_util.dir/exception.cc.o
[ 21%] Building CXX object util/CMakeFiles/kenlm_util.dir/file.cc.o
[ 23%] Building CXX object util/CMakeFiles/kenlm_util.dir/file_piece.cc.o
[ 24%] Building CXX object util/CMakeFiles/kenlm_util.dir/float_to_string.cc.o
[ 25%] Building CXX object util/CMakeFiles/kenlm_util.dir/integer_to_string.cc.o
[ 26%] Building CXX object util/CMakeFiles/kenlm_util.dir/mmap.cc.o
[ 28%] Building CXX object util/CMakeFiles/kenlm_util.dir/murmur_hash.cc.o
[ 29%] Building CXX object util/CMakeFiles/kenlm_util.dir/parallel_read.cc.o
[ 30%] Building CXX object util/CMakeFiles/kenlm_util.dir/pool.cc.o
[ 32%] Building CXX object util/CMakeFiles/kenlm_util.dir/read_compressed.cc.o
[ 33%] Building CXX object util/CMakeFiles/kenlm_util.dir/scoped.cc.o
[ 34%] Building CXX object util/CMakeFiles/kenlm_util.dir/string_piece.cc.o
[ 35%] Building CXX object util/CMakeFiles/kenlm_util.dir/usage.cc.o
[ 35%] Built target kenlm_util
Scanning dependencies of target bit_packing_test
[ 37%] Building CXX object util/CMakeFiles/bit_packing_test.dir/bit_packing_test.cc.o
Linking CXX executable ../bin/bit_packing_test
CMakeFiles/kenlm_util.dir/read_compressed.cc.o: In function `util::(anonymous namespace)::StreamCompressed<util::(anonymous namespace)::GZip>::~StreamCompressed()':
read_compressed.cc:(.text+0x219): undefined reference to `
inflateEnd'
CMakeFiles/kenlm_util.dir/read_compressed.cc.o: In function `util::(anonymous namespace)::StreamCompressed<util::(anonymous namespace)::GZip>::~StreamCompressed()'
:

read_compressed
.cc:(.text+0x359): undefined reference to `inflateEnd'
CMakeFiles/kenlm_util.dir/read_compressed.cc.o: In function `
util::(anonymous namespace)::ReadFactory(int, unsigned long&, void const*, unsigned long, bool)':
read_compressed.cc:(.text+0x931): undefined reference to `inflateInit2_'

CMakeFiles/kenlm_util.dir/read_compressed.cc.o: In function `util::(anonymous namespace)::StreamCompressed<util::(anonymous namespace)::GZip>::Read(void*, unsigned long, util::ReadCompressed&)':
read_compressed.cc:(.text+0xc5f): undefined reference to `
inflate'

collect2: error: ld returned 1 exit status
util/CMakeFiles/bit_packing_test.dir/build.make:146: recipe for target 'bin/bit_packing_test' failed

make[2]: *** [bin/bit_packing_test] Error 1
CMakeFiles/Makefile2:964: recipe for target 'util/CMakeFiles/bit_packing_test.dir/all' failed

make[1]: *** [util/CMakeFiles/bit_packing_test.dir/all] Error 2
Makefile:127: recipe for target 'all' failed

make: *** [all] Error 2
cp: cannot stat '
bin/query': No such file or directory
cp: cannot stat '
bin/lmplz': No such file or directory
cp: cannot stat '
bin/build_binary': No such file or directory

Reza Lesmana

unread,
Nov 23, 2015, 1:17:55 PM11/23/15
to Joshua Technical Support, joa.b...@gmail.com
Hi, 

Sorry for late reply. I've been switching from Ubuntu to Centos 7.1 and the build is working great. So, the problem only persists for Ubuntu (14.04, 15.04, 15.10)

However, thrax is giving me trouble since no final directory available so the get_merge command won't be able to execute, still working out what's the problem. I'll post it on new thread. 

Regards,
Reza Lesmana
...

Matt Post

unread,
Nov 23, 2015, 1:33:42 PM11/23/15
to joshua_...@googlegroups.com, joa.b...@gmail.com
Thanks for this update, Reza. I don't understand what the issue is with Ubuntu, but it's good to have the problem isolated, somewhat. 

The Hadoop issue likely has something to do with disk space, but I'll watch for your other post.

matt


joa.b...@gmail.com

unread,
Nov 24, 2015, 10:38:07 AM11/24/15
to Joshua Technical Support, joa.b...@gmail.com
I also tried a different OS now, and the build works like a charm for Debian 3.2.68.

Thanks for helping!

Matt Post

unread,
Nov 24, 2015, 10:39:55 AM11/24/15
to joshua_...@googlegroups.com, joa.b...@gmail.com
Great to hear, thanks for the followup. 

If anyone is inclined, it would be great to submit this as a bug to KenLM (doesn't compile with cmake on Ubuntu).


matt


On Nov 24, 2015, at 10:07 AM, joa.b...@gmail.com wrote:

I also tried a different OS now, and the build works like a charm for Debian 3.2.68.

Thanks for helping!

Gideon

unread,
Apr 6, 2016, 6:01:38 AM4/6/16
to Joshua Technical Support, joa.b...@gmail.com
Dear Matt and others,
I've also struggled with these issues recently.
I run Kubuntu 14.04 on my work computer and we have an outdated CentOS
installation on some of our servers, which necessitates local installation of Boost
and static linking against it , causing more problems.
I've made a patch that fixes the issues I encountered:
1. (Not) Linking to Zlib (at least not in a way working on ubuntu)
2. Not dealing properly with static binding against boost
3. Not adding linking to the "real time" library ( often added as option"-lrt")

My fix is available on my Joshua fork https://github.com/gwenniger/joshua/tree/gideon/fixJoshuaKenlmCMakeFiles
and will possibly be integrated in the Joshua main branch in one form or another at some point.
Cheers.

Gideon

Matt Post

unread,
Apr 6, 2016, 3:50:57 PM4/6/16
to joshua_...@googlegroups.com, joa.b...@gmail.com
Thanks very much, Gideon! I just made a note on your PR that you should submit the patch to KenLM instead, since we just pull in a copy of their code. We also plan to soon switch to using them as a proper submodule, so it's better for you to get it fixed upstream. From my own frustration with this issue, I imagine others (on other decoders) have the same problems and will appreciate the fix.

matt


Reply all
Reply to author
Forward
0 new messages