Re: [joshua_development:868] Re: Have an LM

[Hieu Hoang]

Hi fellow MTers and brothers in arms,

i've been bashing this about with ken in moses. It's been repackaged so
that the lm calc functionality is in a lib so it can be integrated into
the decoder. You guys might wanna grab the moses version to avoid
duplicating effort. i guess once all the bugs have been ironed out, the
lib can go in a standalone project so we can all use it without
depending on each other.

ps. does anyone know why chart decoding uses so much more mem that
phrase-based? what part is the main memory guzzler? *scratch head*

On 13/09/2010 20:53, Chris Callison-Burch wrote:
> Awesome! We'll try to integrate this with Joshua sometime this term.
>
> --Chris
>
> On Sep 13, 2010, at 3:49 PM, Kenneth Heafield wrote:
>
>> Off and on, I have promised you language model inference code that's
>> easy to compile, mmaps, returns probabilities faster by caching backoff
>> in state, and loads ARPAs faster than SRI. You may download it from
>> http://kheafield.com/code/infer.tar.gz . It works on Linux (and
>> probably anything POSIX) with only the basic C++ installed.
>>
>> Helping you break your SRI dependence,
>>
>> Kenneth

[Hieu Hoang]

ah, update, the kenlm in moses contains moses-specific kludges so u
should get the source directly from ken's website if you wanna integrate
it into a different decoder