svn revision 26366: link-parser.exe fails on start with the following output

[Evgenii Philippov]

C:\opencog\lg_svn1\link-grammar\msvc6\link_grammar_exe\Debug>link-parser.exe
Info:
GetModuleFileName=C:\opencog\lg_svn1\link-grammar\msvc6\link_grammar_exe\Debug\link-grammar.dll
Info: data_dir=C:\opencog\lg_svn1\link-grammar\msvc6\link_grammar_exe\Debug
Info: dictopen() trying
C:\opencog\lg_svn1\link-grammar\msvc6\link_grammar_exe\Debug\en\4.0.dict
Info:
GetModuleFileName=C:\opencog\lg_svn1\link-grammar\msvc6\link_grammar_exe\Debug\link-grammar.dll
Info: data_dir=C:\opencog\lg_svn1\link-grammar\msvc6\link_grammar_exe\Debug
Info: dictopen() trying
C:\opencog\lg_svn1\link-grammar\msvc6\link_grammar_exe\Debug\en/words/entities.given-female.sing
Info:
GetModuleFileName=C:\opencog\lg_svn1\link-grammar\msvc6\link_grammar_exe\Debug\link-grammar.dll
Info: data_dir=C:\opencog\lg_svn1\link-grammar\msvc6\link_grammar_exe\Debug
Info: dictopen() trying
C:\opencog\lg_svn1\link-grammar\msvc6\link_grammar_exe\Debug\en/words/entities.given-male.sing
Info:
GetModuleFileName=C:\opencog\lg_svn1\link-grammar\msvc6\link_grammar_exe\Debug\link-grammar.dll
Info: data_dir=C:\opencog\lg_svn1\link-grammar\msvc6\link_grammar_exe\Debug
Info: dictopen() trying
C:\opencog\lg_svn1\link-grammar\msvc6\link_grammar_exe\Debug\en/words/entities.given-bisex.sing
Info:
GetModuleFileName=C:\opencog\lg_svn1\link-grammar\msvc6\link_grammar_exe\Debug\link-grammar.dll
Info: data_dir=C:\opencog\lg_svn1\link-grammar\msvc6\link_grammar_exe\Debug
Info: dictopen() trying
C:\opencog\lg_svn1\link-grammar\msvc6\link_grammar_exe\Debug\en/words/entities.organizations.sing
Info:
GetModuleFileName=C:\opencog\lg_svn1\link-grammar\msvc6\link_grammar_exe\Debug\link-grammar.dll
Info: data_dir=C:\opencog\lg_svn1\link-grammar\msvc6\link_grammar_exe\Debug
Info: dictopen() trying
C:\opencog\lg_svn1\link-grammar\msvc6\link_grammar_exe\Debug\en/words/entities.locations.sing
Info:
GetModuleFileName=C:\opencog\lg_svn1\link-grammar\msvc6\link_grammar_exe\Debug\link-grammar.dll
Info: data_dir=C:\opencog\lg_svn1\link-grammar\msvc6\link_grammar_exe\Debug
Info: dictopen() trying
C:\opencog\lg_svn1\link-grammar\msvc6\link_grammar_exe\Debug\en/words/words.n.1
Info:
GetModuleFileName=C:\opencog\lg_svn1\link-grammar\msvc6\link_grammar_exe\Debug\link-grammar.dll
Info: data_dir=C:\opencog\lg_svn1\link-grammar\msvc6\link_grammar_exe\Debug
Info: dictopen() trying
C:\opencog\lg_svn1\link-grammar\msvc6\link_grammar_exe\Debug\en/words/words.n.2.s
Info:
GetModuleFileName=C:\opencog\lg_svn1\link-grammar\msvc6\link_grammar_exe\Debug\link-grammar.dll
Info: data_dir=C:\opencog\lg_svn1\link-grammar\msvc6\link_grammar_exe\Debug
Info: dictopen() trying
C:\opencog\lg_svn1\link-grammar\msvc6\link_grammar_exe\Debug\en/words/words.n.2.s.biolg
Info:
GetModuleFileName=C:\opencog\lg_svn1\link-grammar\msvc6\link_grammar_exe\Debug\link-grammar.dll
Info: data_dir=C:\opencog\lg_svn1\link-grammar\msvc6\link_grammar_exe\Debug
Info: dictopen() trying
C:\opencog\lg_svn1\link-grammar\msvc6\link_grammar_exe\Debug\en/words/words.n.2.x
Info:
GetModuleFileName=C:\opencog\lg_svn1\link-grammar\msvc6\link_grammar_exe\Debug\link-grammar.dll
Info: data_dir=C:\opencog\lg_svn1\link-grammar\msvc6\link_grammar_exe\Debug
Info: dictopen() trying
C:\opencog\lg_svn1\link-grammar\msvc6\link_grammar_exe\Debug\en/words/words.n.3
Info:
GetModuleFileName=C:\opencog\lg_svn1\link-grammar\msvc6\link_grammar_exe\Debug\link-grammar.dll
Info: data_dir=C:\opencog\lg_svn1\link-grammar\msvc6\link_grammar_exe\Debug
Info: dictopen() trying
C:\opencog\lg_svn1\link-grammar\msvc6\link_grammar_exe\Debug\en/words/words.n.4
Info:
GetModuleFileName=C:\opencog\lg_svn1\link-grammar\msvc6\link_grammar_exe\Debug\link-grammar.dll
Info: data_dir=C:\opencog\lg_svn1\link-grammar\msvc6\link_grammar_exe\Debug
Info: dictopen() trying
C:\opencog\lg_svn1\link-grammar\msvc6\link_grammar_exe\Debug\en/words/words.n.4.biolg
Info:
GetModuleFileName=C:\opencog\lg_svn1\link-grammar\msvc6\link_grammar_exe\Debug\link-grammar.dll
Info: data_dir=C:\opencog\lg_svn1\link-grammar\msvc6\link_grammar_exe\Debug
Info: dictopen() trying
C:\opencog\lg_svn1\link-grammar\msvc6\link_grammar_exe\Debug\en/words/entities.us-states.sing
Info:
GetModuleFileName=C:\opencog\lg_svn1\link-grammar\msvc6\link_grammar_exe\Debug\link-grammar.dll
Info: data_dir=C:\opencog\lg_svn1\link-grammar\msvc6\link_grammar_exe\Debug
Info: dictopen() trying
C:\opencog\lg_svn1\link-grammar\msvc6\link_grammar_exe\Debug\en/words/entities.national.sing
Info:
GetModuleFileName=C:\opencog\lg_svn1\link-grammar\msvc6\link_grammar_exe\Debug\link-grammar.dll
Info: data_dir=C:\opencog\lg_svn1\link-grammar\msvc6\link_grammar_exe\Debug
Info: dictopen() trying
C:\opencog\lg_svn1\link-grammar\msvc6\link_grammar_exe\Debug\en/words/words.n.t
Info:
GetModuleFileName=C:\opencog\lg_svn1\link-grammar\msvc6\link_grammar_exe\Debug\link-grammar.dll
Info: data_dir=C:\opencog\lg_svn1\link-grammar\msvc6\link_grammar_exe\Debug
Info: dictopen() trying
C:\opencog\lg_svn1\link-grammar\msvc6\link_grammar_exe\Debug\en/words/words.y
Info:
GetModuleFileName=C:\opencog\lg_svn1\link-grammar\msvc6\link_grammar_exe\Debug\link-grammar.dll
Info: data_dir=C:\opencog\lg_svn1\link-grammar\msvc6\link_grammar_exe\Debug
Info: dictopen() trying
C:\opencog\lg_svn1\link-grammar\msvc6\link_grammar_exe\Debug\
Info: dictopen() trying .\
Error opening word file
Error opening word file /
link-parser.exe: Fatal error: Unable to open dictionary.

C:\opencog\lg_svn1\link-grammar\msvc6\link_grammar_exe\Debug>

When I run it the following way, the above happens, too.

set LANG=en_US.UTF-8
link-parser.exe

-ef

[Evgenii Philippov]

Evgenii Philippov пишет:

Same error if called from RelEx:
[...]
Info: dictopen() trying C:\opencog\disco\native\windows\en/words/words.y
Info: GetModuleFileName=C:\opencog\disco\native\windows\link-grammar.dll
Info: data_dir=C:\opencog\disco\native\windows
Info: dictopen() trying C:\opencog\disco\native\windows\

Info: dictopen() trying .\
Error opening word file
Error opening word file /

Failed to process sentence: Alice wrote a book about dinosaurs for the
University of California in Berkeley.
java.lang.RuntimeException: link-parser:
Error: unable to open dictionary
at org.linkgrammar.LinkGrammar.init(Native Method)
at relex.parser.LocalLGParser.init(LocalLGParser.java:30)
at relex.parser.LocalLGParser.parse(LocalLGParser.java:44)
at relex.RelationExtractor.parseSentence(RelationExtractor.java:263)
at
relex.RelationExtractor.processSentence(RelationExtractor.java:204)
at relex.RelationExtractor.main(RelationExtractor.java:485)
[...]

-ef

[Borislav Iordanov]

this look like a dictionary path problem...

--
"Frozen brains tell no tales."

-- Buckethead

[Evgenii Philippov]

Borislav Iordanov пишет:

> this look like a dictionary path problem...
>

unlikely... it accepts a dozen of dictionary files ok, and does not
accept the rest of files? hmm.

-ef

[Evgenii Philippov]

I fixed this in a wrong ("works for me") way:

- Opened en/4.0.dict
- After the line
/en/words/words.y: <date-id>;
(not including this line), and before the line
/en/words/words.v.1.1:
(not including this line)
replaced all characters `/' with underscores.

Now, it does not try to open the dictionary file `/' referenced from
`en/4.0.dict'.

Probably, gnuwin32 regex compiled by msvc6 behaves in a way different
than the standard Unix regex... I don't know.

I don't need a better solution, but one's mileage may vary.

-ef

[Linas Vepstas]

2009/5/18 Evgenii Philippov <efil...@gmail.com>:

>
> I fixed this in a wrong ("works for me") way:
>
> - Opened en/4.0.dict
> - After the line
> /en/words/words.y: <date-id>;
> (not including this line), and before the line
> /en/words/words.v.1.1:
> (not including this line)
> replaced all characters `/' with underscores.

?? you mean the back-quote? or the slash?
why would this make a difference?

can you narrow down the problem?

> Probably, gnuwin32 regex compiled by msvc6 behaves in a way different
> than the standard Unix regex... I don't know.

the dictionary loading does not use regex, at all.
The regex is only for finding certain word endings
(words ending with -ing, -ly, -ed, etc.)

--linas

p.s. see also
http://ftp.gnome.org/pub/gnome/binaries/win32/dependencies/regex.README

[Evgenii Philippov]

Linas Vepstas пишет:

> 2009/5/18 Evgenii Philippov <efil...@gmail.com>:
>
>> I fixed this in a wrong ("works for me") way:
>>
>> - Opened en/4.0.dict
>> - After the line
>> /en/words/words.y: <date-id>;
>> (not including this line), and before the line
>> /en/words/words.v.1.1:
>> (not including this line)
>> replaced all characters `/' with underscores.
>>
>
> ?? you mean the back-quote? or the slash?
> why would this make a difference?
>
> can you narrow down the problem?
>

slash! not a quote.

LG dictionary syntax /dictfilename: means "take dictfilename and parse it"

i will try to narrow when i'll finish the work with that my JNI patch.

LG can be easily debugged with MSVC, so chances are that I will be able
to find the exact cause for LG misbehaving.

>
>> Probably, gnuwin32 regex compiled by msvc6 behaves in a way different
>> than the standard Unix regex... I don't know.
>>
>
> the dictionary loading does not use regex, at all.
> The regex is only for finding certain word endings
> (words ending with -ing, -ly, -ed, etc.)
>
> --linas
>
> p.s. see also
> http://ftp.gnome.org/pub/gnome/binaries/win32/dependencies/regex.README
>
>

Thank you! gnu regex build that I downloaded uses "regex2.dll" which is
not described in this README...

-ef

[Evgenii Philippov]

btw i've tested LG4.5.4 tarball, same problem with 4.5.4.

[Evgenii Philippov]

Evgenii Philippov пишет:
> Linas Vepstas пишет:

>>
>> ?? you mean the back-quote? or the slash?
>> why would this make a difference?
>>
>> can you narrow down the problem?
>>
> slash! not a quote.
>
> LG dictionary syntax /dictfilename: means "take dictfilename and parse
> it"
>

So LG gets a command "/" from somewhere and tries to load a dictionary
file named "" from a data_path.

-ef

[Vector.M]

A quick fix is to apply this patch in read-dict.c (line 1321):

==========read-dict.c==========
1321c1321
< if (dict->token[0] == '/')
---
> if (dict->token[0] == '/' && dict->token[1])

===============================

-Vector.

[Evgenii Philippov]

It doesn't fix the root cause of the problem.

-ef

Vector.M пишет:

[Linas Vepstas]

2009/5/20 Evgenii Philippov <efil...@gmail.com>:

>
> It doesn't fix the root cause of the problem.

I'm guessing that the root cause is line 759 of 4.0.dict
which has a slash in it. Can you comment that out and
see if that fixes it?

--linas

[Evgenii Philippov]

Linas Vepstas пишет:

I took 4.5.4 tarball and commented out lines as follows:

% "verb" use, e.g. "( p < 0.01 )" for "( p is less than 0.01 )"
%=.v <.v >.v *.v "/".v +.v -.v x.v:
%(S- or EQ-) & O+ &
%(EQ+ or (Xd- & (Xc+ or [[[()]]]) & MX-));

Everything went smoothly, without mentioned errors.

Made it into the original state:

% "verb" use, e.g. "( p < 0.01 )" for "( p is less than 0.01 )"
=.v <.v >.v *.v "/".v +.v -.v x.v:
(S- or EQ-) & O+ &
(EQ+ or (Xd- & (Xc+ or [[[()]]]) & MX-));

That error has triggered.

Am checking for this behavior with the latest SVN rev.

-ef

[Linas Vepstas]

>> I'm guessing that the root cause is line 759 of 4.0.dict

OK, I think the following should fix this:

Index: link-grammar/read-dict.c
===================================================================
--- link-grammar/read-dict.c (revision 26100)
+++ link-grammar/read-dict.c (working copy)
@@ -1318,7 +1318,9 @@
}

/* if it's a word-file name */
- if (dict->token[0] == '/')
+ /* However, be careful to reject "/.v" which is the division symbol
+ * used in equations (.v means verb-like) */
+ if ((dict->token[0] == '/') && (dict->token[1] != '.'))
{
dn = read_word_file(dict, dn, dict->token);
if (dn == NULL)
Index: data/en/4.0.dict
===================================================================
--- data/en/4.0.dict (revision 26120)
+++ data/en/4.0.dict (working copy)
@@ -756,7 +756,7 @@
%EQUATIONS ETC.

% "verb" use, e.g. "( p < 0.01 )" for "( p is less than 0.01 )"

-=.v <.v >.v *.v "/".v +.v -.v x.v:
+=.v <.v >.v *.v "/.v" +.v -.v x.v:

[Evgenii Philippov]

> Am checking for this behavior with the latest SVN rev.

LG svn revision 26417: Same behavior as in 4.5.4 wrt this error.

-ef

[Linas Vepstas]

2009/5/20 Evgenii Philippov <efil...@gmail.com>:

>
>
>> Am checking for this behavior with the latest SVN rev.
> LG svn revision 26417: Same behavior as in 4.5.4 wrt this error.

the fix is in 26418

--linas