A few bugs while installing a test version of OpenCog

[rocketpwr.com]

Dear OpenCog Development Community,

First, thank you for your hard work on this project.

I found a few bugs that I have been able to work around that you might find helpful.  Hopefully the below description of the problems and my work-arounds will help you or someone else who comes across the same problems.

1) Really really long compile times on loading up the conceptnet4.scm test dataset in test-datasets/conceptnet/conceptnet4.scm.  I left it to run overnight, and it still had not finished. Actually, I believe it was going to take 70+ hours to load and compile this 14.5 MB file.  This seemed strange to me as smaller files loaded much faster.  What I determined by experiment is that, for some unknown reason, the Guile Scheme compiler had a compile time that is proportional approximately n^2 of the number of rules.  Here are some results on my test rig (a 4 core 4 GB Ubuntu 16.04 setup):

Rules loadtime rules/sec
 
259   7        37.00
634   31       20.45
1000  70       14.29
1249  110      11.35

Because the Conceptnet4.scm file is about 60.5K I estimate that the compile time would be around 71 hours or 3 days.  I am guessing this is a bug/feature in Guile Scheme which was not really meant as a database.

My work-around:  a) split the 60K rule file into 61 files of 1K rules each, and then compile them.  Each file compiles in around 14 seconds, reducing the compile time to around 1.5 hours.  Once compiled, the rules do not need to be compiled again.  b) I wrote a small loader that loads all 61 files into Guile in a single command.

2) Really long execution time for (cog-bind deduction-inheritance-rule): I am experimenting with PLN, using the how-to tutorial: PLN by hand (http://wiki.opencog.org/w/PLN_by_Hand).  After loading up conceptnet4, I tried to run the inheritance rule as per the tutorial: 

(cog-bind deduction-inheritance-rule)

Again, I left the computer to run overnight, and by morning it wasn't complete. I searched the machine and found the errors in the opencog.log file which has grown to 1 GB.  It was filled with a repeat of this message (with some other hex-addresses changed):

In unknown file:

   ?: 7 [opencog-extension cog-bind-first-n (# -1)]

In ice-9/boot-9.scm:

 157: 6 [catch #t #<catch-closure 2b750a0> ...]

In unknown file:

   ?: 5 [apply-smob/1 #<catch-closure 2b750a0>]

In ice-9/boot-9.scm:

 157: 4 [catch #t #<catch-closure 2b77be0> ...]

In unknown file:

   ?: 3 [apply-smob/1 #<catch-closure 2b77be0>]

In opencog.scm:

  89: 2 [cog-undefined-handle]

In ice-9/boot-9.scm:

 102: 1 [#<procedure 95b1940 at ice-9/boot-9.scm:97:6 (thrown-k . args)> wrong-number-of-args ...]

In unknown file:

   ?: 0 [apply-smob/1 #<catch-closure 2b77ba0> wrong-number-of-args ...]

ERROR: In procedure apply-smob/1:

ERROR: Wrong number of arguments to #<procedure cog-undefined-handle (X)>

ABORT: wrong-number-of-args

I have been researching this for a few hours and I was about to give up.  I then decided to experiment with smaller numbers of atoms (rules).  I used the Guile profiler and it showed that the majority of the time was being wasted in error "Catch" and formatting functions. After setting debug breakpoints, I decided that the problem was actually in this cog-undefined-handle function, and not the cog-bind-first-n function, or whatever apply-smob/1 is. (I can't find any documentation on apply-smob/1.) The cog-undefined-handle function was in /usr/local/share/opencog/scm/opencog.scm:

I changed this line:

(define-public (cog-undefined-handle X) '())

to

(define-public (cog-undefined-handle . X) '())

Note: I am not expert in Guile, Scheme, or Lisp. A poster on stack exchange said that by inserted the dot character in the argument list, it allows a Scheme function to take any number of arguments.  This seems to work -- now the opencog.log is clean and the (cog-bind deduction-inheritance-rule) runs on the entire data set in only a few minutes.

3) Hard coded gcc/g++ 4.8 compiler version in octool prevents installation on Ubuntu 16.04 LTS.  I understand that the base Linux configuration for Opencog is still 14.04 LTS.  However, that version is already 3 years old and I had just installed a 16.04 LTS image, and I did not want to reinstall after investing the time to set it up properly.  To get octool to finish the compilation and subsequent installation, I did the following change:

$ diff octool octool~
843,844c843
< #CXX=g++-4.8 cmake .. -DCMAKE_BUILD_TYPE=Release
< CXX=g++ cmake .. -DCMAKE_BUILD_TYPE=Release
---
> CXX=g++-4.8 cmake .. -DCMAKE_BUILD_TYPE=Release

A more clever programmer can ensure that the a version of g++ <= 4.8 is used by octool script.

4) Old python conceptnet converter.py doesn't match newer 5.5.0 version of conceptnet.  [NOTE: The following python code is likely deprecated in favor of newer c++ converter found here:  https://github.com/opencog/external-tools/tree/master/cnet_importer  ] 

Earlier in the week, I tried to convert an English only conceptnet-assertions-5.5.0.csv file prepared by grep filtering out the non-English assertions.  Unfortunately the Python converter.py that was downloaded (I believe by octool) does not work on this file.  Note: this version of converter.py was installed in my ~/opencog/opencog/opencog/python/conceptnet/ directory.  I further modified the conceptnet-assertions.5.5.0.csv file work by changing the column format to the same as the conceptnet4.csv file. This allowed the python converter script to run to the end of the file without terminating prematurely.  

-----------

Overall I am still working to understand Opencog and its capabilities.  

Finally, if anyone has uploaded a recent dbpedia ruleset to Opencog, I would like to experiment with it as well.  Please drop a line.

Thank you again.

[rocketpwr.com]

Correction:

"Each file compiles in around 14 seconds, reducing the compile time to around 1.5 hours. "  should be 70 seconds each (see the table in my post).  70*61=4270 seconds = 1.2 hours.  

[Nil Geisweiller]

Thanks a lot for your detailed bug report! Ideally you would create
github issues if each of these, but this is already greatly appreciated.

I'll have a careful look at them, already though regarding the slow down
due to cog-undefined-handle this is probably because it is getting
obsolete and I haven't removed it from all PLN rules (doing that now).

Nil

On 05/29/2017 02:50 AM, rocketpwr.com wrote:
> Dear OpenCog Development Community,
>
> First, thank you for your hard work on this project.
>
> I found a few bugs that I have been able to work around that you might
> find helpful. Hopefully the below description of the problems and my
> work-arounds will help you or someone else who comes across the same
> problems.
>

> /*1) Really really long compile times on loading up the conceptnet4.scm
> test dataset in test-datasets/conceptnet/conceptnet4.scm*./ I left it

> to run overnight, and it still had not finished. Actually, I believe it
> was going to take 70+ hours to load and compile this 14.5 MB file. This
> seemed strange to me as smaller files loaded much faster. What I
> determined by experiment is that, for some unknown reason, the Guile
> Scheme compiler had a compile time that is proportional approximately
> n^2 of the number of rules. Here are some results on my test rig (a 4
> core 4 GB Ubuntu 16.04 setup):
>
> |
> Rulesloadtime rules/sec
>
> 259737.00
> 6343120.45
> 10007014.29
> 124911011.35
> |
>
>
> Because the Conceptnet4.scm file is about 60.5K I estimate that the
> compile time would be around 71 hours or 3 days. I am guessing this is
> a bug/feature in Guile Scheme which was not really meant as a database.
>
> My work-around: a) split the 60K rule file into 61 files of 1K rules
> each, and then compile them. Each file compiles in around 14 seconds,
> reducing the compile time to around 1.5 hours. Once compiled, the rules
> do not need to be compiled again. b) I wrote a small loader that loads
> all 61 files into Guile in a single command.
>

> */2) Really long execution time for (cog-bind
> deduction-inheritance-rule):/* I am experimenting with PLN, using the

> (define-public(cog-undefined-handle .X)'())

> |
>
> Note: I am not expert in Guile, Scheme, or Lisp. A poster on stack
> exchange said that by inserted the dot character in the argument list,
> it allows a Scheme function to take any number of arguments. This seems
> to work -- now the opencog.log is clean and the (cog-bind
> deduction-inheritance-rule) runs on the entire data set in only a few
> minutes.

> */
> /*
> */3) Hard coded gcc/g++ 4.8 compiler version in octool prevents
> installation on Ubuntu 16.04 LTS/*. I understand that the base Linux

> configuration for Opencog is still 14.04 LTS. However, that version is
> already 3 years old and I had just installed a 16.04 LTS image, and I
> did not want to reinstall after investing the time to set it up
> properly. To get octool to finish the compilation and subsequent
> installation, I did the following change:
>
> |
> $ diff octool octool~
> 843,844c843
> <#CXX=g++-4.8 cmake .. -DCMAKE_BUILD_TYPE=Release
> <CXX=g++cmake ..-DCMAKE_BUILD_TYPE=Release
> ---
>>CXX=g++-4.8cmake ..-DCMAKE_BUILD_TYPE=Release
> |
>
>
> A more clever programmer can ensure that the a version of g++ <= 4.8 is
> used by octool script.
>

> */4) Old python conceptnet converter.py doesn't match newer 5.5.0
> version of conceptnet. /* [NOTE: The following python code is likely

> deprecated in favor of newer c++ converter found here:
> https://github.com/opencog/external-tools/tree/master/cnet_importer ]
>
> Earlier in the week, I tried to convert an English only
> conceptnet-assertions-5.5.0.csv file prepared by grep filtering out the
> non-English assertions. Unfortunately the Python converter.py that was
> downloaded (I believe by octool) does not work on this file. Note: this
> version of converter.py was installed in my
> ~/opencog/opencog/opencog/python/conceptnet/ directory. I further
> modified the conceptnet-assertions.5.5.0.csv file work by changing the
> column format to the same as the conceptnet4.csv file. This allowed the
> python converter script to run to the end of the file without
> terminating prematurely.
>
> -----------
>
> Overall I am still working to understand Opencog and its capabilities.
>
> Finally, if anyone has uploaded a recent dbpedia ruleset to Opencog, I
> would like to experiment with it as well. Please drop a line.
>
> Thank you again.
>
>
>

> --
> You received this message because you are subscribed to the Google
> Groups "opencog" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to opencog+u...@googlegroups.com
> <mailto:opencog+u...@googlegroups.com>.
> To post to this group, send email to ope...@googlegroups.com
> <mailto:ope...@googlegroups.com>.
> Visit this group at https://groups.google.com/group/opencog.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/opencog/608d7d27-f608-4eb9-a998-c607e4a4f34a%40googlegroups.com
> <https://groups.google.com/d/msgid/opencog/608d7d27-f608-4eb9-a998-c607e4a4f34a%40googlegroups.com?utm_medium=email&utm_source=footer>.
> For more options, visit https://groups.google.com/d/optout.

[Nil Geisweiller]

Linas,

based on

https://github.com/opencog/atomspace/blob/master/opencog/scm/opencog.scm#L113

I assume that cog-undefined-handle is getting obsolete. However it is
still on the wiki page

http://wiki.opencog.org/w/Scheme#cog-undefined-handle

Could you update that accordingly? (I wanted to do it but I'm not fully
confident).

Thanks,
Nil

[Nil Geisweiller]

Hi,

On 05/29/2017 02:50 AM, rocketpwr.com wrote:

> My work-around: a) split the 60K rule file into 61 files of 1K rules
> each, and then compile them. Each file compiles in around 14 seconds,
> reducing the compile time to around 1.5 hours. Once compiled, the rules
> do not need to be compiled again. b) I wrote a small loader that loads
> all 61 files into Guile in a single command.

The workaround we use is to call guile with

--no-auto-compile

however you're workaround might speed up subsequent loading. How faster
is it once already compiled?

>
> */2) Really long execution time for (cog-bind
> deduction-inheritance-rule):/* I am experimenting with PLN, using the

> how-to tutorial: PLN by hand (http://wiki.opencog.org/w/PLN_by_Hand).
> After loading up conceptnet4, I tried to run the inheritance rule as
> per the tutorial:

Hopefully this should now be fixed, do you still experience that problem?

Thanks,
Nil

[rocketpwr.com]

"however you're workaround might speed up subsequent loading. How faster 

is it once already compiled? "

scheme@(guile-user)> ,t (load-from-path "loadConceptnet4.scm")

compiled splits/cnet4.1.scm

compiled splits/cnet4.2.scm

compiled splits/cnet4.3.scm

[...redacted loading files 4-58...] 

compiled splits/cnet4.59.scm

compiled splits/cnet4.60.scm

compiled splits/cnet4.61.scm

;; 13.424310s real time, 13.906361s run time.  1.137787s spent in GC.

[Nil Geisweiller]

That seems pretty good. Do you know how that compares to using

--no-auto-compile

in the first place?

Thanks,
Nil

>
> --
> You received this message because you are subscribed to the Google
> Groups "opencog" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to opencog+u...@googlegroups.com
> <mailto:opencog+u...@googlegroups.com>.
> To post to this group, send email to ope...@googlegroups.com
> <mailto:ope...@googlegroups.com>.
> Visit this group at https://groups.google.com/group/opencog.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/opencog/e6de7f7a-be7c-488a-8d00-ad559dac76c2%40googlegroups.com
> <https://groups.google.com/d/msgid/opencog/e6de7f7a-be7c-488a-8d00-ad559dac76c2%40googlegroups.com?utm_medium=email&utm_source=footer>.

[rocketpwr.com]

Nil, give me a few days and I will run it for you.

-mike

[Linas Vepstas]

Hi,

On Sun, May 28, 2017 at 6:50 PM, rocketpwr.com <comm...@rocketpwr.com> wrote:

1) Really really long compile times on loading up the conceptnet4.scm test dataset in test-datasets/conceptnet/conceptnet4.scm.  

This is a known bug/feature .. err. known to us. My standard solution is to disable compiling for these kinds of files. So: use primitive-load, instead of load.   https://www.gnu.org/software/guile/manual/html_node/Loading.html

Its much much easier than splitting the file into chunks.

In the meanwhile, I should ask you to complain to the guile folks that it takes too long to compile large files. I think they're sick of hearing it from me :-)

ERROR: Wrong number of arguments to #<procedure cog-undefined-handle (X)>

ABORT: wrong-number-of-args-number-of-args ...]

ERROR: In procedure apply-smob/1

There is no such thing as "cog-undefined-handle".  Its totally bogus.  Wherever it is, it should be removed.  Its some left-over mistake ...

I changed this line:

(define-public (cog-undefined-handle X) '())

to

(define-public (cog-undefined-handle . X) '())

Yeah, I guess that's maybe OK, but it's even better to just let it throw the error. That tells the user that thier code is broken, and that they should fix their code.

3) Hard coded gcc/g++ 4.8 compiler version in octool prevents installation on Ubuntu 16.04 LTS.

That's a bug should be fixed.  Beats me why it's hard-coded.

--linas

[Linas Vepstas]

Hi Nil,

That function only made sense back when the TLB lived inside the atomspace. Now that it doesn't, it doesn't.  The problem was that I caught Mandeep liberally sprinkling it into brand-new code, which I subdued with blunt trauma (the code, not Mandeep or his mental health).  It did not occur to me to search through the external-tools codebase for other possible uses.

--linas

To unsubscribe from this group and stop receiving emails from it, send an email to opencog+unsubscribe@googlegroups.com <mailto:opencog+unsubscribe@googlegroups.com>.
To post to this group, send email to ope...@googlegroups.com <mailto:opencog@googlegroups.com>.

[rocketpwr.com]

Hi Nil,

Here is the result, but i don't really understand it:

$ guile --no-auto-compile

GNU Guile 2.0.11

Copyright (C) 1995-2014 Free Software Foundation, Inc.

Guile comes with ABSOLUTELY NO WARRANTY; for details type `,show w'.

This program is free software, and you are welcome to redistribute it

under certain conditions; type `,show c' for details.

Enter `,help' for help.

scheme@(guile-user)> (use-modules (opencog atom-types))

started1 atom-types.scm

finished atom-types.scm

scheme@(guile-user)> ,t (load-from-path "conceptnet/opencog/conceptnet4.scm")

$1 = (InheritanceLink (stv 1 0.000625)

   (ConceptNode "computer" (stv 0.00018 0.001186))

   (ConceptNode "playroom" (stv 1e-06 0.001186))

)

;; 81.438993s real time, 83.566977s run time.  4.311136s spent in GC.

scheme@(guile-user)> ,t (load-from-path "/home/rocket/mysrc/atom_utils.scm")

;; 0.001712s real time, 0.001706s run time.  0.000000s spent in GC.

scheme@(guile-user)> (cnt-all)

$2 = 139916

The Answer of course is that it loads the 60K rules into atomspace in 81 seconds which is only 5.5x slower than loading the compiled rules in using my (now unnecessary) workaround.  However, I noted that it did not compile the Conceptnet4.scm file into a Conceptnet4.go file.

So what I don't understand, is why does Scheme/Guile bother to compile the rules at all for this file into byte code if it can execute it without compilation and if compilation takes a really long time due to the n^2 execution proportion factor.  Your work-around is better than mine (as we don't have to split the files up manually and compile them).  I just don't understand why Guile does byte code compilation at all for these cases.

I also see that I can manually compile a Scheme/Guile .scm file  using 

$guild compile foo.scm

I could experiment later to see if compiling outside of the Guile shell eliminates the n^2 problem compile execution time problem, i.e. a linear time compilation would be nice.  But, if we don't need to compile for performance, I would not bother with it.

Your thoughts?

Best,

Mike

On Thursday, June 1, 2017 at 3:06:51 PM UTC-4, Nil wrote:

[rocketpwr.com]

"This is a known bug/feature .. err. known to us. My standard solution is to disable compiling for these kinds of files. So: use primitive-load, instead of load.   https://www.gnu.org/software/guile/manual/html_node/Loading.html

Its much much easier than splitting the file into chunks.

In the meanwhile, I should ask you to complain to the guile folks that it takes too long to compile large files. I think they're sick of hearing it from me :-)"

Hi Linas,

Thank you and Nils for being so responsive.  It's like having Linux Torvalds, Greg Kroah-Hartmann or Alan Cox help a newbee.

I agree with your work-around.  

With respect to Guile, I think the problem is that Guile is not a popular enough Scheme implementation (combined with the fact that Scheme is not a popular enough language) to have a large core community working to fix performance problems.  If it was Python, this compile time issue would repaired without us having to ask.  In this situation, it would more than likely be up to us to spend 4 weeks learning enough about Guile to correct it.  I'd rather learn about OpenCog.

Thanks for looking at the problem.  I surely will uncover new performance issues later-on that you have worked around, plus I am just learning PLN + Atomspace and I might need some help.  I appreciate your time.

Best,

Mike

[Linas Vepstas]

On Sat, Jun 3, 2017 at 5:16 PM, rocketpwr.com <comm...@rocketpwr.com> wrote:

$ guile --no-auto-compile

Please use primitive-load instead of using this flag. You want most things compiled.
 

.

So what I don't understand, is why does Scheme/Guile bother to compile the rules at all for this file into byte code if it can execute it without compilation

Because compiled code runs faster than interpreted code. This is always generically true for any language. Not all languages have interpreters; scheme is so friggingly awesomely simple that writing a scheme interpreter is a homework exercise in comp-sci 102. This is not the case for most languages.
 

and if compilation takes a really long time due to the n^2 execution proportion factor.

Because the n^2 is due to a guile bug. Or maybe its a "feature".

--linas