A bug with the SBCL, or Shen

[Antti Ylikoski]

I started trying the Shen/Common LISP interface (so to speak) and got

the following.

The error handler routine says, to report this as a bug.

The load-cl.lisp file is as follows:

------------------------------------------------------------

(DEFUN load-cl (File)

  (SETF (READTABLE-CASE *READTABLE*) :UPCASE)

  (trap-error (LOAD File) (/. E (output (error-to-string E))))

  (SETF (READTABLE-CASE *READTABLE*) :PRESERVE))

  

------------------------------------------------------------

And following is a transcript of my SBCL Shen session which made the

SBCL crash:

------------------------------------------------------------

antti@mydebian:~/SBCL$ ./Shen

Shen, copyright (C) 2010-2015 Mark Tarver

www.shenlanguage.org, Shen 19.2

running under Common Lisp, implementation: SBCL

port 2.0 ported by Mark Tarver

(0-) (LOAD "load-cl.lisp")

T

(1-) (load-cl "SHRDLU.lisp")

PRESERVE

(2-) (SET-BLOCKS)

[]

(3-) (DESCRIBE-BLOCKS)

COMMON-LISP-USER::TABLE

  [symbol]

Symbol-plist:

  SUPPORTED-BY -> NIL

  DIRECTLY-SUPPORTS -> NIL

  POSITION -> (0 0)

  HEIGHT -> 0

  WIDTH -> 20

  IS-A -> TABLE

COMMON-LISP-USER::B1

  [symbol]

Symbol-plist:

  SUPPORTED-BY -> TABLE

  DIRECTLY-SUPPORTS -> NIL

  POSITION -> (0 0)

  HEIGHT -> 2

  WIDTH -> 2

  IS-A -> BRICK

COMMON-LISP-USER::B2

  [symbol]

Symbol-plist:

  SUPPORTED-BY -> TABLE

  DIRECTLY-SUPPORTS -> NIL

  POSITION -> (2 0)

  HEIGHT -> 2

  WIDTH -> 2

  IS-A -> BRICK

COMMON-LISP-USER::B3

  [symbol]

Symbol-plist:

  SUPPORTED-BY -> TABLE

  DIRECTLY-SUPPORTS -> NIL

  POSITION -> (4 0)

  HEIGHT -> 4

  WIDTH -> 4

  IS-A -> BRICK

COMMON-LISP-USER::B4

  [symbol]

Symbol-plist:

  SUPPORTED-BY -> TABLE

  DIRECTLY-SUPPORTS -> NIL

  POSITION -> (8 0)

  HEIGHT -> 2

  WIDTH -> 2

  IS-A -> BRICK

COMMON-LISP-USER::W5

  [symbol]

Symbol-plist:

  SUPPORTED-BY -> TABLE

  DIRECTLY-SUPPORTS -> NIL

  POSITION -> (10 0)

  HEIGHT -> 4

  WIDTH -> 2

  IS-A -> WEDGE

COMMON-LISP-USER::B6

  [symbol]

Symbol-plist:

  SUPPORTED-BY -> TABLE

  DIRECTLY-SUPPORTS -> NIL

  POSITION -> (12 0)

  HEIGHT -> 2

  WIDTH -> 4

  IS-A -> BRICK

COMMON-LISP-USER::W7

  [symbol]

Symbol-plist:

  SUPPORTED-BY -> TABLE

  DIRECTLY-SUPPORTS -> NIL

  POSITION -> (16 0)

  HEIGHT -> 2

  WIDTH -> 2

  IS-A -> WEDGE

COMMON-LISP-USER::L8

  [symbol]

Symbol-plist:

  SUPPORTED-BY -> TABLE

  DIRECTLY-SUPPORTS -> NIL

  POSITION -> (18 0)

  HEIGHT -> 2

  WIDTH -> 2

  IS-A -> BALL

[]

(4-) (PUT-ON 'B1 'B4)

[[GRASP 'B1] [MOVE-OBJECT 'B1 [SPACE ABOVE 'B4 FOR 'B1]]]

(5-) (PUT-ON 'B2 'B4)

INFO: Control stack guard page unprotected

Control stack guard page temporarily disabled: proceed with caution

debugger invoked on a SB-KERNEL::CONTROL-STACK-EXHAUSTED in thread

#<THREAD "main thread" RUNNING {10041765E3}>:

  Control stack exhausted (no more space for function call frames).

This is probably due to heavily nested or infinitely recursive function

calls, or a tail call that SBCL cannot or has not optimized away.

PROCEED WITH CAUTION.

Type HELP for debugger help, or (SB-EXT:EXIT) to exit from SBCL.

(no restarts: If you didn't do this on purpose, please report it as a bug.)

(SB-KERNEL::CONTROL-STACK-EXHAUSTED-ERROR)

0] (SB-EXT:EXIT)

antti@mydebian:~/SBCL$ 

------------------------------------------------------------

yours sincerely, Dr A. J. Y.

HELSINKI

Finland, the Continental Europe

[Ramil Farkhshatov]

SB-KERNEL::CONTROL-STACK-EXHAUSTED-ERROR means that stack is overused.
It may happen on deep non-tail recursive calls. You may increase SBCL
stack-size calling it with

--control-stack-size <megabytes>

flag (default size is 2MB btw).

I am not sure that `./Shen` executable supports that flag so you may need to
rebuild it by calling sbcl with proper `--control-stack-size` value and adding
`:save-runtime-options T` to `save-lisp-and-die` function call.

It won't help if recursion depth in your program is arbitrary. In that case
using either tail recursion or custom implementation of stack would help.
(shameless-plug: Shen-js doesn't have stack size restriction, although not
as fast as Shen-sbcl).

[Ramil Farkhshatov]

Antti Ylikoski <antti.y...@gmail.com> wrote:

> I tried this one:
>
> ./Shen --control-stack-size 200MB
>
> but still I got the identical stack overflow.

Stack size change can be checked with this expression:

(lisp.eval (lisp.read-from-string "(- SB-VM:*CONTROL-STACK-END* SB-VM:*CONTROL-STACK-START*)"))

The result is in machine words (as far as I understood).

I still have a suspicion that ./Shen binary itself doesn't process flags as
`sbcl` binary so it must be rebuilt. I've added support for
`control-stack-size` in `build` script[1].

1. https://gist.github.com/gravicappa/10015430/raw/e7f42493470e25d0325221d7d8432f5bc893f804/build-shen

> I suspect there is a bug somewhere, which bug causes some kind of an
> infinite recursion.

It also may be the case.

> tiistai 29. syyskuuta 2015 11.31.42 UTC+3 Ramil Farkhshatov kirjoitti:

[Antti Ylikoski]

I tried this one:

./Shen --control-stack-size 200MB

but still I got the identical stack overflow.

I suspect there is a bug somewhere, which bug causes some kind of an infinite recursion.

Dr A. J. Y.

HELSINKI

Finland

tiistai 29. syyskuuta 2015 11.31.42 UTC+3 Ramil Farkhshatov kirjoitti:

[Pierpaolo Bernardi]

On Tue, Sep 29, 2015 at 12:16 PM, Antti Ylikoski
<antti.y...@gmail.com> wrote:

> I suspect there is a bug somewhere, which bug causes some kind of an
> infinite recursion.

This may be caused by a bug in SBCL, or by a bug in Shen.

But, just for completeness, here's a third hypotesis: could the bug
be in your code?

[Mark Tarver]

If you're loading CL I doubt very much that this is a Shen problem.   I doubt its an SBCL problem too.  Probably you've got some intensive computation that needs to be written tail recursively or simply you have non-termination i.e. the problem is in the program.

Mark

[Pierpaolo Bernardi]

On Tue, Sep 29, 2015 at 9:31 PM, Mark Tarver <dr.mt...@gmail.com> wrote:
> If you're loading CL I doubt very much that this is a Shen problem. I
> doubt its an SBCL problem too. Probably you've got some intensive
> computation that needs to be written tail recursively or simply you have
> non-termination i.e. the problem is in the program.

Actually, Antti has a point. The program runs fine from a CL. It
does not appear to use a lot of resources. I couldn't find anything
wrong in the CL program from a quick perusal (including some tracing).

At the moment I have no clues.

[Pierpaolo Bernardi]

On Tue, Sep 29, 2015 at 9:45 PM, Pierpaolo Bernardi <olop...@gmail.com> wrote:

> Actually, Antti has a point. The program runs fine from a CL. It
> does not appear to use a lot of resources. I couldn't find anything
> wrong in the CL program from a quick perusal (including some tracing).

(Assuming that the program that gives the problem is as published
here: http://koti.mbnet.fi/bluejay/SHRDLU.lisp)

[Antti Ylikoski]

Following is an example of the use of the FRL, in CL, with inheritance

through several A-KIND-OF frame slots, ie. taxonomy categories:

------------------------------------------------------------

; SLIME 2014-10-10

CL-USER> (load "FRL.lisp")

; file: /home/antti/Documents/FRL/FRL.lisp

; in: DEFUN FPUT

;     (FGET-FRAME FRAME)

; 

; caught STYLE-WARNING:

;   undefined function: FGET-FRAME

; 

; compilation unit finished

;   Undefined function:

;     FGET-FRAME

;   caught 1 STYLE-WARNING condition

; in: DEFUN FGET-I

;     (FGET-CLASSES FRAME)

; 

; caught STYLE-WARNING:

;   undefined function: FGET-CLASSES

;     (FGET-I1 (FGET-CLASSES FRAME) SLOT)

; 

; caught STYLE-WARNING:

;   undefined function: FGET-I1

; 

; compilation unit finished

;   Undefined functions:

;     FGET-CLASSES FGET-I1

;   caught 2 STYLE-WARNING conditions

; in: DEFUN FGET-CLASSES

;     (FGET-CLASSES-AUX FRAME)

; 

; caught STYLE-WARNING:

;   undefined function: FGET-CLASSES-AUX

; 

; compilation unit finished

;   Undefined function:

;     FGET-CLASSES-AUX

;   caught 1 STYLE-WARNING condition

; in: DEFUN FGET-N

;     (FGET-D FRAME SLOT)

; 

; caught STYLE-WARNING:

;   undefined function: FGET-D

;     (FGET-P FRAME SLOT)

; 

; caught STYLE-WARNING:

;   undefined function: FGET-P

; 

; compilation unit finished

;   Undefined functions:

;     FGET-D FGET-P

;   caught 2 STYLE-WARNING conditions

T

CL-USER> (fput 'antti 'a-kind-of 'value 'man)

MAN

CL-USER> (fput 'man 'a-kind-of 'value 'human)

HUMAN

CL-USER> (fput 'human 'a-kind-of 'value 'mammal)

MAMMAL

CL-USER> (fput 'mammal 'a-kind-of 'value 'vertebrate)

VERTEBRATE

CL-USER> (fput 'vertebrate 'organism 'value 'endothermic)

ENDOTHERMIC

CL-USER> (fget-i 'antti 'organism)

(ENDOTHERMIC)

CL-USER> 

------------------------------------------------------------

Following comes the same exercise with the Shen/CL connection:

------------------------------------------------------------

antti@mydebian:~/Documents/FRL$ file ./Shen

./Shen: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 2.6.32, BuildID[sha1]=4a4c4bc33cfb455f42358132ed1de8eeae24c999, stripped

antti@mydebian:~/Documents/FRL$ ls -l Shen

-rwxr-xr-x 1 antti antti 68976688 Sep 30 02:19 Shen

antti@mydebian:~/Documents/FRL$ ./Shen

Shen, copyright (C) 2010-2015 Mark Tarver

www.shenlanguage.org, Shen 19.2

running under Common Lisp, implementation: SBCL

port 2.0 ported by Mark Tarver

(0-) (LOAD "load-cl.lisp")

T

(1-) (load-cl "FRL.lisp")

PRESERVE

(2-) (FPUT 'ANTTI 'A-KIND-OF 'VALUE 'MAN)

'MAN

(3-) (FPUT 'MAN 'A-KIND-OF 'VALUE 'HUMAN)

'HUMAN

(4-) (FPUT 'HUMAN 'A-KIND-OF 'VALUE 'MAMMAL)

'MAMMAL

(5-) (FPUT 'MAMMAL 'A-KIND-OF 'VALUE 'VERTEBRATE)

'VERTEBRATE

(6-) (FPUT 'VERTEBRATE 'ORGANISM 'VALUE 'ENDOTHERMIC)

'ENDOTHERMIC

(7-) (FGET-I 'ANTTI 'ORGANISM)

[]

(8-) ((protect QUIT))

antti@mydebian:~/Documents/FRL$ 

antti@mydebian:~/Documents/FRL$ 

------------------------------------------------------------

When comparing these two runs, one can see that there is there

genuinely something wrong, somewhere.

(No criticism for Dr Tarver, really.  Both the books and the program

are truly fine.)

yours sincerely, Dr A. J. Y.

HELSINKI

Finland, the Continental Europe

[Bruno Deferrari]

Antti, you are quoting (with ') the symbols you want to pass on to PUT-ON, but in Shen thats not necessary.

I just tried your code (the one on your first message) but without the single quotes and it worked fine.

Doing the same but with the single quote prefixes gives me the same error you got:

(4-) (PUT-ON B1 B2)      
[[GRASP B1] [MOVE-OBJECT B1 [SPACE ABOVE B2 FOR B1]] [UNGRASP B1]]

(5-) (PUT-ON B2 B4)
[[GRASP B1] [MOVE-OBJECT B1 [SPACE ABOVE TABLE FOR B1]] [UNGRASP B1] [GRASP B2] [MOVE-OBJECT B2 [SPACE ABOVE B4 FOR B2]] [UNGRASP B2]]

(6-) (PUT-ON 'B1 'B4)

[[GRASP 'B1] [MOVE-OBJECT 'B1 [SPACE ABOVE 'B4 FOR 'B1]]]

(7-) (PUT-ON 'B2 'B4)

INFO: Control stack guard page unprotected
Control stack guard page temporarily disabled: proceed with caution

debugger invoked on a SB-KERNEL::CONTROL-STACK-EXHAUSTED in thread

#<THREAD "main thread" RUNNING {1003126583}>:

  Control stack exhausted (no more space for function call frames).
This is probably due to heavily nested or infinitely recursive function
calls, or a tail call that SBCL cannot or has not optimized away.

PROCEED WITH CAUTION.

Type HELP for debugger help, or (SB-EXT:EXIT) to exit from SBCL.

(no restarts: If you didn't do this on purpose, please report it as a bug.)

(SB-KERNEL::CONTROL-STACK-EXHAUSTED-ERROR)

--
You received this message because you are subscribed to the Google Groups "Shen" group.
To unsubscribe from this group and stop receiving emails from it, send an email to qilang+un...@googlegroups.com.
To post to this group, send email to qil...@googlegroups.com.

Visit this group at http://groups.google.com/group/qilang.
For more options, visit https://groups.google.com/d/optout.

--

BD

[Bruno Deferrari]

Ok, forget what I said, I closed Shen-SBCL, repeated everything again, but this time it failed.

--

BD

[Bruno Deferrari]

Doing the same but with Shen-CLisp results in a stack overflow too.

--

BD

[fuzzy wozzy]

of course stack overflow occurs when you use stack too much,
for example if you iterate from 1 to 10 million using list, the first couple of times will be fine,
after that you will get stack overflow message sooner or later if you repeat the same iteration,

closing and reopening shen repl sometimes isn't enough, you have to restart the computer even,
so if this is the case, then it makes sense that it works fine without a quote but when repeating,
stack is exhuasted, thus stack overflow error.

if you exhaust stack space using sbcl, then don't think that moving over to clisp will solve it,
your computer will try to use the same stack and find that there is no space, then just restart the computer.

dr. tarver already said as much, just wondering why people must go on speculating nonetheless...

[Bruno Deferrari]

On Tue, Sep 29, 2015 at 11:35 PM, fuzzy wozzy <swtch...@gmail.com> wrote:

of course stack overflow occurs when you use stack too much,
for example if you iterate from 1 to 10 million using list, the first couple of times will be fine,
after that you will get stack overflow message sooner or later if you repeat the same iteration,

closing and reopening shen repl sometimes isn't enough, you have to restart the computer even,
so if this is the case, then it makes sense that it works fine without a quote but when repeating,
stack is exhuasted, thus stack overflow error.

if you exhaust stack space using sbcl, then don't think that moving over to clisp will solve it,
your computer will try to use the same stack and find that there is no space, then just restart the computer.

Not necessarily. CLisp and SBCL implement the same language, but are very different implementations, in particular, they are different in how they handle tail calls (in particular, CLisp optimises only self tail-calls, but not the more general version, SBCL on the other hand does handle the more general case, but that depends on the debug settings). Other than implementation differences, even if unlikely, a particular version either CLisp or SBCL can be buggy, having the same program crash in the same way in both implementations discards that possibility in far less time that it takes to analyse the code.

dr. tarver already said as much, just wondering why people must go on speculating nonetheless...

Because running the code takes a minute and already discards some possibilities. Reading the code Antti is loading to try to find what could be causing a stack overflow is time consuming, more so considering that it is not going to be obvious what to look for knowing that the code doesn't result in a stack overflow in plain Common Lisp when running under either CLisp or SBCL, but fails under Shen's REPL in both versions.

Anyway, Antti, the issue is what I mentioned on my initial email, the reason it crashed for me the second time is that I made a mistake and didn't call (SET-BLOCKS) in that one opportunity, so it was crashing for a different reason.

After looking at set-block's source code I get an idea of why it breaks. The list of known objects is fixed, and calling PUT-ON with the name of an object it doesn't know about causes problems (why? I don't know, I didn't read that much code).

If you don't try to quote your symbols in Shen when calling PUT-ON (so that the actual symbol gets passed, instead of something like _quote1957B4 which is how Shen passes the symbol down to Common Lisp if you add a quote), it should work fine.

 

--
You received this message because you are subscribed to the Google Groups "Shen" group.
To unsubscribe from this group and stop receiving emails from it, send an email to qilang+un...@googlegroups.com.
To post to this group, send email to qil...@googlegroups.com.
Visit this group at http://groups.google.com/group/qilang.
For more options, visit https://groups.google.com/d/optout.

--

BD

[fuzzy wozzy]

maybe the easiest thing to do is for someone to write it in shen?

[Antti Ylikoski]

Thank you

But I still get the same error, unfortunately:

-----------------------------------------------------------------------

antti@mydebian:~/SBCL$ ./Shen

Shen, copyright (C) 2010-2015 Mark Tarver

www.shenlanguage.org, Shen 19.2

running under Common Lisp, implementation: SBCL

port 2.0 ported by Mark Tarver

(0-) (LOAD "load-cl.lisp")

T

(1-) (load-cl "SHRDLU.lisp")

PRESERVE

(2-) (PUT-ON B1 B4)

[[GRASP B1] [MOVE-OBJECT B1 [SPACE ABOVE B4 FOR B1]]]

(3-) (PUT-ON B2 B1)

INFO: Control stack guard page unprotected

Control stack guard page temporarily disabled: proceed with caution

debugger invoked on a SB-KERNEL::CONTROL-STACK-EXHAUSTED in thread

#<THREAD "main thread" RUNNING {10041765E3}>:

  Control stack exhausted (no more space for function call frames).

This is probably due to heavily nested or infinitely recursive function

calls, or a tail call that SBCL cannot or has not optimized away.

PROCEED WITH CAUTION.

Type HELP for debugger help, or (SB-EXT:EXIT) to exit from SBCL.

(no restarts: If you didn't do this on purpose, please report it as a bug.)

(SB-KERNEL::CONTROL-STACK-EXHAUSTED-ERROR)

0] (SB-EXT:EXIT)

antti@mydebian:~/SBCL$

---------------------------------------------------------

Yours, Dr AJY

HELSINKI

Finland

[Antti Ylikoski]

Sorry - I BLUNDERED

This kind of thing may happen if one is so very enthusiastic about computer science that he will stay awake overnight working.......

I simply forgot to call (SET-BLOCKS) in the Shen version.  Dr Tarver's code is impeccable.

Sorry Folks

Dr AJY

HELSINKI

Finland