Heap exhausted, game over.

jimka...@gmail.com

unread,

Jun 23, 2015, 8:33:31 AM6/23/15

to

I'm working on a program that I don't not yet understand, which was edited by lots of people before me. After running a few tests, sbcl always seems to exhaust its heap, and dump me, with a very welcoming message, into the low-level debugger.

I'm hoping someone can give me a suggestion about how to debug this problem.
For example, can I get some sort of summary of what kind of memory is allocated?
Or do I simply need to get a bigger machine?

Heap exhausted during garbage collection: 32 bytes available, 48 requested.
Gen StaPg UbSta LaSta LUbSt Boxed Unboxed LB LUB !move Alloc Waste Trig WP GCs Mem-age
0: 0 0 0 0 0 0 0 0 0 0 0 10737418 0 0 0.0000
1: 0 0 0 0 0 0 0 0 0 0 0 10737418 0 0 0.0000
2: 32689 0 0 0 15885 3 0 0 9 520122288 495696 10737418 0 0 1.2521
3: 3756 3391 0 0 1245 123 4043 60 43 178064048 1209680 2000000 3230 0 0.0000
4: 0 0 0 0 0 0 0 0 0 0 0 2000000 0 0 0.0000
5: 0 0 0 0 0 0 0 0 0 0 0 2000000 0 0 0.0000
6: 0 0 0 0 1220 241 0 0 0 47874048 0 2000000 1066 0 0.0000
Total bytes allocated = 1071924032
Dynamic-space-size bytes = 1073741824
GC control variables:
*GC-INHIBIT* = true
*GC-PENDING* = true
fatal error encountered in SBCL pid 2890:
Heap exhausted, game over.

Error opening /dev/tty: Device not configured
Welcome to LDB, a low-level debugger for the Lisp runtime environment.
ldb>

Matthew Carter

unread,

Jun 23, 2015, 11:36:37 AM6/23/15

to

Shot in the dark - but are you using some tail recursion thats
exhausting your stack?

You can also set the dynamic space size on sbcl start up (I think the
option you want is --dynamic-space-size).

When running an sbcl instance with my lisp code on a tiny VPS (~400MB) I
had some problems similar to your own, and using a smaller dynamic space
size seemed to help.

--
Matthew Carter (m...@ahungry.com)
http://ahungry.com

jimka...@gmail.com

unread,

Jun 24, 2015, 5:02:09 AM6/24/15

to

I notice in the output that *GC-INHIBIT* has a value of true.
Is this normal? Does this mean something in my program has explicitly disabled GC?

I've grepped for "inhibit" in the code base and didn't find it.
What else might I search for to see of some function has disabled the GC, and perhaps forgotten to turn it back on?

Matthew Carter

unread,

Jun 24, 2015, 12:49:01 PM6/24/15

to

Maybe this is related to your issue?

https://bugs.launchpad.net/sbcl/+bug/1446962

Large arrays that are using #'declare causing crashes.

Maybe take out any #'declare calls you have and use basic array setups?

jimka...@gmail.com

unread,

Jun 26, 2015, 8:01:56 AM6/26/15

to

In the case you reference, I also see that *GC-INHIBIT* = true
I guess that means that my code is probably not intentionally disabling GC.
BTW, the code I'm running does indeed allocate large arrays.
It does not seem to use any type declarations.

Does anyone know if I can ask ldb to print me out a summary of how much memory each type is using?

Jim Newton

unread,

Jun 29, 2015, 7:40:41 AM6/29/15

to

I have a program which demonstrates this issue.
Maybe I'm really allocating more memory than my machine has?

(defclass cell ()
((data :initarg :data)))

(defun test-allocate (x)
(declare (optimize (safety 3) (debug 3) (space 0) (speed 0)))

(let ((w 2562)
(h 1602)
(d 3))
(let ((arr2d (make-array (list w h)))
(arr3d (make-array (list w h d))))

(dotimes (i w)
(dotimes (j h)
(let ((vec (make-array (list d))))
(let ((value (make-instance 'cell :data vec)))
(setf (aref arr2d i j) value)
(dotimes (k d)
(setf (aref vec k) x))))))

(dotimes (i w)
(dotimes (j h)
(dotimes (k d)
(setf (aref arr3d i j k) x))))

(list arr2d arr3d))))

(test-allocate 3.1)

* Heap exhausted during garbage collection: 0 bytes available, 32 requested.

Gen StaPg UbSta LaSta LUbSt Boxed Unboxed LB LUB !move Alloc Waste Trig WP GCs Mem-age
0: 0 0 0 0 0 0 0 0 0 0 0 10737418 0 0 0.0000
1: 0 0 0 0 0 0 0 0 0 0 0 10737418 0 0 0.0000

2: 1515 0 0 0 14529 90 0 0 43 478264576 770816 2000000 0 0 2.1092
3: 0 0 0 0 0 0 0 0 0 0 0 2000000 0 0 0.0000

4: 0 0 0 0 0 0 0 0 0 0 0 2000000 0 0 0.0000
5: 0 0 0 0 0 0 0 0 0 0 0 2000000 0 0 0.0000

6: 0 0 0 0 1220 241 0 0 0 47874048 0 2000000 1092 0 0.0000
Total bytes allocated = 1072272272

Dynamic-space-size bytes = 1073741824
GC control variables:
*GC-INHIBIT* = true
*GC-PENDING* = true

fatal error encountered in SBCL pid 9337:

Jim Newton

unread,

Jun 29, 2015, 7:42:57 AM6/29/15

to

Here is the info about my laptop, how large an array should I be able to allocate?

Hardware Overview:

Model Name: MacBook Pro
Model Identifier: MacBookPro6,2
Processor Name: Intel Core i5
Processor Speed: 2,4 GHz
Number of Processors: 1
Total Number of Cores: 2
L2 Cache (per Core): 256 KB
L3 Cache: 3 MB
Memory: 8 GB
Processor Interconnect Speed: 4.8 GT/s
Boot ROM Version: MBP61.0057.B0F
SMC Version (system): 1.58f17
Serial Number (system): W80171WEAGX
Sudden Motion Sensor:
State: Enabled

Pascal J. Bourguignon

unread,

Jun 29, 2015, 8:03:19 AM6/29/15

to

Jim Newton <jimka...@gmail.com> writes:

> I have a program which demonstrates this issue.
> Maybe I'm really allocating more memory than my machine has?

No, just more memory than the artificial limit that sbcl imposes on the
heap size. You can ask for more with the --dynamic-space-size option.

--
__Pascal Bourguignon__ http://www.informatimago.com/
“The factory of the future will have only two employees, a man and a
dog. The man will be there to feed the dog. The dog will be there to
keep the man from touching the equipment.” -- Carl Bass CEO Autodesk

Message has been deleted

Jim Newton

unread,

Jun 29, 2015, 8:09:46 AM6/29/15

to

oops, victim of mac-auto-correction. Sorry.

It seems to work fine in CLISP ... I have not tried anything else yet.

Jim Newton

unread,

Jun 29, 2015, 8:24:04 AM6/29/15

to

Hi Pascal, do you happen to know how --dynamic-space-size is supposed to work?
The dynamic space size (by default) seems to be 1073741824.
I cannot run sbcl with twice this. But if use a much smaller number,
sbcl seems to be much happier, and my program seems to run with no heap allocation problems,
at least at first test.

csh> sbcl --dynamic-space-size 3741824
This is SBCL 1.2.12.61-9741c60, an implementation of ANSI Common Lisp.
More information about SBCL is available at <http://www.sbcl.org/>.

SBCL is free software, provided as is, with absolutely no warranty.
It is mostly in the public domain; some portions are provided under
BSD-style licenses. See the CREDITS and COPYING files in the
distribution for more information.
* (compile-file "~/common-lisp/climb/alloc.lisp")
(compile-file "~/common-lisp/climb/alloc.lisp")

; compiling file "/Users/jnewton/sw/climb/alloc.lisp" (written 29 JUN 2015 01:51:06 PM):
; compiling (DEFCLASS CELL ...)
; compiling (DEFUN TEST-ALLOCATE ...)

; ~/common-lisp/climb/alloc.fasl written
; compilation finished in 0:00:00.065
#P"/Users/jnewton/sw/climb/alloc.fasl"
NIL
NIL
* (load "~/common-lisp/climb/alloc.fasl")
(load "~/common-lisp/climb/alloc.fasl")

T
* (test-allocate 3.1)
(test-allocate 3.1)

2
* (test-allocate 3.1)
(test-allocate 3.1)

2
* (test-allocate 3.1)
(test-allocate 3.1)

2
* (test-allocate 3.1)
(test-allocate 3.1)

2
* (test-allocate 3.1)
(test-allocate 3.1)

2
* (test-allocate 3.1)
(test-allocate 3.1)

2
* (test-allocate 3.1)
(test-allocate 3.1)

Pascal J. Bourguignon

unread,

Jun 29, 2015, 8:37:33 AM6/29/15

to

Jim Newton <jimka...@gmail.com> writes:

> Hi Pascal, do you happen to know how --dynamic-space-size is supposed
> to work?

Why do you ask me? How would I know? Ask sbcl! Ask the manual!

sbcl --help
man sbcl

Xiaogang Liu

unread,

Jun 30, 2015, 9:15:43 PM6/30/15

to

On Monday, June 29, 2015 at 10:24:04 PM UTC+10, Jim Newton wrote:
> Hi Pascal, do you happen to know how --dynamic-space-size is supposed to work?

Hi Jim,

A quick look at the manual shows that the command line options --dynamic-space-size accepts numbers in MB instead of bytes. The number 1073741824 (1024MB) in your dump is in BYTES. So the correct command line to double the size is sbcl --dynamic-space-size 2048

Cheers,
Xiaogang