ACL and low-level GC
David Bakhash
I'm using ACL, v4.3. I have noticed something which probably
shouldn't scare me too much, but i'd like some justification.
I started Lisp going, and then I stored a huge Lisp object. It raised
the size of the Lisp image to 18 Meg. Then I deleted that object and
did an (excl:gc t) (i.e. a global GC). So, of course it took a while
to do it. But what I found was that the size of the process didn't
change. i.e. the Lisp process itself didn't go down in size.
Actually, I wasn't really expecting it to. You figure that Lisp had
already asked the OS to allocate a chunk of memory for its deeds, and
the OS gave Lisp that memory. I guess Lisp wouldn't want to give that
up, and then (later) have to do it all over again.
On the other hand, you figure that every time Lisp does a global GC,
it's gotta go through all that memory again. I wonder the following:
can Lisp be told to free memory back to the OS? I know this is
low-level stuff, but for my app, it would be a nice option. Even
if its implementation-specific, I'd like to know about it. (note,
this is similar to what (I think) (X)Emacs does with buffers. I
believe that (X)Emacs uses (s)brk, or maybe some version of malloc
that allows similar behavior.
dave
Erik Naggum
| On the other hand, you figure that every time Lisp does a global GC, it's
| gotta go through all that memory again.
this is a false assumption.
| I wonder the following:
|
| can Lisp be told to free memory back to the OS?
yes. if you use 4.3 under Linux, the manuals came in PDF format. the
chapter on garbage collection is really very good, and a lot better
source of information than any rehashed understanding anyone might
provide here.
#:Erik
--
Attention, Republican members of the House Judiciary Committee! We have
intercepted a coded transmission from Bill Clinton to Saddam Hussein that
puts your life in jeopardy. Clinton is prepared to cease fire if all of
you are killed by Iraqi terrorists, whom he won't prosecute. Be warned!
David Bakhash
> * David Bakhash <ca...@bu.edu>
> | On the other hand, you figure that every time Lisp does a global GC, it's
> | gotta go through all that memory again.
>
> this is a false assumption.
Though I don't know the internals of ACL nearly as well as you, I have
noticed that the two global GCs in a row (after allocating that 18 meg
block) are _both_ pretty slow. This implies that the 2nd GC was
spending significant time in that block, even though it had been GC'd
just before.
dave
Martti Halminen
> | I wonder the following:
> |
> | can Lisp be told to free memory back to the OS?
>
> yes. if you use 4.3 under Linux, the manuals came in PDF format. the
> chapter on garbage collection is really very good, and a lot better
> source of information than any rehashed understanding anyone might
> provide here.
There might be more to this than can be found in that manual; I've been
seeing a situation where, when running the same application both on ACL
4.3 on a Sparcstation and ACL 4.3.2 on NT, a global GC on Sun does free
memory to the OS, but on NT it doesn't.
Any ideas on what is going on?
--
________________________________________________________________
^. Martti Halminen
/ \`. Design Power Europe Oy
/ \ `. Tekniikantie 12, FIN-02150 Espoo, Finland
/\`. \ | Tel:+358 9 4354 2306, Fax:+358 9 455 8575
/__\|___\| Mailto:Martti....@dpe.fi http://www.dpe.fi
Axel Schairer
> On the other hand, you figure that every time Lisp does a global GC,
> following:
> block) are _both_ pretty slow. This implies that the 2nd GC was
> spending significant time in that block, even though it had been GC'd
> just before.
I assume that storing a huge object and deleting it is something like
(setf x (comput-huge-object))
(setf x nil)
If you do this in the repl you end up referencing the huge object
through **. And if you emit the gc command, the object is still in
***. At least with ACL 4.3 for Linux you get a smaller process size
if you throw the object out of the history mechanism (see example
session at the end of this post).
So your second GC may spend its time copying the huge object, not
because it has to look at all the memory (that would be too bad
indeed; and is, as far as I know, no longer true for basically any
serious GC implementation. am I wrong here?) but because the huge
object is still reachable.
Hope this helps,
Axel
Example session:
; Size=707
(setf x (compute-huge-object)) ; Size now=4662
(setf x nil) ; Size now=4663
(excl::gc t) ; Size now=5330
*** ; Size now=5332
;; => #<the huge object>
;; Type nil at repl three times, increases size slightly
(excl:gc t) ; Size now=1096
--
=== Axel Schairer, http://www.dfki.de/~schairer/ ===
Espen Vestre
> Erik Naggum <er...@naggum.no> writes:
>
> > * David Bakhash <ca...@bu.edu>
> > | On the other hand, you figure that every time Lisp does a global GC, it's
> > | gotta go through all that memory again.
> >
> > this is a false assumption.
>
> Though I don't know the internals of ACL nearly as well as you, I have
> noticed that the two global GCs in a row (after allocating that 18 meg
> block) are _both_ pretty slow. This implies that the 2nd GC was
> spending significant time in that block, even though it had been GC'd
> just before.
maybe this was due to _swapping_, not the GC itself?
--
espen
Martin Rodgers
> Though I don't know the internals of ACL nearly as well as you, I have
> noticed that the two global GCs in a row (after allocating that 18 meg
> block) are _both_ pretty slow. This implies that the 2nd GC was
> spending significant time in that block, even though it had been GC'd
> just before.
There ain't no such thing as a free lunch. Henry Baker's "Thermodynamics
and Garbage Collection" paper explains how this relates to memory.
Examine your assumptions, and then throw them away.
First, make damn sure you're not still refering to your object. Lisp has
features to help you hang on to an object. Be sure you're not accidently
using any of them. This is the general advice.
Creating a huge Lisp object is always going to cost _something_. You
don't deleted an object when you use a GC. You just forget it, and
eventually the GC will reclaim the space it occupied. As you've noticed,
this has a side effect. Well, so does everything you do!
You've chosen to use a tool that manages memory for you. That's what it
does. Now you can do something more productive. If you're still
interested in managing memory then you may have study the subject and
your chosen tool in greater detail.
--
Remove insect from address to email me | You can never browse enough
will write code that writes code that writes code for food
Mike McDonald
Martti Halminen <m...@dpe.fi> writes:
> Erik Naggum wrote:
>
>> | I wonder the following:
>> |
>> | can Lisp be told to free memory back to the OS?
>>
>> yes. if you use 4.3 under Linux, the manuals came in PDF format. the
>> chapter on garbage collection is really very good, and a lot better
>> source of information than any rehashed understanding anyone might
>> provide here.
>
> There might be more to this than can be found in that manual; I've been
> seeing a situation where, when running the same application both on ACL
> 4.3 on a Sparcstation and ACL 4.3.2 on NT, a global GC on Sun does free
> memory to the OS, but on NT it doesn't.
> Any ideas on what is going on?
A lot of OS's don't have any way for a process to return a page of memory to
the OS, other than when the process dies. I don't know if this is true of NT
or not. I do know that as of a couple of years ago, most Unixes couldn't do
it.
Mike McDonald
mik...@mikemac.com
Erik Naggum
| Though I don't know the internals of ACL nearly as well as you,
I have only read the excellent manuals.
| I have noticed that the two global GCs in a row (after allocating that 18
| meg block) are _both_ pretty slow. This implies that the 2nd GC was
| spending significant time in that block, even though it had been GC'd
| just before.
I thought you said you had deleted the object and two global GC's in a
row were still pretty slow? I can't reproduce that. global GC takes the
same time no matter how much free space there is in the old space.