refcounts and DOD

[Michal Wallace]

Hi all,

So I'm still thinking about a generic
wrapper for python modules. I would like
to be able to recompile the python standard
library (and other libraries) to run on parrot
with only a few minor patches.

I realize this is probably completely foolish,
but I'm lazy, so... :)

I've done experiments in the past running
PythonObjects outside the python VM. This
should be possible by replacing a few generic
routines that refer to the interpreter.

One of the major issues is garbage collection.
Python's VM does refcounting. There are two
macros it uses: Py_INCREF(x) and Py_DECREF(x).
They're pretty self-explanitory, but the docs
are here:

http://www.python.org/doc/2.4/ext/refcountsInPython.html

It seems to me that these macros could be
replaced with something that puts the objects
in the right place for parrot to deal with them.

I've read the memory_internals and pdd09_gc
docs but I'm still trying to understand how
this would work.

It seems that instead of looking at the *count*
of references, the DOD system actually walks
through the graph of references. So it seems
you could fake refcounting just by adding
references and removing pointers from
somewhere in the tree that gets walked.

So: Py_INCREF(x) could be rewritten to (for example)
append a reference to x in a parrot array,
and Py_DECREF(x) would pop it off the array.

Am I on the right track here? I'm sure this is
a very naive and inefficient implementation, but
I don't see why it wouldn't work. If it would work,
is there a better way?

Also, my understanding is that parrot uses only
a Mark and Sweep system. Is that correct? pdd09
describes the three general methods, but doesn't
say which one(s) parrot actually usess. memory_internals.html
seems to imply the mark and sweep system only...
Or does parrot do all three?

- Michal
http://withoutane.com/

[Nicholas Clark]

On Wed, May 25, 2005 at 06:08:42PM -0400, Michal Wallace wrote:

> So: Py_INCREF(x) could be rewritten to (for example)
> append a reference to x in a parrot array,
> and Py_DECREF(x) would pop it off the array.

I think that you can use Parrot_register_pmc and Parrot_unregister_pmc
IIRC they count up the number of registrations, and only stop tracking that
PMC when the count returns to zero.

Nicholas Clark

[Leopold Toetsch]

Michal Wallace wrote:

> It seems that instead of looking at the *count*
> of references, the DOD system actually walks
> through the graph of references. So it seems
> you could fake refcounting just by adding references and removing
> pointers from somewhere in the tree that gets walked.

Yes, as Nick said, using the DOD register opcodes/API is an albeit slow
equivalent for the Py_{INC,DEC}REF() macros.

> So: Py_INCREF(x) could be rewritten to (for example)
> append a reference to x in a parrot array, and Py_DECREF(x) would pop it
> off the array.

Actually its a hash with the PMC's address being the key and the value
is the reference count.

> Am I on the right track here? I'm sure this is
> a very naive and inefficient implementation, but
> I don't see why it wouldn't work. If it would work,
> is there a better way?

Not really, except the compiler can determine that the object is live
anyway, because it's used elesewhere too.

> Also, my understanding is that parrot uses only
> a Mark and Sweep system. Is that correct? pdd09
> describes the three general methods, but doesn't
> say which one(s) parrot actually usess. memory_internals.html
> seems to imply the mark and sweep system only...
> Or does parrot do all three?

Parrot has currently a Mark & Sweep GC for PObjs (PMCs, Strings,...) -
the DOD in the docs and a Mark & Copy collector for string and buffer
memory (the GC in the docs). The latter depends on the mark phase of the
Mark & Sweep collector.

But the implementation is likely to change.

> - Michal

leo

[Dan Sugalski]

At 6:08 PM -0400 5/25/05, Michal Wallace wrote:
>So I'm still thinking about a generic
>wrapper for python modules. I would like to be able to recompile the
>python standard library (and other libraries) to run on parrot
>with only a few minor patches.

If you're doing this to make the python library parrot extensions,
then just go and make the inc/dec macros noops, since they're not
necessary what with us tracking the pointers and all. (The only time
they'll be needed is if the pointers to the refcounted things are in
places parrot can't find 'em)
--
Dan

--------------------------------------it's like this-------------------
Dan Sugalski even samurai
d...@sidhe.org have teddy bears and even
teddy bears get drunk