global destruction
Nicholas Clark
"destruction" and "finalisation". One is about cleanup actions on objects/
PMCs that need something actively cleaned up, rendering active objects down
to bare memory, and the other is about reclaiming bare memory for re-use.
For starters, I'm interested in the first one - rendering down active objects
to bare memory, whatever it's called. I'm also not directly interested in
"high priority DOD" for "timely destruction" of hot objects. What I want to
know is
does parrot make any guarantee that all objects will be rendered down to
bare memory before program exit?
At interpreter exit perl 5 scans all the active allocated scalars to find
objects, and causes all references to them to be undefined, which will cause
DESTROY to be called on all of them. This will even catch objects trapped in
reference counting loops.
Second question is that optionally perl 5 can run with complete global
destruction. This is primarily intended for embedded interpreters, where
the default implementation (just exit the process to free all memory) is
clearly not an option. Does parrot have any sort of built in ability to
change mode to force all memory ever allocated to be freed up?
At the moment ponie is tracking all PMCs it allocates so that it can emulate
both the object destruction, and the full destruction. I'm wondering if there
is parrot infrastructure I can use to avoid needing to do this.
Nicholas Clark
Leopold Toetsch
> For starters, in glossary.pod I'm failing to find definitions for
> "destruction" and "finalisation". One is about cleanup actions on objects/
> PMCs that need something actively cleaned up, rendering active objects down
> to bare memory, and the other is about reclaiming bare memory for re-use.
1) destruction: low-level object cleanup like free(3) of malloced memory
2) finalisation: high-level, user-visible, overridable, e.g. close DB
connection
> For starters, I'm interested in the first one - rendering down active objects
> to bare memory, whatever it's called. I'm also not directly interested in
> "high priority DOD" for "timely destruction" of hot objects. What I want to
> know is
>
> does parrot make any guarantee that all objects will be rendered down to
> bare memory before program exit?
Parrot calls the "destroy" vtable function for all PMCs that have marked
themselves with PObj_active_destroy_FLAG, when the object gets recycled
due to GC *or* at program exit.
There is no finalise or finalize vtable function yet. I think that these
two task should be distinct
> At interpreter exit perl 5 scans all the active allocated scalars to find
> objects, and causes all references to them to be undefined, which will cause
> DESTROY to be called on all of them. This will even catch objects trapped in
> reference counting loops.
I presume this is the DESTROY {} user hook. That should probably be
called from VTBAE_finali[sz]e before VTABLE_destroy. What about ordering
these calls?
> Second question is that optionally perl 5 can run with complete global
> destruction. This is primarily intended for embedded interpreters, where
> the default implementation (just exit the process to free all memory) is
> clearly not an option. Does parrot have any sort of built in ability to
> change mode to force all memory ever allocated to be freed up?
parrot --leak-test # or the same:
parrot --destroy-at-end
should free all memory allocated (but doesn't yet).
> Nicholas Clark
leo
Nicholas Clark
> Nicholas Clark wrote:
> >For starters, in glossary.pod I'm failing to find definitions for
> >"destruction" and "finalisation". One is about cleanup actions on objects/
> >PMCs that need something actively cleaned up, rendering active objects down
> >to bare memory, and the other is about reclaiming bare memory for re-use.
>
> 1) destruction: low-level object cleanup like free(3) of malloced memory
> 2) finalisation: high-level, user-visible, overridable, e.g. close DB
> connection
Thanks. I added 2 paragraphs to glossary.pod to record this.
> > does parrot make any guarantee that all objects will be rendered down to
> > bare memory before program exit?
>
> Parrot calls the "destroy" vtable function for all PMCs that have marked
> themselves with PObj_active_destroy_FLAG, when the object gets recycled
> due to GC *or* at program exit.
>
> There is no finalise or finalize vtable function yet. I think that these
> two task should be distinct
It seems reasonable to make these 2 tasks distinct, given that only finali[zs]e
needs to be explicit, and avoiding destroy with exit() could be a big
efficiency gain.
But what we have is already useful enough for Ponie. Thanks for the
clarification.
Nicholas Clark
Nicholas Clark
> Nicholas Clark wrote:
> >Second question is that optionally perl 5 can run with complete global
> >destruction. This is primarily intended for embedded interpreters, where
> >the default implementation (just exit the process to free all memory) is
> >clearly not an option. Does parrot have any sort of built in ability to
> >change mode to force all memory ever allocated to be freed up?
>
> parrot --leak-test # or the same:
> parrot --destroy-at-end
>
> should free all memory allocated (but doesn't yet).
Are these likely to get implemented soon?
And will there be a way for anyone embedding a parrot interpreter to easily
turn them on via an API?
Nicholas Clark
Leopold Toetsch
On Sep 10, 2005, at 21:58, Nicholas Clark wrote:
>
> On Wed, Sep 07, 2005 at 02:37:33PM +0200, Leopold Toetsch wrote:
>> parrot --leak-test # or the same:
>> parrot --destroy-at-end
>>
>> should free all memory allocated (but doesn't yet).
>
> Are these likely to get implemented soon?
It is implemented, but e.g. STD* IO objects are leaking memory.
> And will there be a way for anyone embedding a parrot interpreter to
> easily
> turn them on via an API?
Parrot_set_flag(interp, PARROT_DESTROY_FLAG);
> Nicholas Clark
leo
Nicholas Clark
> Parrot calls the "destroy" vtable function for all PMCs that have marked
> themselves with PObj_active_destroy_FLAG, when the object gets recycled
> due to GC *or* at program exit.
If I have 2 objects in a self-referencing loop, what happens?
A---+
^ |
| v
+---B
Clearly mark and sweep can spot these things before the end of the world,
but how does all the destruction work? The GC has to call one PMC's destroy
first, so let's call that one A.
A's destroy gets called. At this time A is pointing to a valid B object,
so it can make calls on B.
At the end of this call, A is (presumably) in a consistent state, but not
a state consistent with having other methods called on it. But is its
vtable still active? Or is the vtable replaced with something else?
Because what must come next is that B's destroy gets called. And as B still
has a pointer to A, B might (naïvely) try to make calls on A.
Or are all objects that actively destroy themselves also expected to
actively swap their vtables to something safe?
I'm not sure if this messy "call on a destructed object" scenario can occur
for non-active-destroy objects.
Nicholas Clark
Matt Fowles
Dan's plan was to do a topographic sort of the object tree to avoid
this problem for non-loops and to break loops randomly. I believe he
was even tempted to call rand in there just to make sure people didn't
come to depend on a behavior...
I think it might be wise for us to make all the destruction calls
before memory reclamation and replace each objects vtable with undef
after its call.
Matt
--
"Computer Science is merely the post-Turing Decline of Formal Systems Theory."
-Stan Kelly-Bootle, The Devil's DP Dictionary
Leopold Toetsch
On Sep 15, 2005, at 18:29, Nicholas Clark wrote:
>
> On Wed, Sep 07, 2005 at 02:37:33PM +0200, Leopold Toetsch wrote:
>
>> Parrot calls the "destroy" vtable function for all PMCs that have
>> marked
>> themselves with PObj_active_destroy_FLAG, when the object gets
>> recycled
>> due to GC *or* at program exit.
>
> If I have 2 objects in a self-referencing loop, what happens?
>
> A---+
> ^ |
> | v
> +---B
>
> Clearly mark and sweep can spot these things before the end of the
> world,
> but how does all the destruction work? The GC has to call one PMC's
> destroy
> first, so let's call that one A.
>
> A's destroy gets called. At this time A is pointing to a valid B
> object,
> so it can make calls on B.
destruction (the low-level kind aka the free(3) thingy) doesn't have
problems with such circular dependencies. At least if these rules are
followed:
* code your vtable destroy handler conservatively:
if (SELF->mem) {
free(SELF->mem);
SELF->mem = NULL;
}
* don't call VTABLE_destroy on parent PMCs or such - GC calls destroy
for you (if the object is marked with the active_destroy flag bit)
Therefore above case can't happen - you just don't call B's destroy
method.
But all you are mentioning is true for VTABLE_finalize, which is
unimplemented. It will have to deal with finalize dependencies and even
with user code that tries to revive already dead objects.
> At the end of this call, A is (presumably) in a consistent state, but
> not
> a state consistent with having other methods called on it. But is its
> vtable still active? Or is the vtable replaced with something else?
For the finalize case: we have to put PMCs that need finalization on a
separate list during GC, then sort this list (to our best knowledge),
call finalize handlers on all, and the eventually destroy the real dead
(or maybe revive some others (if possible).
The objects are still intact at this time, they just have the state
"doomed".
> Because what must come next is that B's destroy gets called. And as B
> still
> has a pointer to A, B might (naïvely) try to make calls on A.
I think there are two kinds of implementation schemes:
- Parrot calls finalize in MRO order
- no automatic finalize calls are done - it's up to user code to do the
rigth thing
This isn't speced yet. The finalize relations A->B and B->A could also
be arbitray and outside the OO model. Then it's of course up to the
user code, to set some flags, or nullify some fields, so that it's
known that finalization is already done.
> Or are all objects that actively destroy themselves also expected to
> actively swap their vtables to something safe?
That's probably as safe as demanding: just don't do unsafe operations
;-)
> Nicholas Clark
leo
Nicholas Clark
> Nick~
>
> Dan's plan was to do a topographic sort of the object tree to avoid
> this problem for non-loops and to break loops randomly. I believe he
> was even tempted to call rand in there just to make sure people didn't
> come to depend on a behavior...
I agree with the use of rand.
> I think it might be wise for us to make all the destruction calls
> before memory reclamation and replace each objects vtable with undef
> after its call.
Setting to undef is in effect what Perl 5 does (although this can only
happen at global destruction time), because when a scalar is reclaimed,
its 24 flag bits are set to all zero, which is consistent with undef.
I'm not sure that there is any other safe way, and it seems better that the
VM does it rather than rely on every PMC duplicating the code. Although the
idea of a PMC that has both finalize and destroy vtable entries clouds the
issue, because it would still need to see its vtable for destroy.
I wonder if it makes sense for these to have an undef derivative that has
destroy and actually stores 1 item - the original vtable entry, and swaps
that vtable back in just to call destroy.
Nicholas Clark
Matt Fowles
On 9/15/05, Nicholas Clark <ni...@ccl4.org> wrote:
I disagree with the choice to have both finalize and destroy... If we
must have both we could ensure that destroy is called immediately
after finalize (but before VTABLE swap).
Matt