Short-lived memory allocation
Luke Palmer
needed a little dynamically allocated memory, just for the duration of
the function. How would be the preferred way of allocating (and
deallocating) it? Should I just use malloc(), or is there a Parrot
alternative?
Thanks,
Luke
Dan Sugalski
The best thing to do is allocate a buffer and call into
Parrot_allocate to fill it. The allocation's pretty fast, there's no
free list to worry about, leaks aren't an issue (so you don't have to
worry about cleaning up), and if for some reason something deep in
your routine throws an exception and backs way out you won't end up
with orphaned memory.
--
Dan
--------------------------------------"it's like this"-------------------
Dan Sugalski even samurai
d...@sidhe.org have teddy bears and even
teddy bears get drunk
Luke Palmer
> >While working on this patch (my first one! yay for me!) I noticed I
> >needed a little dynamically allocated memory, just for the duration of
> >the function. How would be the preferred way of allocating (and
> >deallocating) it? Should I just use malloc(), or is there a Parrot
> >alternative?
>
> The best thing to do is allocate a buffer and call into
> Parrot_allocate to fill it. The allocation's pretty fast, there's no
> free list to worry about, leaks aren't an issue (so you don't have to
> worry about cleaning up), and if for some reason something deep in
> your routine throws an exception and backs way out you won't end up
> with orphaned memory.
Ok. Er, forgive my stupidity, but how does it know when I'm done with
it? Here's what I'm doing:
Buffer* buf = new_buffer_header(interp);
Parrot_allocate(interp, buf, sizeof(UINTVAL) * (len + 1));
ffun = (UINTVAL*) buf->bufstart;
/* use ffun */
Is that right? (Sorry, I'm completely new at the whole Parrot thing)
Luke
Steve Fink
> > At 7:33 PM -0600 4/15/03, Luke Palmer wrote:
> > >While working on this patch (my first one! yay for me!) I noticed I
> > >needed a little dynamically allocated memory, just for the duration of
> > >the function. How would be the preferred way of allocating (and
> > >deallocating) it? Should I just use malloc(), or is there a Parrot
> > >alternative?
> >
> > The best thing to do is allocate a buffer and call into
> > Parrot_allocate to fill it. The allocation's pretty fast, there's no
> > free list to worry about, leaks aren't an issue (so you don't have to
> > worry about cleaning up), and if for some reason something deep in
> > your routine throws an exception and backs way out you won't end up
> > with orphaned memory.
>
> Ok. Er, forgive my stupidity, but how does it know when I'm done with
> it?
It's definitely not a stupid question. The answer is garbage
collection. It won't bother to notice that you're done with it until
it needs more memory, and then it's more a matter of not noticing that
you're still using it. Which leads to some gotchas to watch out for:
> Here's what I'm doing:
>
> Buffer* buf = new_buffer_header(interp);
> Parrot_allocate(interp, buf, sizeof(UINTVAL) * (len + 1));
> ffun = (UINTVAL*) buf->bufstart;
> /* use ffun */
>
> Is that right? (Sorry, I'm completely new at the whole Parrot thing)
That is right -- but already somewhat dangerous. I believe that
currently your code will work fine, since we are still walking the
system stack to find in-use memory (so if Parrot happens to decide it
needs more memory during your Parrot_allocate() call, it won't try to
get it by freeing up your 'buf' local var since it will be found on
the system stack.)
More future-proof would be to emulate stacks.c:
stack->buffer = new_buffer_header(interpreter);
/* Block DOD from murdering our newly allocated stack->buffer. */
Parrot_block_DOD(interpreter);
Parrot_allocate(interpreter, stack->buffer,
sizeof(Stack_Entry_t) * STACK_CHUNK_DEPTH);
Parrot_unblock_DOD(interpreter);
(DOD is Dead Object Detection.)
But in either case, be sure that you've anchored your 'buf' variable
to the root set somehow. If you're doing this in a PMC, that probably
means putting your Buffer* into the ->data field of the PMC and
setting the PObj_is_buffer_ptr_FLAG on the PMC.
Then the next thing you'll have to worry about is your ffun floating
away from you. If you do anything that might trigger a GC run, then
buf->bufstart may be moved somewhere else.
If you're wondering how you're supposed to psychically infer all of
this, then stop answering your own questions. Or read
docs/pdds/pdd09_gc.pod, which is sort of up to date. Or
docs/memory_internals.pod, which will tell you more than you wanted to
know but less than you need to know. Or if you're freaking out about
all the restrictions, then read docs/dev/infant.dev to hear other
people freaking out too. Unfortunately, we don't have a "here's
everything a Parrot internals hacker needs to know about memory
management" document, at least as far as I know. pdd09 says some of
it, but cursorily.
Oh, and welcome to the party! (At the moment, most of the partygoers
appear to be passed out on the lawn, but welcome anyway.)
Steve Fink
probably not going to be stuffing things into PMCs, huh?
I think some of what I said is still relevant, though...
Dan Sugalski
>On Apr-15, Luke Palmer wrote:
> > Here's what I'm doing:
>>
>> Buffer* buf = new_buffer_header(interp);
>> Parrot_allocate(interp, buf, sizeof(UINTVAL) * (len + 1));
>> ffun = (UINTVAL*) buf->bufstart;
>> /* use ffun */
>>
>> Is that right? (Sorry, I'm completely new at the whole Parrot thing)
>
>That is right -- but already somewhat dangerous. I believe that
>currently your code will work fine, since we are still walking the
>system stack to find in-use memory (so if Parrot happens to decide it
>needs more memory during your Parrot_allocate() call, it won't try to
>get it by freeing up your 'buf' local var since it will be found on
>the system stack.)
We're always going to be walking the system stack, so you should be
safe, as long as you don't hold on to the dereferenced pointer to the
buffer's contents, as the GC may move it around. (We ought to expose
a fixed memory API. On the list)
Sounds like we need to update the memory docs and get this clearly
codified, too.
Benjamin Goldberg
>
> At 8:09 PM -0700 4/15/03, Steve Fink wrote:
> >On Apr-15, Luke Palmer wrote:
> > > Here's what I'm doing:
> >>
> >> Buffer* buf = new_buffer_header(interp);
> >> Parrot_allocate(interp, buf, sizeof(UINTVAL) * (len + 1));
> >> ffun = (UINTVAL*) buf->bufstart;
> >> /* use ffun */
> >>
> >> Is that right? (Sorry, I'm completely new at the whole Parrot
> >> thing)
> >
> >That is right -- but already somewhat dangerous. I believe that
> >currently your code will work fine, since we are still walking the
> >system stack to find in-use memory (so if Parrot happens to decide it
> >needs more memory during your Parrot_allocate() call, it won't try to
> >get it by freeing up your 'buf' local var since it will be found on
> >the system stack.)
>
> We're always going to be walking the system stack,
Is that guaranteed for all future implementations?
I thought that scanning the system stack was unportable.
> so you should be safe, as long as you don't hold on to the dereferenced
> pointer to the buffer's contents, as the GC may move it around. (We
> ought to expose a fixed memory API. On the list)
So that 'ffun' pointer in Luke's example code might become invalid, due
to the contents of 'buf' (and buf->bufstart in particular) being moved
around. I think.
> Sounds like we need to update the memory docs and get this clearly
> codified, too.
--
$a=24;split//,240513;s/\B/ => /for@@=qw(ac ab bc ba cb ca
);{push(@b,$a),($a-=6)^=1 for 2..$a/6x--$|;print "$@[$a%6
]\n";((6<=($a-=6))?$a+=$_[$a%6]-$a%6:($a=pop @b))&&redo;}
Dan Sugalski
>Dan Sugalski wrote:
>>
>> At 8:09 PM -0700 4/15/03, Steve Fink wrote:
>> >On Apr-15, Luke Palmer wrote:
>> > > Here's what I'm doing:
>> >>
>> >> Buffer* buf = new_buffer_header(interp);
>> >> Parrot_allocate(interp, buf, sizeof(UINTVAL) * (len + 1));
>> >> ffun = (UINTVAL*) buf->bufstart;
>> >> /* use ffun */
>> >>
>> >> Is that right? (Sorry, I'm completely new at the whole Parrot
>> >> thing)
>> >
>> >That is right -- but already somewhat dangerous. I believe that
>> >currently your code will work fine, since we are still walking the
>> >system stack to find in-use memory (so if Parrot happens to decide it
>> >needs more memory during your Parrot_allocate() call, it won't try to
>> >get it by freeing up your 'buf' local var since it will be found on
>> >the system stack.)
>>
>> We're always going to be walking the system stack,
>
>Is that guaranteed for all future implementations?
Yes.
>I thought that scanning the system stack was unportable.
It is, in the general case.
> > so you should be safe, as long as you don't hold on to the dereferenced
>> pointer to the buffer's contents, as the GC may move it around. (We
>> ought to expose a fixed memory API. On the list)
>
>So that 'ffun' pointer in Luke's example code might become invalid, due
>to the contents of 'buf' (and buf->bufstart in particular) being moved
>around. I think.
Yep, that's a possibility.
Benjamin Goldberg
> Benjamin Goldberg wrote:
>> Dan Sugalski wrote:
>>>
>>> At 8:09 PM -0700 4/15/03, Steve Fink wrote:
>>>>On Apr-15, Luke Palmer wrote:
>>>>> Here's what I'm doing:
>>>>>
>>>>> Buffer* buf = new_buffer_header(interp);
>>>>> Parrot_allocate(interp, buf, sizeof(UINTVAL) * (len + 1));
>>>>> ffun = (UINTVAL*) buf->bufstart;
>>>>> /* use ffun */
>>>>>
>>>>> Is that right? (Sorry, I'm completely new at the whole Parrot
>>>>> thing)
>>>>
>>>> That is right -- but already somewhat dangerous. I believe that
>>>> currently your code will work fine, since we are still walking the
>>>> system stack to find in-use memory (so if Parrot happens to decide it
>>>> needs more memory during your Parrot_allocate() call, it won't try to
>>>> get it by freeing up your 'buf' local var since it will be found on
>>>> the system stack.)
>>>
>>> We're always going to be walking the system stack,
>>
>> Is that guaranteed for all future implementations?
>
> Yes.
>
>> I thought that scanning the system stack was unportable.
>
> It is, in the general case.
I think that there's some mutual exclusivity there -- if it's unportable
in the general case, then not only won't the same code work on all systems,
but there might be some systems where the system stack can't be walked *at
all*.
Thus, if parrot is implemented on such a system that the system stack isn't
walkable, then the system stack won't be walked, and the guarantee is violated.
Unless you're also saying that parrot is garunteed to never be implemented
on those systems where the system stack can't be walked.
Dan Sugalski
Well... there used to be. Systems of that sort are generally
long-dead--if a system has a C89 compiler and working threads we can
generally assume that it's not too weird.
>Thus, if parrot is implemented on such a system that the system stack isn't
>walkable, then the system stack won't be walked, and the guarantee
>is violated.
>
>Unless you're also saying that parrot is garunteed to never be implemented
>on those systems where the system stack can't be walked.
Yep, that's what I'm saying. Either we get access to the C auto
variable chain, or we can't run there.
Kurt Stephens
If the system stack can't be walked implicitly, it could be walked
explicitly by temporarily registering local variables addresses with the
GC's root set, the way that Emacs Lisp does it, for example. It means
more code, but for those that say it's un-portable inferring stack
frames from things like local variable addresses, it would work.
I cant think of any modern environments in wide-spread use that don't
have a contiguous, downward-growing, activation record stack that can't
be rooted by getting a local variable address.
There are Scheme GC implementations that explicity use the C stack
pointer to do allocations, via alloca(), instead of from a heap. In
this case, the GC *actually* collects garbage from within the C stack.
This is all done with a tiny amount of "non-portable" C code.
But then again: nothing is portable until it's ported. #ifdefs are a
fact of life. :)