[perl #42320] [BUG] Memory leak with String pmc

2 views
Skip to first unread message

Mehmet Yavuz Selim Soyturk

unread,
Apr 5, 2007, 7:56:26 PM4/5/07
to bugs-bi...@rt.perl.org
# New Ticket Created by "Mehmet Yavuz Selim Soyturk"
# Please include the string: [perl #42320]
# in the subject line of all future correspondence about this issue.
# <URL: http://rt.perl.org/rt3/Ticket/Display.html?id=42320 >


The next program causes a memory leak for me.

.sub main :main
loop:
$P0 = new .String
goto loop
.end


Interestingly, no memory leak with:

.sub main :main
loop:
$S0 = "foo"
$P0 = new .String
goto loop
.end


Summary of my parrot 0.4.10 (r17974) configuration:
configdate='Thu Apr 5 00:16:26 2007 GMT'
Platform:
osname=linux, archname=i486-linux-gnu-thread-multi
jitcapable=1, jitarchname=i386-linux,
jitosname=LINUX, jitcpuarch=i386
execcapable=1
perl=/usr/bin/perl
Compiler:
cc='cc', ccflags='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -DDEBIAN -
pipe -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
Linker and Libraries:
ld='cc', ldflags=' -L/usr/local/lib',
cc_ldflags='',
libs='-ldl -lm -lpthread -lcrypt -lrt -lgmp -lreadline -lncurses'
Dynamic Linking:
share_ext='.so', ld_share_flags='-shared -L/usr/local/lib -fPIC',
load_ext='.so', ld_load_flags='-shared -L/usr/local/lib -fPIC'
Types:
iv=long, intvalsize=4, intsize=4, opcode_t=long, opcode_t_size=4,
ptrsize=4, ptr_alignment=1 byteorder=1234,
nv=double, numvalsize=8, doublesize=8
--
Mehmet

Chromatic

unread,
Apr 23, 2007, 2:23:15 PM4/23/07
to perl6-i...@perl.org, Mehmet Yavuz Selim Soyturk, bugs-bi...@rt.perl.org
On Thursday 05 April 2007 16:56, Mehmet Yavuz Selim Soyturk wrote:

> The next program causes a memory leak for me.
>
> .sub main :main
> loop:
> $P0 = new .String
> goto loop
> .end
>
>
> Interestingly, no memory leak with:
>
> .sub main :main
> loop:
> $S0 = "foo"
> $P0 = new .String
> goto loop
> .end

I can't explain that, but here's some suspicious code in
Parrot_allocate_string() in src/resources.c:

new_size = aligned_string_size(str, size);
mem = (char *)mem_allocate(interp, new_size, pool);
mem += sizeof (void*);

PObj_bufstart(str) = str->strstart = mem;
PObj_buflen(str) = new_size - sizeof (void*);

If I identify and read the related freeing code correctly (Parrot_dod_sweep()
in src/gc/dod.c):

else if (PObj_sysmem_TEST(b) && PObj_bufstart(b)) {
/* has sysmem allocated, e.g. string_pin */
mem_sys_free(PObj_bufstart(b));
PObj_bufstart(b) = NULL;
PObj_buflen(b) = 0;
}

... then there's a leak the sizeof (void *).

I don't guarantee that I've identified the appropriate code clearly though;
digging through this is tricky.

Does this sound familiar or interesting or fun to anyone else?

-- c

Chromatic

unread,
Apr 23, 2007, 2:52:26 PM4/23/07
to jerry gay, perl6-i...@perl.org, via RT Mehmet Yavuz Selim Soyturk, bugs-bi...@rt.perl.org
On Monday 23 April 2007 11:42, jerry gay wrote:

> > I don't guarantee that I've identified the appropriate code clearly
> > though; digging through this is tricky.

> > Does this sound familiar or interesting or fun to anyone else?

> sounds to me like it could be a reason for the pge garbage collection
> problems that andy and i have experienced.

That's exactly what I was thinking. I know I fixed some leaks the other day
which might help somewhat, but if we're not freeing Strings properly, there's
a problem.

Another possibility is that these PMCs get marked inappropriately when they're
no longer reachable from P-regs. I find that more difficult to believe, but
it's possible.

-- c

Jerry Gay

unread,
Apr 23, 2007, 2:42:49 PM4/23/07
to chromatic, perl6-i...@perl.org, via RT Mehmet Yavuz Selim Soyturk, bugs-bi...@rt.perl.org
sounds to me like it could be a reason for the pge garbage collection
problems that andy and i have experienced.
~jerry

Leopold Toetsch

unread,
Apr 23, 2007, 3:04:51 PM4/23/07
to perl6-i...@perl.org, chromatic
Am Montag, 23. April 2007 20:23 schrieb chromatic:
> On Thursday 05 April 2007 16:56, Mehmet Yavuz Selim Soyturk wrote:
> > The next program causes a memory leak for me.
> >
> > .sub main :main
> > loop:
> > $P0 = new .String
> > goto loop
> > .end

That's an endless loop. How does one measure, if it's leaking memory?

Anway:

> I can't explain that, but here's some suspicious code in
> Parrot_allocate_string() in src/resources.c:

[ ... ]

> If I identify and read the related freeing code correctly
> (Parrot_dod_sweep() in src/gc/dod.c):
>
> else if (PObj_sysmem_TEST(b) && PObj_bufstart(b)) {

These 2 snippets aren't related at all. The latter is dealing with freeing
sys-malloced memory as the comment is saying. The former is plain string
memory allocation code.

I'd presume, that the string memory compaction code i.e. 'compact_pool()' is
never run in the loop above.

leo

Chromatic

unread,
Apr 23, 2007, 7:59:30 PM4/23/07
to ma...@diephouse.com, Leopold Toetsch, perl6-i...@perl.org
On Monday 23 April 2007 16:41, Matt Diephouse wrote:

> Leopold Toetsch <l...@toetsch.at> wrote:
> > Am Montag, 23. April 2007 20:23 schrieb chromatic:

> > > > .sub main :main
> > > > loop:
> > > > $P0 = new .String
> > > > goto loop
> > > > .end
> >
> > That's an endless loop. How does one measure, if it's leaking memory?

> Presumably, every iteration of the loop uses the same "physical"
> register. This should free up the String from the previous iteration
> for collection. If there's a leak, memory would climb higher and
> higher; if there's not, it should level out.

I also threw in my favorite:

$P1 = getinterp
...
$P1.'run_gc'()

top still showed ever-increasing virtual set sizes.

I'll trace what compact_pool() does to see if that sheds any light.

-- c

Matt Diephouse

unread,
Apr 23, 2007, 7:41:26 PM4/23/07
to Leopold Toetsch, perl6-i...@perl.org, chromatic
Leopold Toetsch <l...@toetsch.at> wrote:
> Am Montag, 23. April 2007 20:23 schrieb chromatic:
> > On Thursday 05 April 2007 16:56, Mehmet Yavuz Selim Soyturk wrote:
> > > The next program causes a memory leak for me.
> > >
> > > .sub main :main
> > > loop:
> > > $P0 = new .String
> > > goto loop
> > > .end
>
> That's an endless loop. How does one measure, if it's leaking memory?

Presumably, every iteration of the loop uses the same "physical"


register. This should free up the String from the previous iteration
for collection. If there's a leak, memory would climb higher and
higher; if there's not, it should level out.

--
Matt Diephouse
http://matt.diephouse.com

Chromatic

unread,
Apr 24, 2007, 1:45:39 AM4/24/07
to perl6-i...@perl.org, ma...@diephouse.com, Leopold Toetsch
.sub main :main
loop:
$P0 = new .String
goto loop
.end

This one's fun.

One punchline's in src/gc/resources.c:153, within mem_allocate().

If it looks like there's no reclaimable memory within the allocated arena
pools, there's no sense in compacting them to try to get enough memory to
fulfill the current request.

I'm not sure why that is.

Of course, this test always allocates a new string (of size zero) from
Parrot_allocate_string() in src/gc/resources.c:738. For some reason, this
nibbles away at the buffers but prevents any compaction.

This code looks suspicious to me:

new_size = aligned_string_size(str, size);
mem = (char *)mem_allocate(interp, new_size, pool);
mem += sizeof (void*);

PObj_bufstart(str) = str->strstart = mem;
PObj_buflen(str) = new_size - sizeof (void*);

If size is 0 and new_size gets aligned to 4 and (void *) is 4 bytes long, then
the buflen will be 0. That may not be good at all, especially looking at
compact_pool() in src/gc/resources.c:334. With a buflen of 0, the buffer's
not copyable.

My conjecture is that if you fill up a memory pool with buffers that have the
4-byte overhead but no actual buffer lengths, you get pools that appear
uncompactable, and you have to allocate more and more new pools.

Here's my solution; don't allocate zero-sized buffers. Let them be empty.

All tests pass. This either means that this is the right solution or that we
don't have enough tests of the String PMC.

--c

fix_string_leak.patch

Leopold Toetsch

unread,
Apr 24, 2007, 2:36:03 PM4/24/07
to chromatic, perl6-i...@perl.org, ma...@diephouse.com
Am Dienstag, 24. April 2007 07:45 schrieb chromatic:
> Here's my solution; don't allocate zero-sized buffers.  Let them be empty.

Great catch. Thanks.
Indeed - zero-size allocs should just be ignored or/and even the source of
such requests be weeded out.

leo

Reply all
Reply to author
Forward
0 new messages