observations on arenas and fragmentation

[Eric Wong]

Hello, I'm using a hacky LD_PRELOAD malloc wrapper[1] on Perl
daemons and noticing some 4080 byte allocations made late in
process lifetime lingering forever(?).

I'm not an expert in Perl internals, but my reading of sv.c
doesn't reveal arenas getting freed outside of process teardown.

With long-lived processes, permanent allocations made well after
a process enters steady state tends to cause fragmentation.
(steady state being when it's in a main loop and expected to
mainly do short-lived allocations)

For C10K network servers, a sudden burst of traffic to certain
endpoints well after startup seems likely to trigger this
fragmentation.

Unfortunately, C stdlib malloc has no way to declare the
expected lifetime of an allocation.

Perl itself could probably use anonymous mmap for arenas on
platforms where it's supported. mmap would also allow using
page-sized arenas instead of the awkward 4080 size.

Unfortunately, mmap would be slower for short-lived processes.
Figuring out a way to release arenas during runtime could be
done, but would also add more complexity and might hurt
performance.

A hybrid approach that switches to mmap in long-lived processes
might work, but Perl would need a way to know when it's entered
a steady state of a long-lived processes.

A possible mitigations for long-lived Perl code:

# attempt to create many arenas at startup
# (where PREALLOC_NR is a really big number in env)
BEGIN {
if (my $nr = $ENV{PREALLOC_NR}) {
my @tmp = map { $_ } (0..$nr);
}
}

There's also a lot of other stuff (regexps, stratchpads, `state')
which can create late permanent allocations. I'm not sure what
to do about those, yet.

Maybe it's just easier to restart processes if it grows to a
certain size. *shrug*


[1] git clone https://80x24.org/mwrap-perl.git
PS: I used mwrap-perl to find an Encode leak (RT #139622)
some years back. *Sigh* I can't view rt.cpan.org anymore
due to JS: <https://rt.cpan.org/Ticket/Display.html?id=139622>