How to have perl allocate pointers predictably.

[Shlomi Fish]

Hi all,

with the included script below, Devel::Leak keeps outputting different pointers each time for
the leaked SVs. This makes debugging harder. Can I have perl somehow allocate the same pointers
each time when an allocation is being requested? I don't mind rebuilding a dedicated perl for that
using perlbrew or whatever. Note that this also happens if make_trouble() is as simple as
"my $x; $x = \$x;".

Regards,

-- Shlomi Fish

# Feeding XML::LibXML with an invalid file, triggering memory leaks

use strict;
use warnings;

use Devel::Leak;

use Encode;
use XML::LibXML;

use Carp;

binmode STDOUT, ':encoding(UTF-8)';

check_libxml_memory();

sub check_libxml_memory {
print "running\n";
my $handle;
my $leaveCount = 0;
my $enterCount = Devel::Leak::NoteSV($handle);
print STDERR "\nENTER: $enterCount SVs\n";
{
make_trouble(); # Trace how loading a bad doc affects memory
}
$leaveCount = Devel::Leak::CheckSV($handle);
print STDERR "\nLEAVE: $leaveCount SVs\n";
}

sub make_trouble {
# Tries to load a bad XML file into XML::LibXML
my $filenameIn = 'libxml-trouble-sample.html';
local $/; #Read whole file
open(my $FILEIN,'<', $filenameIn)
or die "Can't read file '$filenameIn' [$!]\n";
my $str = <$FILEIN>;
close ($FILEIN);

# Feeds the bad file to XML::LibXML.
my $parser = XML::LibXML->new;
my $success=1;
# if recover is set to 0 or 1, the problem ceases to exist
my $doc = $parser->parse_html_string($str,
{recover => 2, encoding => 'UTF-8'}
);

return;
}

1;

--
-----------------------------------------------------------------
Shlomi Fish http://www.shlomifish.org/
Funny Anti-Terrorism Story - http://shlom.in/enemy

What is is. Perceive It. Integrate it. Act on it. Idealize it.
— Leonard Peikoff

Please reply to list if it's a mailing list post - http://shlom.in/reply .

[Elizabeth Mattijsen]

I can't help but think this may be caused by hash randomization. Have you tried running this with PERL_HASH_SEED=0 ?

See https://metacpan.org/module/perlrun#PERL_HASH_SEED .


Liz
============

[bulk 88]

----------------------------------------
> Date: Thu, 23 Aug 2012 20:33:53 +0300
> From: shl...@shlomifish.org
> To: per...@perl.org
> Subject: How to have perl allocate pointers predictably.

>
> Hi all,
>
> with the included script below, Devel::Leak keeps outputting different pointers each time for
> the leaked SVs. This makes debugging harder. Can I have perl somehow allocate the same pointers
> each time when an allocation is being requested? I don't mind rebuilding a dedicated perl for that
> using perlbrew or whatever. Note that this also happens if make_trouble() is as simple as
> "my $x; $x = \$x;".

On Windows XP I always get the same SV *s assuming all inputs to the process are the same on each run.
____________________________________________________
C:\Documents and Settings\Owner\Desktop>perl -e "print \\\"perl\""
SCALAR(0x8fe04c)
C:\Documents and Settings\Owner\Desktop>perl -e "print \\\"perl\""
SCALAR(0x8fe04c)
C:\Documents and Settings\Owner\Desktop>perl -e "print \\\"perl\""
SCALAR(0x8fe04c)
C:\Documents and Settings\Owner\Desktop>perl -e "print \\\"perl\""
SCALAR(0x8fe04c)
C:\Documents and Settings\Owner\Desktop>perl -e "print \\\"perl\""
SCALAR(0x8fe04c)
C:\Documents and Settings\Owner\Desktop>perl -e "print \\\"perl\""
SCALAR(0x8fe04c)
C:\Documents and Settings\Owner\Desktop>
____________________________________________________
C:\Documents and Settings\Owner\Desktop>perl -e "print unpack('J', pack('p', \"p
erl\"))"
9468532
C:\Documents and Settings\Owner\Desktop>perl -e "print unpack('J', pack('p', \"p
erl\"))"
9468532
C:\Documents and Settings\Owner\Desktop>perl -e "print unpack('J', pack('p', \"p
erl\"))"
9468532
C:\Documents and Settings\Owner\Desktop>perl -e "print unpack('J', pack('p', \"p
erl\"))"
9468532
C:\Documents and Settings\Owner\Desktop>perl -e "print unpack('J', pack('p', \"p
erl\"))"
9468532
C:\Documents and Settings\Owner\Desktop>
____________________________________________________
Hash number randomization is irrelevant since the width of the hash array is determined by the number of keys/HEKs usually, not the hash number. So the memory allocation sizes and order of them to malloc are the same. Its it quite popular in "modern" OSes to randomize heaps/dlls/stacks/etc per boot or per process nowadays. That is probably what is happening in your case.

[Shlomi Fish]

Hi Liz,

On Thu, 23 Aug 2012 20:54:13 +0200
Elizabeth Mattijsen <l...@dijkmat.nl> wrote:

> I can't help but think this may be caused by hash randomization.
> Have you tried running this with PERL_HASH_SEED=0 ?
>
> See https://metacpan.org/module/perlrun#PERL_HASH_SEED .
>

Here goes:

<<<<<<<<<<<<<
shlomif[XML-LibXML]:$trunk$ cat TEST.pl

# Feeding XML::LibXML with an invalid file, triggering memory leaks

use strict;
use warnings;

use Devel::Leak;

binmode STDOUT, ':encoding(UTF-8)';

check_libxml_memory();

sub check_libxml_memory {
print "running\n";
my $handle;
my $leaveCount = 0;
my $enterCount = Devel::Leak::NoteSV($handle);
print STDERR "\nENTER: $enterCount SVs\n";
{
make_trouble(); # Trace how loading a bad doc affects memory
}
$leaveCount = Devel::Leak::CheckSV($handle);
print STDERR "\nLEAVE: $leaveCount SVs\n";
}

sub make_trouble {

my $x;

$x = \$x;

return;
}

1;
shlomif[XML-LibXML]:$trunk$ PERL_HASH_SEED=0 perl TEST.pl
running

ENTER: 9318 SVs
new 0x13a14e8 :

LEAVE: 9319 SVs
shlomif[XML-LibXML]:$trunk$ PERL_HASH_SEED=0 perl TEST.pl
running

ENTER: 9318 SVs
new 0xf074e8 :

LEAVE: 9319 SVs
shlomif[XML-LibXML]:$trunk$ which perl
/bin/perl
shlomif[XML-LibXML]:$trunk$
>>>>>>>>>>>>>

So it did not appear to have helped.

Regards,

Shlomi Fish

Why I Love Perl - http://shlom.in/joy-of-perl

Xena the warrior princess can meet King David for breakfast and Julius Caesar
for lunch. Without time travel.

[Shlomi Fish]

Hi bulk 88,

thanks for your message. See below for my response.

Thanks for the lead. This seems to have been the case. After searching in
vein for "perl heap randomization" and stuff like that, I searched for
"linux heap randomization" and got to this page:

http://stackoverflow.com/questions/1455904/how-to-disable-address-space-randomization-for-a-binary-on-linux

This referred me to the setarch -R command and doing this:

$ ( PERL_HASH_SEED=0 setarch x86_64 -R perl libxml2.pl ) 2>&1

now gives me consistent results on each run. It may be enabled by default in gdb, though.

Regards,

Shlomi Fish

--
-----------------------------------------------------------------
Shlomi Fish http://www.shlomifish.org/

Free (Creative Commons) Music Downloads, Reviews and more - http://jamendo.com/

No self‐respecting tomboy would use Mandriva.