SOLVED: slow perl string concats, malloc, and other performance issues
Raistlin Majere
FreeBSD once again reigns in speed.
I ran the following perl script on a linux system and on a freebsd
system both running identical hardware and perl 5.6.1 to find that linux
was able to execute the script in 9 seconds and freebsd was taking
almost 56 seconds.
#!/usr/bin/perl
$result ="";
$begin = time;
for ($i = 0; $i < 1000000;) {
$i++;
#comment the following line to get freebsd perl to run as fast as linux
perl
$result .= "$i\n";
}
$duration = time - $begin;
print "duration = $duration.\n";
I called in my genius friend and mentor Marc Frajola to assist me in
debugging this odd slowness and he discovered through profiling
(recompiling perl with the -pg option to the compiler AND to the linker)
where the majority of the clock cycles were going. We then ran the
script in the background and fired up a vmstat session to find the
system was thrashing.
procs memory page disks faults
cpu
r b w avm fre flt re pi po fr sr ad0 md0 in sy cs us
sy id
1 0 0 189240 62440 71 0 0 0 73 1 0 0 235 1788 76 1
1 98
1 0 0 189980 62072 11965 0 0 0 11868 0 0 0 1145 1968 262
82 18 0
1 0 0 190676 62332 12280 0 0 0 12342 0 0 0 1149 1845 241
82 18 0
1 0 0 191380 61400 12296 0 0 0 12060 0 2 0 1150 1816 241
79 21 0
1 0 0 189036 63744 12277 0 0 0 12860 0 0 0 1148 1949 260
81 19 0
1 0 0 191608 60792 12999 0 0 0 12258 0 0 0 1147 1820 247
78 22 0
The enormous amount of page faults led us to look at the memory
allocation parts of the perl program and we identified that malloc was
the culprit. Malloc wasn't able to keep up with the gigantic string
which kept growing and growing through the for loop.
We did a "make clean" in the perl source directory and did a new
./Configure, this time we told it to use its own malloc,
(-Dusemymalloc) ran "make; make test; make install;" and found the above
script which took 56 seconds now took 5 seconds. (4 seconds faster than
the linux system!)
The stock build of perl from the port tree does NOT implement this
important change. You have to build from source and tell the Configure
script to use its own malloc. The default when running ./Configure is
also NOT to use its own malloc, so you have to be looking for that
question in the mess of others. (or edit the config.sh file)
I -=highly=- recommend recompiling your version of perl today. I
believe you'll notice a significant performance increase.
Special Thanks to Marc for his wonderful help!
-- Raistlin Alexander Majere
---------------------------------------------------------------
E R A I S T S O F T W A R E
Raistlin Majere http://www.e-raist.com rais...@e-raist.com
* Knowledge for Hire *
PGPFP: 1ED8 AF84 9E90 3CFC E4A4 8E69 728D D193 00CF 70FF
---------------------------------------------------------------
To Unsubscribe: send mail to majo...@FreeBSD.org
with "unsubscribe freebsd-questions" in the body of the message