5.8.2 RC1

7 views
Skip to first unread message

Nicholas Clark

unread,
Oct 27, 2003, 6:49:28 PM10/27/03
to perl5-...@perl.org
They went to sea in a Sieve, they did,
In a Sieve they went to sea:
In spite of all their friends could say,
On a winter's morn, on a stormy day,
In a Sieve they went to sea!
And when the Sieve turned round and round,
And everyone cried, "You'll all be drowned!"
They cried aloud, "Our Sieve ain't big,
But we don't care a button, we don't care a fig!
In a Sieve we'll go to sea!"

Far and few, far and few,
Are the lands where the Jumblies live;
Their heads are green, and their hands are blue,
And they went to sea in a Sieve.

(The Jumblies - Edward Lear)

Well, here is our very own sea-going sieve. :-)
Please test it thoroughly and find all the holes *before* we set sail.
Currently it's only at

http://www.ccl4.org/~nick/P/perl-5.8.2-RC1.tar.bz2

(or s/bz2$/gz/ if you really want a 25% larger download.) Please don't
publicise that URL outside p5p - instead

http://www.cpan.org/authors/id/N/NW/NWCLARK/perl-5.8.2-RC1.tar.bz2

Once it's propagated round the CPAN mirrors I'll make an announcement
on use.perl

I'm particularly interested in

0: binary compatibility between 5.8.2 and modules compiled under 5.8.0

1: binary compatibility between 5.8.2 and modules compiled under 5.8.1

2: how scripts that hash large amounts of data behave

3: reassurance that it still build on Windows, VMS, OS/2, ...

4: whether OS X and Solaris build all XS modules correctly now

5: is modperl still happy

6: does PAR work smoothly

I suspect that I've forgotten something important from that list

rsync ftp.linux.activestate.com::perl-5.8.x/

will be at RC1 for the next 16 hours or so.

Nicholas Clark

Tim Bunce

unread,
Oct 28, 2003, 5:56:44 AM10/28/03
to Nicholas Clark, perl5-...@perl.org
On Mon, Oct 27, 2003 at 11:49:28PM +0000, Nicholas Clark wrote:
>
> 2: how scripts that hash large amounts of data behave

Idle curiosity: is there some way to dump the lengths of the lists
associated with each hash bucket?

Tim.

Gisle Aas

unread,
Oct 28, 2003, 6:03:52 AM10/28/03
to Tim Bunce, Nicholas Clark, perl5-...@perl.org
Tim Bunce <Tim....@pobox.com> writes:

Devel::Peek shows this information. The ARRAY line has this
information. For example:

$ perl -MDevel::Peek -e 'Dump(\%INC)' 2>&1 | grep ARRAY
ARRAY = 0x8180f00 (0:5, 1:3)

This shows that 5 buckets had 0 elements and 3 buckets had 1 element.

Regards,
Gisle

Alexey Tourbin

unread,
Oct 28, 2003, 6:21:39 AM10/28/03
to perl5-...@perl.org
On Mon, Oct 27, 2003 at 11:49:28PM +0000, Nicholas Clark wrote:
> I'm particularly interested in
>
> 0: binary compatibility between 5.8.2 and modules compiled under 5.8.0
>
> 1: binary compatibility between 5.8.2 and modules compiled under 5.8.1

Hi,

weird things discovered with perl@21501. Those are about
use/package/eval scope semantic changes.

With perl-5.8.1, I was perfectly able to build BerkeleyDB-0.24 module
from CPAN, including all tests pass. With perl@21501, some tests fail.
As I investigated the causes of the problem, I found out that "module
visibility" behavior changed.

Now I can build BerkeleyDB-0.24 only with the following patch that
fixes tests. I am going to investigate more and make a test case.
I hope this problem is resolved before 5.8.2 is released.


--- BerkeleyDB-0.24~/t/btree.t 2003-06-14 15:53:36 +0000
+++ BerkeleyDB-0.24/t/btree.t 2003-10-21 12:48:46 +0000
@@ -819,6 +819,7 @@ EOM
my %h ;
my $X ;
eval '
+ use BerkeleyDB;
$X = tie(%h, "SubDB", -Filename => "dbbtree.tmp",
-Flags => DB_CREATE,
-Mode => 0640 );
--- BerkeleyDB-0.24~/t/hash.t 2003-06-14 16:17:17 +0000
+++ BerkeleyDB-0.24/t/hash.t 2003-10-21 12:49:51 +0000
@@ -700,6 +700,7 @@ EOM
my %h ;
my $X ;
eval '
+ use BerkeleyDB;
$X = tie(%h, "SubDB", -Filename => "dbhash.tmp",
-Flags => DB_CREATE,
-Mode => 0640 );
--- BerkeleyDB-0.24~/t/queue.t 2003-09-27 08:45:10 +0000
+++ BerkeleyDB-0.24/t/queue.t 2003-10-28 10:49:25 +0000
@@ -662,6 +662,7 @@ EOM
my $X ;
my $rec_len = 34 ;
eval '
+ use BerkeleyDB;
$X = tie(@h, "SubDB", -Filename => "dbqueue.tmp",
-Flags => DB_CREATE,
-Mode => 0640 ,
--- BerkeleyDB-0.24~/t/recno.t 2002-06-04 17:17:22 +0000
+++ BerkeleyDB-0.24/t/recno.t 2003-10-28 10:50:01 +0000
@@ -613,6 +613,7 @@ EOM
my @h ;
my $X ;
eval '
+ use BerkeleyDB;
$X = tie(@h, "SubDB", -Filename => "dbrecno.tmp",
-Flags => DB_CREATE,
-Mode => 0640 );

Abigail

unread,
Oct 28, 2003, 8:08:53 AM10/28/03
to Nicholas Clark, perl5-...@perl.org
On Mon, Oct 27, 2003 at 11:49:28PM +0000, Nicholas Clark wrote:

Using Gisle's advice and Tim's idea, here's a program that populates
a hash with more than a million keys. It then finds the largests
bucket, and average non-empty bucket size. There doesn't seem to be
a significant difference in the bucket size distribution between
5.8.0, 5.8.1 and 5.8.2-RC1. However, 5.8.2-RC1 is quite a bit faster
than 5.8.0 and 5.8.1.

Abigail

#!/usr/bin/perl

use strict;
use warnings;
no warnings qw /syntax/;

my $opts = '-MDevel::Peek';
my $prog = 'my %hash; while (<>) {$hash {$_} ++} Dump \%hash';
my @args = qw !/home/abigail/Words/woorden.max
/home/abigail/Words/allwords.txt!;
my $time = '/usr/bin/time';
my $frmt = '%S %U';

my @vers = qw /5.8.0 5.8.1 5.8.2-RC1/;
my $runs = 10;

print "Version Time Bucket-length\n";
print " Avg Max\n";
for my $version (@vers) {
my $perl = "/opt/perl/$version/bin/perl";
for my $c (1 .. $runs) {
print STDERR "$version: $c\n";
my @info = `$time -f '$frmt' $perl $opts -e '$prog' @args 2>&1`;
my ($user, $sys) = split ' ', $info [-1];
my $cpu = $user + $sys;
my $info = (grep {/ARRAY/} @info) [0];
$info =~ s/^[^(]*[(]//;
$info =~ s/[)][^)]*$//;
my ($max_length, $total_buckets, $total_keys) = (0, 0, 0);
while ($info =~ /(\d+):(\d+)/g) {
my ($length, $buckets) = ($1, $2);
next unless $length;
$total_keys += $length * $buckets;
$total_buckets += $buckets;
$max_length = $length if $length > $max_length;
}
my $avg_length = $total_keys / $total_buckets;
printf "%-10s (%5.2f) %5.2f %3d\n" =>
$version, $cpu, $avg_length, $max_length;
last if $version eq '5.8.0';
}
}

__END__
Version Time Bucket-length
Avg Max
5.8.0 (17.44) 1.27 7
5.8.1 (17.67) 1.27 8
5.8.1 (17.57) 1.27 7
5.8.1 (17.45) 1.27 7
5.8.1 (17.54) 1.27 7
5.8.1 (17.46) 1.27 7
5.8.1 (17.50) 1.27 7
5.8.1 (17.45) 1.27 7
5.8.1 (17.45) 1.27 6
5.8.1 (17.67) 1.27 7
5.8.1 (17.44) 1.27 7
5.8.2-RC1 (13.47) 1.27 6
5.8.2-RC1 (13.53) 1.27 7
5.8.2-RC1 (13.59) 1.27 7
5.8.2-RC1 (13.64) 1.27 6
5.8.2-RC1 (13.68) 1.27 7
5.8.2-RC1 (13.54) 1.27 7
5.8.2-RC1 (13.69) 1.27 8
5.8.2-RC1 (13.61) 1.27 7
5.8.2-RC1 (13.58) 1.27 7
5.8.2-RC1 (13.45) 1.27 6

Rafael Garcia-Suarez

unread,
Oct 28, 2003, 9:12:21 AM10/28/03
to perl5-...@perl.org
Alexey Tourbin wrote:
>
> So here goes the test case. The difference between perl-5.8.1 and
> perl@21501 is the following:
>
> perl-5.8.1:
>
> $ perl testcase.pl
> $
>
> perl@21501:
>
> $ perl testcase.pl
> Bareword "DB_CREATE" not allowed while "strict subs" in use at (eval 2) line 2.
> $

Wild guess : try reverting patch #21452.

Fix bug [perl #24212] : improper error recovery in the
tokenizer after an unknown filetest operator.

Alexey Tourbin

unread,
Oct 28, 2003, 9:36:58 AM10/28/03
to perl5-...@perl.org
On Tue, Oct 28, 2003 at 03:12:21PM +0100, Rafael Garcia-Suarez wrote:
> Wild guess : try reverting patch #21452.
>
> Fix bug [perl #24212] : improper error recovery in the
> tokenizer after an unknown filetest operator.

Where can I get the patch?
Sorry for being rather novice to p5p stuff.

H.Merijn Brand

unread,
Oct 28, 2003, 9:37:35 AM10/28/03
to Nicholas Clark, Perl 5 Porters
On Tue 28 Oct 2003 00:49, Nicholas Clark <ni...@ccl4.org> wrote:
> Well, here is our very own sea-going sieve. :-)
> Please test it thoroughly and find all the holes *before* we set sail.
> Currently it's only at
>
> http://www.ccl4.org/~nick/P/perl-5.8.2-RC1.tar.bz2

I'm working on dor-5.8.2.diff, and remember someone said me to remember to
change to diff to include some new feature.

Anyone?

And what about the plans to take the diff into core?
I've got no problem with maintaining it on my CPAN home

--
H.Merijn Brand Amsterdam Perl Mongers (http://amsterdam.pm.org/)
using perl-5.6.1, 5.8.0, & 5.9.x, and 806 on HP-UX 10.20 & 11.00, 11i,
AIX 4.3, SuSE 8.2, and Win2k. http://www.cmve.net/~merijn/
http://archives.develooper.com/daily...@perl.org/ per...@perl.org
send smoke reports to: smokers...@perl.org, QA: http://qa.perl.org

Rafael Garcia-Suarez

unread,
Oct 28, 2003, 9:36:24 AM10/28/03
to Alexey Tourbin, perl5-...@perl.org, andreas...@anima.de
Alexey Tourbin wrote:
>
> On Tue, Oct 28, 2003 at 03:12:21PM +0100, Rafael Garcia-Suarez wrote:
> > Wild guess : try reverting patch #21452.
> >
> > Fix bug [perl #24212] : improper error recovery in the
> > tokenizer after an unknown filetest operator.
>
> Where can I get the patch?

from the APC :
http://public.activestate.com/gsar/APC/

but don't bother, I tried it and this regression was not caused by
this patch. Maybe something with Exporter ??

(Andreas, maybe it's time to try it with your binsearchaperl. I'm
trying to reduce the test case.)

Rafael Garcia-Suarez

unread,
Oct 28, 2003, 9:42:42 AM10/28/03
to H.Merijn Brand, ni...@ccl4.org, perl5-...@perl.org
H.Merijn Brand wrote:
> I'm working on dor-5.8.2.diff, and remember someone said me to remember to
> change to diff to include some new feature.

most probably :

[ 21406] By: ams on 2003/10/06 04:22:11
Log: Subject: Re: [perl #24076] "<> err EXPR" and warnings.
From: Rafael Garcia-Suarez <rgarci...@free.fr>
Date: Sun, 5 Oct 2003 21:27:54 +0200
Message-Id: <20031005212754.5ef...@free.fr>
(Applied with tweaks to op.c and a comment.)
Branch: perl
! op.c t/lib/warnings/op

Abigail

unread,
Oct 28, 2003, 9:56:30 AM10/28/03
to H.Merijn Brand, Nicholas Clark, Perl 5 Porters
On Tue, Oct 28, 2003 at 03:37:35PM +0100, H.Merijn Brand wrote:
> On Tue 28 Oct 2003 00:49, Nicholas Clark <ni...@ccl4.org> wrote:
> > Well, here is our very own sea-going sieve. :-)
> > Please test it thoroughly and find all the holes *before* we set sail.
> > Currently it's only at
> >
> > http://www.ccl4.org/~nick/P/perl-5.8.2-RC1.tar.bz2
>
> I'm working on dor-5.8.2.diff, and remember someone said me to remember to
> change to diff to include some new feature.
>
> Anyone?
>
> And what about the plans to take the diff into core?

It would make (my) life easier if said diff was distributed with the core.

Abigail

H.Merijn Brand

unread,
Oct 28, 2003, 10:00:16 AM10/28/03
to Rafael Garcia-Suarez, Perl 5 Porters
On Tue 28 Oct 2003 15:42, Rafael Garcia-Suarez <rgarci...@free.fr> wrote:
> H.Merijn Brand wrote:
> > I'm working on dor-5.8.2.diff, and remember someone said me to remember to
> > change to diff to include some new feature.
>
> most probably :

Yes. That's it. Integrated. Now testing ...
Expect a diff soon

> [ 21406] By: ams on 2003/10/06 04:22:11
> Log: Subject: Re: [perl #24076] "<> err EXPR" and warnings.
> From: Rafael Garcia-Suarez <rgarci...@free.fr>
> Date: Sun, 5 Oct 2003 21:27:54 +0200
> Message-Id: <20031005212754.5ef...@free.fr>
> (Applied with tweaks to op.c and a comment.)
> Branch: perl
> ! op.c t/lib/warnings/op

--

Arthur Bergman

unread,
Oct 28, 2003, 10:05:28 AM10/28/03
to Abigail, H.Merijn Brand, Nicholas Clark, Perl 5 Porters

On Tuesday, October 28, 2003, at 02:56 pm, Abigail wrote:

> It would make (my) life easier if said diff was distributed with the
> core.
>
>
>
> Abigail
>


Why not make it a compile time option instead of a diff?

Arthur

Rafael Garcia-Suarez

unread,
Oct 28, 2003, 10:01:54 AM10/28/03
to Arthur Bergman, abi...@abigail.nl, h.m....@hccnet.nl, ni...@ccl4.org, perl5-...@perl.org

Binary compatibilty. As this adds a new opcode, it requires
rerunning opcode.pl before compiling. ETOOHAIRY (but patches
welcome :)

Tim Bunce

unread,
Oct 28, 2003, 10:10:44 AM10/28/03
to Gisle Aas, Tim Bunce, Nicholas Clark, perl5-...@perl.org

Ah, cool. Thanks.

Tim.

H.Merijn Brand

unread,
Oct 28, 2003, 10:15:00 AM10/28/03
to Rafael Garcia-Suarez, Perl 5 Porters

Which the diff prevents, since it includes the changed that would have been
made by opcode.pl

> ETOOHAIRY (but patches welcome :)

We decided in Paris to do it as a seperate diff, because people who want
defined-or would probably know how to apply the diff. Including it in the
Configure phase (later cannot be done) is not worth the time and effort, since
5.9.x and 5.10.x will have dor anyway.

The only problem is probably that people will have to change^wcheck the diff
after every patch, which is no fun!

Abigail

unread,
Oct 28, 2003, 10:16:30 AM10/28/03
to Rafael Garcia-Suarez, Arthur Bergman, h.m....@hccnet.nl, ni...@ccl4.org, perl5-...@perl.org


Would a Configure flag that runs "patch -p0 < dor.diff" be
acceptable?

Abigail

H.Merijn Brand

unread,
Oct 28, 2003, 10:22:30 AM10/28/03
to Abigail, Perl 5 Porters

I would not like that. How about a message in the beginning of Configure that
states something like


# Configure -des
:
:
5.8.1 and on /can/ give you the new defined-or functionality
found in perl6 and perl5.10, but you will have to apply a
patch by hand and re-run Configure.

Note that applying this patch will make your perl binary incompatible
with non-dor perl binaries.

Do you want defined-or support? Yes

now run "patch -p1 <dor.diff" and then rerun Configure
#


> Abigail

H.Merijn Brand

unread,
Oct 28, 2003, 10:32:01 AM10/28/03
to Nicholas Clark, Perl 5 Porters
On Tue 28 Oct 2003 00:49, Nicholas Clark <ni...@ccl4.org> wrote:
> Well, here is our very own sea-going sieve. :-)
> Please test it thoroughly and find all the holes *before* we set sail.
> Currently it's only at
>
> http://www.ccl4.org/~nick/P/perl-5.8.2-RC1.tar.bz2

The uploaded file

dor-5.8.2-rc1.diff

has entered CPAN as

file: $CPAN/authors/id/H/HM/HMBRAND/dor-5.8.2-rc1.diff
size: 382134 bytes
md5: 57fc0f57c95f5276917f78d11972d9d2

Steve Hay

unread,
Oct 28, 2003, 10:56:56 AM10/28/03
to Nicholas Clark, perl5-...@perl.org
Nicholas Clark wrote:

>I'm particularly interested in
>

> 3: reassurance that it still build on Windows, VMS, OS/2, ...
>

> 5: is modperl still happy
>
>
>

Builds/tests OK on Windows XP with MSVC++, and both mod_perl 1.29 and
mod_perl 1.99_11-dev pass all their tests.

- Steve

Rafael Garcia-Suarez

unread,
Oct 28, 2003, 11:03:04 AM10/28/03
to Alexey Tourbin, perl5-...@perl.org
Alexey Tourbin wrote:
> weird things discovered with perl@21501. Those are about
> use/package/eval scope semantic changes.

Blame patch 21418.

Subject: [PATCH] Fixing UNIVERSAL.pm's bit of unpleasantness
From: sch...@pobox.com
Date: Mon, 6 Oct 2003 13:14:36 -0700
Message-Id: <2003100613...@ttul.org>

Here's a self-contained test case :
(the line 'use UNIVERSAL qw( isa );' triggers the failure)

BEGIN {
package U;
use Exporter;
@ISA = qw(Exporter);
sub FOO () { 42 };
@EXPORT = 'FOO';
use UNIVERSAL qw( isa );
package T;
use Exporter;
U->import();
@EXPORT = @U::EXPORT;
}
BEGIN { T->import(); }
use strict;
FOO;

Rafael Garcia-Suarez

unread,
Oct 28, 2003, 11:26:02 AM10/28/03
to perl5-...@perl.org
Rafael Garcia-Suarez wrote:
>
> Alexey Tourbin wrote:
> > weird things discovered with perl@21501. Those are about
> > use/package/eval scope semantic changes.
>
> Blame patch 21418.
>
> Subject: [PATCH] Fixing UNIVERSAL.pm's bit of unpleasantness
> From: sch...@pobox.com
> Date: Mon, 6 Oct 2003 13:14:36 -0700
> Message-Id: <2003100613...@ttul.org>

FWIW I think the bug is in BerkeleyDB, as it relies on an import()
method that is not here, except by an unfortunate side-effect of
UNIVERSAL.pm.

Here's the relevant excerpt of the summary at http://nntp.x.perl.org/group/perl.perl5.summary/76 :

When you load UNIVERSAL
Slaven Rezic reports that explicitely loading the UNIVERSAL module
breaks things, because UNIVERSAL.pm defines an import() method (aliased
to Exporter::import()), which is thus inherited by all classes whenever
UNIVERSAL.pm has been use'd. Michael G Schwern provides a patch,
ensuring that UNIVERSAL::import() does nothing when invoked on anything
else than the UNIVERSAL package.

http://xrl.us/wuw

That said, I'm not convinced that changing the guts of UNIVERSAL
is suitable for maint -- i.e. does it solve enough problems to justify
breaking compatibilty. Nicholas decides !

Nick Ing-Simmons

unread,
Oct 28, 2003, 12:31:40 PM10/28/03
to abi...@abigail.nl, perl5-...@perl.org, Nicholas Clark
Abigail <abi...@abigail.nl> writes:
>There doesn't seem to be
>a significant difference in the bucket size distribution between
>5.8.0, 5.8.1 and 5.8.2-RC1. However, 5.8.2-RC1 is quite a bit faster
>than 5.8.0 and 5.8.1.

Now that is the kind of backward (in)?compatibility I can live with :-)


Stas Bekman

unread,
Oct 28, 2003, 1:33:05 PM10/28/03
to Nicholas Clark, perl5-...@perl.org
Nick, what about that test that causes rehashing you promised to add? As I
mentioned before I can't tell how mod_perl 2.0 will be affected without trying
with this test (modified to create lots of subroutine entries).

__________________________________________________________________
Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide ---> http://perl.apache.org
mailto:st...@stason.org http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org http://ticketmaster.com

Nicholas Clark

unread,
Oct 28, 2003, 2:09:47 PM10/28/03
to Stas Bekman, Nicholas Clark, perl5-...@perl.org
On Tue, Oct 28, 2003 at 10:33:05AM -0800, Stas Bekman wrote:
> Nick, what about that test that causes rehashing you promised to add? As I
> mentioned before I can't tell how mod_perl 2.0 will be affected without
> trying with this test (modified to create lots of subroutine entries).

Yes, sorry. TODO. Will try tonight, else tomorrow night.
[currently I feel shattered]

Nicholas Clark

Nicholas Clark

unread,
Oct 28, 2003, 2:33:08 PM10/28/03
to Steve Hay, perl5-...@perl.org

modperl on XP? Oooh. Nice. Reassurance squared. Thanks for testing this.

Nicholas Clark

Stas Bekman

unread,
Oct 28, 2003, 3:03:57 PM10/28/03
to Nicholas Clark, Steve Hay, perl5-...@perl.org

Steve, it's not there yet. So far we had the hashing fixups for 5.8.1+. I need
to cut off that + and do it only for 5.8.1. I was waiting for the dust to
settle down with the plan C. Please watch the commit logs.

Michael G Schwern

unread,
Oct 28, 2003, 3:12:41 PM10/28/03
to Rafael Garcia-Suarez, perl5-...@perl.org, Alexey Tourbin, pm...@cpan.org
On Tue, Oct 28, 2003 at 05:26:02PM +0100, Rafael Garcia-Suarez wrote:
> FWIW I think the bug is in BerkeleyDB, as it relies on an import()
> method that is not here, except by an unfortunate side-effect of
> UNIVERSAL.pm.
<snip>

> That said, I'm not convinced that changing the guts of UNIVERSAL
> is suitable for maint -- i.e. does it solve enough problems to justify
> breaking compatibilty. Nicholas decides !

Fortunately its not a problem with BerkeleyDB itself but just the tests.
The snippet Alexey posted is from the btree.t sub-class test and it
inherits from BerkeleyDB::Btree but copies @BerkeleyDB::EXPORT and never
inherits from exporter.

Since this is just test code doing some slightly dodgy things, I think we're
ok fixing the UNIVERSAL::import() bug. It won't effect installed versions
of BerkeleyDB. All we need is a patch for a fixed version of BDB for new
installs. And wow, there's one attached to this very email! :)


--
Michael G Schwern sch...@pobox.com http://www.pobox.com/~schwern/
We don't know. But if we did, we wouldn't tell you.

BDB.patch

Nicholas Clark

unread,
Oct 28, 2003, 3:25:10 PM10/28/03
to Abigail, perl5-...@perl.org
On Tue, Oct 28, 2003 at 02:08:53PM +0100, Abigail wrote:
> Using Gisle's advice and Tim's idea, here's a program that populates
> a hash with more than a million keys. It then finds the largests
> bucket, and average non-empty bucket size. There doesn't seem to be
> a significant difference in the bucket size distribution between
> 5.8.0, 5.8.1 and 5.8.2-RC1. However, 5.8.2-RC1 is quite a bit faster
> than 5.8.0 and 5.8.1.

> 5.8.1 (17.67) 1.27 8


> 5.8.1 (17.57) 1.27 7
> 5.8.1 (17.45) 1.27 7
> 5.8.1 (17.54) 1.27 7
> 5.8.1 (17.46) 1.27 7
> 5.8.1 (17.50) 1.27 7
> 5.8.1 (17.45) 1.27 7
> 5.8.1 (17.45) 1.27 6
> 5.8.1 (17.67) 1.27 7
> 5.8.1 (17.44) 1.27 7
> 5.8.2-RC1 (13.47) 1.27 6
> 5.8.2-RC1 (13.53) 1.27 7
> 5.8.2-RC1 (13.59) 1.27 7
> 5.8.2-RC1 (13.64) 1.27 6
> 5.8.2-RC1 (13.68) 1.27 7
> 5.8.2-RC1 (13.54) 1.27 7
> 5.8.2-RC1 (13.69) 1.27 8
> 5.8.2-RC1 (13.61) 1.27 7
> 5.8.2-RC1 (13.58) 1.27 7
> 5.8.2-RC1 (13.45) 1.27 6

I didn't have a million words, and could really replicate your nice
clear timings.

I hope that this speedup is real, and if so I suspect that the cause
is due to a change in when hashes split. Firstly I duplicated
Tye McQueen's bug fix (19423) into hv_store_ent, so that hashes split
somewhat earlier.

I then changed the split trigger so that a split also happens if any of
the linked lists gets too long. My hunch is think that this latter
change is the cause of any speedup, given that the end result is hashes
with similar distributions to 5.8.1. I'd hope that earlier splits are
causing less linear scans of the hashes during data insertion.

Could you try changing "too long" from 4 to 3, and see if that changes
the speed of RC1?

Nicholas Clark

--- hv.c.orig 2003-10-26 21:10:13.000000000 +0000
+++ hv.c 2003-10-28 19:52:09.000000000 +0000
@@ -20,7 +20,7 @@
#define PERL_IN_HV_C
#include "perl.h"

-#define HV_MAX_LENGTH_BEFORE_SPLIT 4
+#define HV_MAX_LENGTH_BEFORE_SPLIT 3

STATIC HE*
S_new_he(pTHX)

Abigail

unread,
Oct 28, 2003, 4:48:51 PM10/28/03
to Nicholas Clark, perl5-...@perl.org
On Tue, Oct 28, 2003 at 08:25:10PM +0000, Nicholas Clark wrote:
>
> Could you try changing "too long" from 4 to 3, and see if that changes
> the speed of RC1?


It doesn't seem to make much of a difference:

Version Time Bucket-length
Avg Max

5.8.0 (17.41) 1.27 7
5.8.1 (18.05) 1.27 7
5.8.1 (17.61) 1.27 7
5.8.1 (17.42) 1.27 7
5.8.1 (17.83) 1.27 7
5.8.1 (17.58) 1.27 7
5.8.1 (17.63) 1.27 7
5.8.1 (17.84) 1.27 7
5.8.1 (17.54) 1.27 6
5.8.1 (17.59) 1.27 7
5.8.1 (18.67) 1.27 7
5.8.2-RC1 (13.69) 1.27 7
5.8.2-RC1 (13.70) 1.27 7
5.8.2-RC1 (13.78) 1.27 7
5.8.2-RC1 (13.66) 1.27 7
5.8.2-RC1 (13.81) 1.27 7
5.8.2-RC1 (13.72) 1.27 8
5.8.2-RC1 (13.87) 1.27 6
5.8.2-RC1 (13.64) 1.27 7
5.8.2-RC1 (13.70) 1.27 7
5.8.2-RC1 (13.83) 1.27 7
5.8.2-RC1.nick (13.81) 1.27 7
5.8.2-RC1.nick (13.86) 1.27 7
5.8.2-RC1.nick (13.68) 1.27 7
5.8.2-RC1.nick (13.64) 1.27 7
5.8.2-RC1.nick (13.52) 1.27 7
5.8.2-RC1.nick (13.56) 1.27 7
5.8.2-RC1.nick (13.58) 1.27 7
5.8.2-RC1.nick (13.54) 1.27 7
5.8.2-RC1.nick (13.64) 1.27 7
5.8.2-RC1.nick (13.61) 1.27 8


5.8.2-RC1.nick is with HV_MAX_LENGTH_BEFORE_SPLIT set to 3.

Abigail

Paul Marquess

unread,
Oct 28, 2003, 5:34:29 PM10/28/03
to Michael G Schwern, Rafael Garcia-Suarez, perl5-...@perl.org, Alexey Tourbin

> From: Michael G Schwern [mailto:sch...@pobox.com]


> On Tue, Oct 28, 2003 at 05:26:02PM +0100, Rafael Garcia-Suarez wrote:
> > FWIW I think the bug is in BerkeleyDB, as it relies on an import()
> > method that is not here, except by an unfortunate side-effect of
> > UNIVERSAL.pm.
> <snip>
> > That said, I'm not convinced that changing the guts of UNIVERSAL
> > is suitable for maint -- i.e. does it solve enough problems to justify
> > breaking compatibilty. Nicholas decides !
>
> Fortunately its not a problem with BerkeleyDB itself but just the tests.
> The snippet Alexey posted is from the btree.t sub-class test and it
> inherits from BerkeleyDB::Btree but copies @BerkeleyDB::EXPORT and never
> inherits from exporter.
>
> Since this is just test code doing some slightly dodgy things, I
> think we're
> ok fixing the UNIVERSAL::import() bug. It won't effect installed versions
> of BerkeleyDB. All we need is a patch for a fixed version of BDB for new
> installs. And wow, there's one attached to this very email! :)

Thanks for the patch Michael.

I'm a bit confused though - there are an identical set of tests in DB_File
(BerkeleyDB inherited a lot of tests from DB_File, including these). If this
is in 5.8.2-tobe, why hasn't DB_File fallen over?

Paul

Rafael Garcia-Suarez

unread,
Oct 28, 2003, 5:43:31 PM10/28/03
to Paul.M...@btinternet.com, Michael G Schwern, perl5-...@perl.org, Alexey Tourbin
Paul Marquess wrote:
>
> I'm a bit confused though - there are an identical set of tests in DB_File
> (BerkeleyDB inherited a lot of tests from DB_File, including these). If this
> is in 5.8.2-tobe, why hasn't DB_File fallen over?

The bug is due to the presence in various places in BerkeleyDB.pm of
"use UNIVERSAL qw( isa );". This doesn't appear in DB_File.

Michael G Schwern

unread,
Oct 28, 2003, 5:43:30 PM10/28/03
to Paul Marquess, Rafael Garcia-Suarez, perl5-...@perl.org, Alexey Tourbin
On Tue, Oct 28, 2003 at 10:34:29PM -0000, Paul Marquess wrote:
> I'm a bit confused though - there are an identical set of tests in DB_File
> (BerkeleyDB inherited a lot of tests from DB_File, including these). If this
> is in 5.8.2-tobe, why hasn't DB_File fallen over?

In the DB_File case, SubDB inherits from DB_File which inherits from Exporter.
It then copies @DB_File::Exporter. This is correct.

In the BerkelyDB case, SubDB inherits from BerkeleyDB::Hash (or Btree or
Queue or Recno) which does not inherit from Exporter. It then copies
@BerkeleyDB::Exporter. This is incorrect. It was only working because
SubDB incidentally inherited import() from UNIVERSAL.

Either SubDB should inherit from both BerkeleyDB and BerkeleyDB::Salad or
the BerkeleyDB::Salad should inherit from BerkeleyDB or SubDB should inherit
from both BerkeleyDB and Exporter. I did the latter, since it had the
smallest impact on your code, but I think since SubDB is exporting
BerkeleyDB functions it should do one of the two former ones. #2 is
probably most correct if you expect folks inheriting from BerkeleyDB::Salad
to be able to export BerkeleyDB functions.

You're the sickest teenager I've ever set my wallet on.

Paul Marquess

unread,
Oct 28, 2003, 6:11:20 PM10/28/03
to Michael G Schwern, Rafael Garcia-Suarez, perl5-...@perl.org, Alexey Tourbin
From: Michael G Schwern [mailto:sch...@pobox.com]

>


Thanks for the explanation. I'll sort it out & get a new BerkeleyDB
uploaded.

Paul

Leon Brocard

unread,
Oct 31, 2003, 9:04:22 AM10/31/03
to Nicholas Clark, perl5-...@perl.org, er...@zeegee.com
Leon Brocard sent the following bits through the ether:

> I'm currently experiencing problems with 5.8.2 RC1 and MIME-tools
> where it appears to go into an infinite loop (RH 7.3 on dual Xeon and
> Gentoo on Celeron). Investigating.

Oh, it looks like it's been fixed sinced RC1 as it works fine under a
freshly-rsynced maint. False alarm: call off the ninjas.

Leon
--
Leon Brocard.............................http://www.astray.com/
scribot.................................http://www.scribot.com/

... Dinner Not Ready... (A)bort (R)etry (P)izza

Paul Marquess

unread,
Nov 1, 2003, 6:15:46 AM11/1/03
to Michael G Schwern, Rafael Garcia-Suarez, perl5-...@perl.org, Alexey Tourbin

From: Paul Marquess [mailto:Paul.M...@btinternet.com]

Just built a snapshot of 5.8.2 (@21558) to see if I can reproduce the bug,
but it refuses to fail. Has something changed since this bug was reported?

Paul

Paul Marquess

unread,
Nov 1, 2003, 6:37:07 AM11/1/03
to Michael G Schwern, Rafael Garcia-Suarez, perl5-...@perl.org, Alexey Tourbin

but I can reporoduce it with 5.9.0 (215538).

Paul

Nicholas Clark

unread,
Nov 1, 2003, 6:40:00 AM11/1/03
to Paul Marquess, Michael G Schwern, Rafael Garcia-Suarez, perl5-...@perl.org, Alexey Tourbin
On Sat, Nov 01, 2003 at 11:37:07AM -0000, Paul Marquess wrote:
> From: Paul Marquess [mailto:Paul.M...@btinternet.com]

> > Just built a snapshot of 5.8.2 (@21558) to see if I can reproduce the bug,


> > but it refuses to fail. Has something changed since this bug was reported?
>
> but I can reporoduce it with 5.9.0 (215538).


I backed it out of maint. I'd said I was going to do this (I think on
another thread) but offhand I can't see if I actually mailed p5p to say
that I had done it.

The change seems to risky right now.

Nicholas Clark

Paul Marquess

unread,
Nov 1, 2003, 6:51:30 AM11/1/03
to Nicholas Clark, Michael G Schwern, Rafael Garcia-Suarez, perl5-...@perl.org, Alexey Tourbin

From: Nicholas Clark

OK.

At least I was able to reproduce the problem with 5.9 and check that the fix
worked.

cheers
Paul

Reply all
Reply to author
Forward
0 new messages