Far and few, far and few,
Are the lands where the Jumblies live;
Their heads are green, and their hands are blue,
And they went to sea in a Sieve.
(The Jumblies - Edward Lear)
Well, here is our very own sea-going sieve. :-)
Please test it thoroughly and find all the holes *before* we set sail.
Currently it's only at
http://www.ccl4.org/~nick/P/perl-5.8.2-RC1.tar.bz2
(or s/bz2$/gz/ if you really want a 25% larger download.) Please don't
publicise that URL outside p5p - instead
http://www.cpan.org/authors/id/N/NW/NWCLARK/perl-5.8.2-RC1.tar.bz2
Once it's propagated round the CPAN mirrors I'll make an announcement
on use.perl
I'm particularly interested in
0: binary compatibility between 5.8.2 and modules compiled under 5.8.0
1: binary compatibility between 5.8.2 and modules compiled under 5.8.1
2: how scripts that hash large amounts of data behave
3: reassurance that it still build on Windows, VMS, OS/2, ...
4: whether OS X and Solaris build all XS modules correctly now
5: is modperl still happy
6: does PAR work smoothly
I suspect that I've forgotten something important from that list
rsync ftp.linux.activestate.com::perl-5.8.x/
will be at RC1 for the next 16 hours or so.
Nicholas Clark
Idle curiosity: is there some way to dump the lengths of the lists
associated with each hash bucket?
Tim.
Devel::Peek shows this information. The ARRAY line has this
information. For example:
$ perl -MDevel::Peek -e 'Dump(\%INC)' 2>&1 | grep ARRAY
ARRAY = 0x8180f00 (0:5, 1:3)
This shows that 5 buckets had 0 elements and 3 buckets had 1 element.
Regards,
Gisle
Hi,
weird things discovered with perl@21501. Those are about
use/package/eval scope semantic changes.
With perl-5.8.1, I was perfectly able to build BerkeleyDB-0.24 module
from CPAN, including all tests pass. With perl@21501, some tests fail.
As I investigated the causes of the problem, I found out that "module
visibility" behavior changed.
Now I can build BerkeleyDB-0.24 only with the following patch that
fixes tests. I am going to investigate more and make a test case.
I hope this problem is resolved before 5.8.2 is released.
--- BerkeleyDB-0.24~/t/btree.t 2003-06-14 15:53:36 +0000
+++ BerkeleyDB-0.24/t/btree.t 2003-10-21 12:48:46 +0000
@@ -819,6 +819,7 @@ EOM
my %h ;
my $X ;
eval '
+ use BerkeleyDB;
$X = tie(%h, "SubDB", -Filename => "dbbtree.tmp",
-Flags => DB_CREATE,
-Mode => 0640 );
--- BerkeleyDB-0.24~/t/hash.t 2003-06-14 16:17:17 +0000
+++ BerkeleyDB-0.24/t/hash.t 2003-10-21 12:49:51 +0000
@@ -700,6 +700,7 @@ EOM
my %h ;
my $X ;
eval '
+ use BerkeleyDB;
$X = tie(%h, "SubDB", -Filename => "dbhash.tmp",
-Flags => DB_CREATE,
-Mode => 0640 );
--- BerkeleyDB-0.24~/t/queue.t 2003-09-27 08:45:10 +0000
+++ BerkeleyDB-0.24/t/queue.t 2003-10-28 10:49:25 +0000
@@ -662,6 +662,7 @@ EOM
my $X ;
my $rec_len = 34 ;
eval '
+ use BerkeleyDB;
$X = tie(@h, "SubDB", -Filename => "dbqueue.tmp",
-Flags => DB_CREATE,
-Mode => 0640 ,
--- BerkeleyDB-0.24~/t/recno.t 2002-06-04 17:17:22 +0000
+++ BerkeleyDB-0.24/t/recno.t 2003-10-28 10:50:01 +0000
@@ -613,6 +613,7 @@ EOM
my @h ;
my $X ;
eval '
+ use BerkeleyDB;
$X = tie(@h, "SubDB", -Filename => "dbrecno.tmp",
-Flags => DB_CREATE,
-Mode => 0640 );
Using Gisle's advice and Tim's idea, here's a program that populates
a hash with more than a million keys. It then finds the largests
bucket, and average non-empty bucket size. There doesn't seem to be
a significant difference in the bucket size distribution between
5.8.0, 5.8.1 and 5.8.2-RC1. However, 5.8.2-RC1 is quite a bit faster
than 5.8.0 and 5.8.1.
Abigail
#!/usr/bin/perl
use strict;
use warnings;
no warnings qw /syntax/;
my $opts = '-MDevel::Peek';
my $prog = 'my %hash; while (<>) {$hash {$_} ++} Dump \%hash';
my @args = qw !/home/abigail/Words/woorden.max
/home/abigail/Words/allwords.txt!;
my $time = '/usr/bin/time';
my $frmt = '%S %U';
my @vers = qw /5.8.0 5.8.1 5.8.2-RC1/;
my $runs = 10;
print "Version Time Bucket-length\n";
print " Avg Max\n";
for my $version (@vers) {
my $perl = "/opt/perl/$version/bin/perl";
for my $c (1 .. $runs) {
print STDERR "$version: $c\n";
my @info = `$time -f '$frmt' $perl $opts -e '$prog' @args 2>&1`;
my ($user, $sys) = split ' ', $info [-1];
my $cpu = $user + $sys;
my $info = (grep {/ARRAY/} @info) [0];
$info =~ s/^[^(]*[(]//;
$info =~ s/[)][^)]*$//;
my ($max_length, $total_buckets, $total_keys) = (0, 0, 0);
while ($info =~ /(\d+):(\d+)/g) {
my ($length, $buckets) = ($1, $2);
next unless $length;
$total_keys += $length * $buckets;
$total_buckets += $buckets;
$max_length = $length if $length > $max_length;
}
my $avg_length = $total_keys / $total_buckets;
printf "%-10s (%5.2f) %5.2f %3d\n" =>
$version, $cpu, $avg_length, $max_length;
last if $version eq '5.8.0';
}
}
__END__
Version Time Bucket-length
Avg Max
5.8.0 (17.44) 1.27 7
5.8.1 (17.67) 1.27 8
5.8.1 (17.57) 1.27 7
5.8.1 (17.45) 1.27 7
5.8.1 (17.54) 1.27 7
5.8.1 (17.46) 1.27 7
5.8.1 (17.50) 1.27 7
5.8.1 (17.45) 1.27 7
5.8.1 (17.45) 1.27 6
5.8.1 (17.67) 1.27 7
5.8.1 (17.44) 1.27 7
5.8.2-RC1 (13.47) 1.27 6
5.8.2-RC1 (13.53) 1.27 7
5.8.2-RC1 (13.59) 1.27 7
5.8.2-RC1 (13.64) 1.27 6
5.8.2-RC1 (13.68) 1.27 7
5.8.2-RC1 (13.54) 1.27 7
5.8.2-RC1 (13.69) 1.27 8
5.8.2-RC1 (13.61) 1.27 7
5.8.2-RC1 (13.58) 1.27 7
5.8.2-RC1 (13.45) 1.27 6
Wild guess : try reverting patch #21452.
Fix bug [perl #24212] : improper error recovery in the
tokenizer after an unknown filetest operator.
Where can I get the patch?
Sorry for being rather novice to p5p stuff.
I'm working on dor-5.8.2.diff, and remember someone said me to remember to
change to diff to include some new feature.
Anyone?
And what about the plans to take the diff into core?
I've got no problem with maintaining it on my CPAN home
--
H.Merijn Brand Amsterdam Perl Mongers (http://amsterdam.pm.org/)
using perl-5.6.1, 5.8.0, & 5.9.x, and 806 on HP-UX 10.20 & 11.00, 11i,
AIX 4.3, SuSE 8.2, and Win2k. http://www.cmve.net/~merijn/
http://archives.develooper.com/daily...@perl.org/ per...@perl.org
send smoke reports to: smokers...@perl.org, QA: http://qa.perl.org
from the APC :
http://public.activestate.com/gsar/APC/
but don't bother, I tried it and this regression was not caused by
this patch. Maybe something with Exporter ??
(Andreas, maybe it's time to try it with your binsearchaperl. I'm
trying to reduce the test case.)
most probably :
[ 21406] By: ams on 2003/10/06 04:22:11
Log: Subject: Re: [perl #24076] "<> err EXPR" and warnings.
From: Rafael Garcia-Suarez <rgarci...@free.fr>
Date: Sun, 5 Oct 2003 21:27:54 +0200
Message-Id: <20031005212754.5ef...@free.fr>
(Applied with tweaks to op.c and a comment.)
Branch: perl
! op.c t/lib/warnings/op
It would make (my) life easier if said diff was distributed with the core.
Abigail
Yes. That's it. Integrated. Now testing ...
Expect a diff soon
> [ 21406] By: ams on 2003/10/06 04:22:11
> Log: Subject: Re: [perl #24076] "<> err EXPR" and warnings.
> From: Rafael Garcia-Suarez <rgarci...@free.fr>
> Date: Sun, 5 Oct 2003 21:27:54 +0200
> Message-Id: <20031005212754.5ef...@free.fr>
> (Applied with tweaks to op.c and a comment.)
> Branch: perl
> ! op.c t/lib/warnings/op
--
> It would make (my) life easier if said diff was distributed with the
> core.
>
>
>
> Abigail
>
Why not make it a compile time option instead of a diff?
Arthur
Binary compatibilty. As this adds a new opcode, it requires
rerunning opcode.pl before compiling. ETOOHAIRY (but patches
welcome :)
Ah, cool. Thanks.
Tim.
Which the diff prevents, since it includes the changed that would have been
made by opcode.pl
> ETOOHAIRY (but patches welcome :)
We decided in Paris to do it as a seperate diff, because people who want
defined-or would probably know how to apply the diff. Including it in the
Configure phase (later cannot be done) is not worth the time and effort, since
5.9.x and 5.10.x will have dor anyway.
The only problem is probably that people will have to change^wcheck the diff
after every patch, which is no fun!
Would a Configure flag that runs "patch -p0 < dor.diff" be
acceptable?
Abigail
I would not like that. How about a message in the beginning of Configure that
states something like
# Configure -des
:
:
5.8.1 and on /can/ give you the new defined-or functionality
found in perl6 and perl5.10, but you will have to apply a
patch by hand and re-run Configure.
Note that applying this patch will make your perl binary incompatible
with non-dor perl binaries.
Do you want defined-or support? Yes
now run "patch -p1 <dor.diff" and then rerun Configure
#
> Abigail
The uploaded file
dor-5.8.2-rc1.diff
has entered CPAN as
file: $CPAN/authors/id/H/HM/HMBRAND/dor-5.8.2-rc1.diff
size: 382134 bytes
md5: 57fc0f57c95f5276917f78d11972d9d2
>I'm particularly interested in
>
> 3: reassurance that it still build on Windows, VMS, OS/2, ...
>
> 5: is modperl still happy
>
>
>
Builds/tests OK on Windows XP with MSVC++, and both mod_perl 1.29 and
mod_perl 1.99_11-dev pass all their tests.
- Steve
Blame patch 21418.
Subject: [PATCH] Fixing UNIVERSAL.pm's bit of unpleasantness
From: sch...@pobox.com
Date: Mon, 6 Oct 2003 13:14:36 -0700
Message-Id: <2003100613...@ttul.org>
Here's a self-contained test case :
(the line 'use UNIVERSAL qw( isa );' triggers the failure)
BEGIN {
package U;
use Exporter;
@ISA = qw(Exporter);
sub FOO () { 42 };
@EXPORT = 'FOO';
use UNIVERSAL qw( isa );
package T;
use Exporter;
U->import();
@EXPORT = @U::EXPORT;
}
BEGIN { T->import(); }
use strict;
FOO;
FWIW I think the bug is in BerkeleyDB, as it relies on an import()
method that is not here, except by an unfortunate side-effect of
UNIVERSAL.pm.
Here's the relevant excerpt of the summary at http://nntp.x.perl.org/group/perl.perl5.summary/76 :
When you load UNIVERSAL
Slaven Rezic reports that explicitely loading the UNIVERSAL module
breaks things, because UNIVERSAL.pm defines an import() method (aliased
to Exporter::import()), which is thus inherited by all classes whenever
UNIVERSAL.pm has been use'd. Michael G Schwern provides a patch,
ensuring that UNIVERSAL::import() does nothing when invoked on anything
else than the UNIVERSAL package.
That said, I'm not convinced that changing the guts of UNIVERSAL
is suitable for maint -- i.e. does it solve enough problems to justify
breaking compatibilty. Nicholas decides !
Now that is the kind of backward (in)?compatibility I can live with :-)
__________________________________________________________________
Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide ---> http://perl.apache.org
mailto:st...@stason.org http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org http://ticketmaster.com
Yes, sorry. TODO. Will try tonight, else tomorrow night.
[currently I feel shattered]
Nicholas Clark
modperl on XP? Oooh. Nice. Reassurance squared. Thanks for testing this.
Nicholas Clark
Steve, it's not there yet. So far we had the hashing fixups for 5.8.1+. I need
to cut off that + and do it only for 5.8.1. I was waiting for the dust to
settle down with the plan C. Please watch the commit logs.
Fortunately its not a problem with BerkeleyDB itself but just the tests.
The snippet Alexey posted is from the btree.t sub-class test and it
inherits from BerkeleyDB::Btree but copies @BerkeleyDB::EXPORT and never
inherits from exporter.
Since this is just test code doing some slightly dodgy things, I think we're
ok fixing the UNIVERSAL::import() bug. It won't effect installed versions
of BerkeleyDB. All we need is a patch for a fixed version of BDB for new
installs. And wow, there's one attached to this very email! :)
--
Michael G Schwern sch...@pobox.com http://www.pobox.com/~schwern/
We don't know. But if we did, we wouldn't tell you.
> 5.8.1 (17.67) 1.27 8
> 5.8.1 (17.57) 1.27 7
> 5.8.1 (17.45) 1.27 7
> 5.8.1 (17.54) 1.27 7
> 5.8.1 (17.46) 1.27 7
> 5.8.1 (17.50) 1.27 7
> 5.8.1 (17.45) 1.27 7
> 5.8.1 (17.45) 1.27 6
> 5.8.1 (17.67) 1.27 7
> 5.8.1 (17.44) 1.27 7
> 5.8.2-RC1 (13.47) 1.27 6
> 5.8.2-RC1 (13.53) 1.27 7
> 5.8.2-RC1 (13.59) 1.27 7
> 5.8.2-RC1 (13.64) 1.27 6
> 5.8.2-RC1 (13.68) 1.27 7
> 5.8.2-RC1 (13.54) 1.27 7
> 5.8.2-RC1 (13.69) 1.27 8
> 5.8.2-RC1 (13.61) 1.27 7
> 5.8.2-RC1 (13.58) 1.27 7
> 5.8.2-RC1 (13.45) 1.27 6
I didn't have a million words, and could really replicate your nice
clear timings.
I hope that this speedup is real, and if so I suspect that the cause
is due to a change in when hashes split. Firstly I duplicated
Tye McQueen's bug fix (19423) into hv_store_ent, so that hashes split
somewhat earlier.
I then changed the split trigger so that a split also happens if any of
the linked lists gets too long. My hunch is think that this latter
change is the cause of any speedup, given that the end result is hashes
with similar distributions to 5.8.1. I'd hope that earlier splits are
causing less linear scans of the hashes during data insertion.
Could you try changing "too long" from 4 to 3, and see if that changes
the speed of RC1?
Nicholas Clark
--- hv.c.orig 2003-10-26 21:10:13.000000000 +0000
+++ hv.c 2003-10-28 19:52:09.000000000 +0000
@@ -20,7 +20,7 @@
#define PERL_IN_HV_C
#include "perl.h"
-#define HV_MAX_LENGTH_BEFORE_SPLIT 4
+#define HV_MAX_LENGTH_BEFORE_SPLIT 3
STATIC HE*
S_new_he(pTHX)
It doesn't seem to make much of a difference:
Version Time Bucket-length
Avg Max
5.8.0 (17.41) 1.27 7
5.8.1 (18.05) 1.27 7
5.8.1 (17.61) 1.27 7
5.8.1 (17.42) 1.27 7
5.8.1 (17.83) 1.27 7
5.8.1 (17.58) 1.27 7
5.8.1 (17.63) 1.27 7
5.8.1 (17.84) 1.27 7
5.8.1 (17.54) 1.27 6
5.8.1 (17.59) 1.27 7
5.8.1 (18.67) 1.27 7
5.8.2-RC1 (13.69) 1.27 7
5.8.2-RC1 (13.70) 1.27 7
5.8.2-RC1 (13.78) 1.27 7
5.8.2-RC1 (13.66) 1.27 7
5.8.2-RC1 (13.81) 1.27 7
5.8.2-RC1 (13.72) 1.27 8
5.8.2-RC1 (13.87) 1.27 6
5.8.2-RC1 (13.64) 1.27 7
5.8.2-RC1 (13.70) 1.27 7
5.8.2-RC1 (13.83) 1.27 7
5.8.2-RC1.nick (13.81) 1.27 7
5.8.2-RC1.nick (13.86) 1.27 7
5.8.2-RC1.nick (13.68) 1.27 7
5.8.2-RC1.nick (13.64) 1.27 7
5.8.2-RC1.nick (13.52) 1.27 7
5.8.2-RC1.nick (13.56) 1.27 7
5.8.2-RC1.nick (13.58) 1.27 7
5.8.2-RC1.nick (13.54) 1.27 7
5.8.2-RC1.nick (13.64) 1.27 7
5.8.2-RC1.nick (13.61) 1.27 8
5.8.2-RC1.nick is with HV_MAX_LENGTH_BEFORE_SPLIT set to 3.
Abigail
> On Tue, Oct 28, 2003 at 05:26:02PM +0100, Rafael Garcia-Suarez wrote:
> > FWIW I think the bug is in BerkeleyDB, as it relies on an import()
> > method that is not here, except by an unfortunate side-effect of
> > UNIVERSAL.pm.
> <snip>
> > That said, I'm not convinced that changing the guts of UNIVERSAL
> > is suitable for maint -- i.e. does it solve enough problems to justify
> > breaking compatibilty. Nicholas decides !
>
> Fortunately its not a problem with BerkeleyDB itself but just the tests.
> The snippet Alexey posted is from the btree.t sub-class test and it
> inherits from BerkeleyDB::Btree but copies @BerkeleyDB::EXPORT and never
> inherits from exporter.
>
> Since this is just test code doing some slightly dodgy things, I
> think we're
> ok fixing the UNIVERSAL::import() bug. It won't effect installed versions
> of BerkeleyDB. All we need is a patch for a fixed version of BDB for new
> installs. And wow, there's one attached to this very email! :)
Thanks for the patch Michael.
I'm a bit confused though - there are an identical set of tests in DB_File
(BerkeleyDB inherited a lot of tests from DB_File, including these). If this
is in 5.8.2-tobe, why hasn't DB_File fallen over?
Paul
The bug is due to the presence in various places in BerkeleyDB.pm of
"use UNIVERSAL qw( isa );". This doesn't appear in DB_File.
In the DB_File case, SubDB inherits from DB_File which inherits from Exporter.
It then copies @DB_File::Exporter. This is correct.
In the BerkelyDB case, SubDB inherits from BerkeleyDB::Hash (or Btree or
Queue or Recno) which does not inherit from Exporter. It then copies
@BerkeleyDB::Exporter. This is incorrect. It was only working because
SubDB incidentally inherited import() from UNIVERSAL.
Either SubDB should inherit from both BerkeleyDB and BerkeleyDB::Salad or
the BerkeleyDB::Salad should inherit from BerkeleyDB or SubDB should inherit
from both BerkeleyDB and Exporter. I did the latter, since it had the
smallest impact on your code, but I think since SubDB is exporting
BerkeleyDB functions it should do one of the two former ones. #2 is
probably most correct if you expect folks inheriting from BerkeleyDB::Salad
to be able to export BerkeleyDB functions.
--
Michael G Schwern sch...@pobox.com http://www.pobox.com/~schwern/
You're the sickest teenager I've ever set my wallet on.