Perl 5.10.1

18 views
Skip to first unread message

Jonathan Leto

unread,
Jun 20, 2009, 4:05:18 PM6/20/09
to perl5-...@perl.org
Howdy,

As an exercise, I checked out the perl-5.10 git tag and then
cherry-picked Rick Delaney's "Big slowdown in 5.10 @_ parameter
passing " commit [1]. I ran all the tests and they pass on my machine.
Obviously a bit more smoke testing is needed, but I took this as a
good sign.

This is in response to chromatic's comments [2]:

Is pointing out that a patch for a known performance regression
languishing, unreleased, in bleadperl for 17 months a problem? Is that
criticism dismissible from everyone who isn't currently a Perl 5
committer or the maintainer of a core module?

I think that releasing Perl 5.10.1 with only this change is very
valuable to the Perl community, so I went and tried to help it along.

Is this a viable option?

Cheers,

[1] http://github.com/leto/perl/tree/perl-5.10.1
[2] http://modernperlbooks.com/mt/2009/06/who-gets-to-criticize-your-free-software-project.html

--

Jonathan Leto
jona...@leto.net
http://leto.net

Nicholas Clark

unread,
Jun 20, 2009, 5:29:37 PM6/20/09
to Michael G Schwern, Jonathan Leto, perl5-...@perl.org
On Sat, Jun 20, 2009 at 01:43:39PM -0700, Michael G Schwern wrote:

> repository is available to all, and Jonathan just did that. Nick has been
> wondering if moving to git was worth it and here it is.

No, I wasn't wondering whether the move to git was worth it. I think that it
was. (Given that we have all the history from perforce)

What I *was* wondering was whether it has changed the activity level in the
core repository. Particularly, whether there was any evidence that the people
who vocally complained that perforce was a big barrier to them hacking on the
core were now hacking on core, given that the barrier has been removed.

There's a lot of noise in the commit rate data, but my current view from it
is that there's no material change in the activity level since the git
conversion.

Nicholas Clark

Nicholas Clark

unread,
Jun 20, 2009, 5:43:09 PM6/20/09
to Jonathan Leto, perl5-...@perl.org
On Sat, Jun 20, 2009 at 01:05:18PM -0700, Jonathan Leto wrote:
> Howdy,
>
> As an exercise, I checked out the perl-5.10 git tag and then
> cherry-picked Rick Delaney's "Big slowdown in 5.10 @_ parameter
> passing " commit [1]. I ran all the tests and they pass on my machine.
> Obviously a bit more smoke testing is needed, but I took this as a
> good sign.
>
> This is in response to chromatic's comments [2]:
>
> Is pointing out that a patch for a known performance regression
> languishing, unreleased, in bleadperl for 17 months a problem? Is that

No, not a problem.

However, in return, I'd like to point out that if the Perl community *had*
tested 5.10 before release, they would have spotted this speed regression,
so to some extent they are getting what they deserve.

> criticism dismissible from everyone who isn't currently a Perl 5
> committer or the maintainer of a core module?

No, it's not being dismissed.

> I think that releasing Perl 5.10.1 with only this change is very
> valuable to the Perl community, so I went and tried to help it along.
>
> Is this a viable option?

1: There would be a lot of stick about "17 months, and that's *all*?"
2: I think it would be unwise - 5.10.0 contains a defective smartmatch
implementation that we want to kill before it takes hold.
(Again, if the Perl community had tested this in the *year or more* before
blead became 5.10.0, it would have been spotted in time.)
Rightly or wrongly, there's a superstition about installing .0 releases.
(extrapolated logically, that will mean that .1 will become the new .0,
then .2 the new .1 is .0)
We have the fixed smartmatch - merge that too.


As is, currently, this is not the route that Dave wants to take. I'm not Dave.
It's not my call. If I understand Dave's view correctly, he'd like to triage
the following bugs before going to RC1:

http://rt.perl.org/rt3//Public/Search/Simple.html?Query=MemberOf%3D66092


So would definitely be helpful to getting Dave to ship 5.10.1 would be to
look through that list of bugs, and pick off any they feel able to make
progress on. Progress includes any of

1: Verifying that the bug is still present in blead
2: Writing a test that will pass once the bug is fixed
3: Using that test to run a bisect to find when the bug was introduced
4: Actually fixing the bug

Obviously number 4 is the goal, but even getting to number 2 is useful.
Some bugs are already bisected.

Note also, that's not an exhaustive list.

RT's statuses don't record that there's no need to look at 60574 and 65582,
because I believe I know how to fix them. (Just no tuits yet)

Nicholas Clark

Jonathan Leto

unread,
Jun 20, 2009, 6:05:38 PM6/20/09
to perl5-...@perl.org
Howdy,

> There's a lot of noise in the commit rate data, but my current view from it
> is that there's no material change in the activity level since the git
> conversion.

I am sure you are correct that raw commits have not skyrocketed since
the conversion to git, but you cannot judge community involvement or
openness in raw commits alone. I know for a fact that the conversion
to git has done wonders for killing FUD about the Perl community.

The barrier to entry for hacking on Perl 5 went from an infinite
potential well to a gentle speed bump. (Yes, I just called git a
gentle speed bump, but in a good way.) The effects of this have only
begun to be felt.

Cheers,

Chromatic

unread,
Jun 20, 2009, 9:44:52 PM6/20/09
to perl5-...@perl.org
It would be my pleasure to write a short note for the perldelta or
README or both to explain that the semantics of 5.10 and 5.10.1 smart
matching are changing in certain circumstances. Would that suffice to
notify reasonable people that Perl 5.10.1 does not further entrench
behavior already changed in bleadperl? (I don't care about unreasonable
people -- they're unpleasable.)

On Sat, Jun 20, 2009 at 10:43:09PM +0100, Nicholas Clark wrote:

> However, in return, I'd like to point out that if the Perl community *had*
> tested 5.10 before release, they would have spotted this speed regression,
> so to some extent they are getting what they deserve.

Anyone who believes that release candidates get any sort of exhaustive
community testing is either dangeriously naive or hasn't paid attention
to the negligible amount of testing release candidates do get.

> (Again, if the Perl community had tested this in the *year or more* before
> blead became 5.10.0, it would have been spotted in time.)

I don't understand this argument. How is it okay for a pumpking to hold
back a release to punish the filthy proles who caught and fixed a
performance regression a couple of weeks on the wrong side of a release,
because cherrypicking patches to make a release is difficult (even if
someone else has just done all of the work, like pumpkings always beg
someone else to do), because regressions are so difficult that long
testing periods are absolutely necessary to ensure that stable releases
do not contain nasty regressions (even though this patch *fixes* a nasty
regression and that there's insufficient testing of unstable
non-releases), and because a misfeature present in two stable releases
is somehow more used and difficult to change in the future than a
misfeature present in one stable release unpatched for seventeen months?

I'm certain that berating and blaming existing testers for missing a
regression (and potential testers for not testing) is a very effective
approach to less -- not more -- testing in the future, let alone further
contributions.

-- c

David Golden

unread,
Jun 20, 2009, 11:59:48 PM6/20/09
to Michael G Schwern, Jonathan Leto, Perl 5 Porters
On Sat, Jun 20, 2009 at 10:57 PM, Michael G Schwern<sch...@pobox.com> wrote:
> First is that there is THE "5.10.1" and that this quick release somehow
> challenges that.  This is an artifact of a very slow release process where we
> have a huge long ramp up to not just "the next stable release" but to "5.10.1"
> (or whatever) and the name becomes significant.

If expectations about the numbering scheme are the holdup, how hard is
it to just change the scheme?

5.10.0.1

And then we can have "major", "minor" and "micro" releases. And
reserve micro releases for break-fix stuff only, not new features,
which will automatically keep the scope constrained.

-- David

David Golden

unread,
Jun 21, 2009, 12:03:18 AM6/21/09
to Jonathan Leto, perl5-...@perl.org
On Sat, Jun 20, 2009 at 6:05 PM, Jonathan Leto<jal...@gmail.com> wrote:
> The barrier to entry for hacking on Perl 5 went from an infinite
> potential well to a gentle speed bump. (Yes, I just called git a
> gentle speed bump, but in a good way.) The effects of this have only
> begun to be felt.

As someone who just recently started sending patches, I'll second
this. Git makes it easy to stay in sync, work on my own private
branch, rebase my branch whenever I need to, squash my own tiny
commits into a single patch, and otherwise protect me from myself that
it's definitely made working on Perl a good bit less intimidating to
consider.

I'm not saying it's a cure for cancer or anything, just that I
probably would never have considered it until the switch to git (or
maybe SVK, but I like git better, now).

-- David

Darren Duncan

unread,
Jun 20, 2009, 5:39:07 PM6/20/09
to perl5-...@perl.org
Jonathan Leto wrote:
> As an exercise, I checked out the perl-5.10 git tag and then
> cherry-picked Rick Delaney's "Big slowdown in 5.10 @_ parameter
> passing " commit [1]. I ran all the tests and they pass on my machine.
> Obviously a bit more smoke testing is needed, but I took this as a
> good sign.
>
> I think that releasing Perl 5.10.1 with only this change is very
> valuable to the Perl community, so I went and tried to help it along.
>
> Is this a viable option?

I think that's a great idea.

Moreover, it will hopefully start a series of more frequent and simpler
releases. Follow the Parrot/Rakudo mold, of just releasing something on X
intervals, containing updates that are ready at the time.

Now that Git is being used, which makes this a lot easier to do, best to exploit it.

Now besides the above bug, there were some other significant regression bugs
that AFAIK also had fairly simple fixes, and those could be cherry-picked into
5.10.1 also, or alternately a soon-after 5.10.2 could contain the fixes that are
both high impact and simple fixes, such that the current release-stopper list
would then just block 5.10.3 or something.

Generally speaking, it should be fine to make any sort of release as long as it
is known to not contain a regression from the immediately prior release.

-- Darren Duncan

Nicholas Clark

unread,
Jun 21, 2009, 2:49:32 AM6/21/09
to chromatic, perl5-...@perl.org
On Sat, Jun 20, 2009 at 06:44:52PM -0700, chromatic wrote:
> It would be my pleasure to write a short note for the perldelta or
> README or both to explain that the semantics of 5.10 and 5.10.1 smart
> matching are changing in certain circumstances. Would that suffice to
> notify reasonable people that Perl 5.10.1 does not further entrench
> behavior already changed in bleadperl? (I don't care about unreasonable
> people -- they're unpleasable.)

Thank you for your kind offer. However, I notice that such a note has already
been written:

http://perl5.git.perl.org/perl.git/blob/maint-5.10:/pod/perl5101delta.pod#l43

> On Sat, Jun 20, 2009 at 10:43:09PM +0100, Nicholas Clark wrote:
>
> > However, in return, I'd like to point out that if the Perl community *had*
> > tested 5.10 before release, they would have spotted this speed regression,
> > so to some extent they are getting what they deserve.
>
> Anyone who believes that release candidates get any sort of exhaustive
> community testing is either dangeriously naive or hasn't paid attention
> to the negligible amount of testing release candidates do get.

Or didn't arrange it themselves. Which is why Matt Trout will be ensuring that
this time the core Catalyst community thoroughly test this one.

> > (Again, if the Perl community had tested this in the *year or more* before
> > blead became 5.10.0, it would have been spotted in time.)
>
> I don't understand this argument. How is it okay for a pumpking to hold
> back a release to punish the filthy proles who caught and fixed a
> performance regression a couple of weeks on the wrong side of a release,
> because cherrypicking patches to make a release is difficult (even if
> someone else has just done all of the work, like pumpkings always beg
> someone else to do), because regressions are so difficult that long

Cherry picking is not always hard. Nor is responding to individual bug reports.
Sometimes it's hard. But

a: it mounts up if you don't keep on top of it
b: the amount of time and mental effort taken to complete a task can balloon
beyond the amount of time you had available in a stretch (for example an
evening)

at which point you hit a demoralising road block, if you're a volunteer, and
all you have is odd evenings and time you "borrow" from work.

I'm not convinced that a decentralised cherry-picking system done as pull
requests is the way to go. However Best Practical have been writing a
commit tagging system for the explicit purpose of decentralising the initial
work of writing a perldelta, and partitioning "compatible" and "incompatible"
changes. I'm not sure what the state of that is - I believe currently it's
being alpha tested by some people outside Best Practical.

Nicholas Clark

Nicholas Clark

unread,
Jun 21, 2009, 2:59:58 AM6/21/09
to Darren Duncan, perl5-...@perl.org
On Sat, Jun 20, 2009 at 02:39:07PM -0700, Darren Duncan wrote:

> Now besides the above bug, there were some other significant regression
> bugs that AFAIK also had fairly simple fixes, and those could be
> cherry-picked into 5.10.1 also, or alternately a soon-after 5.10.2 could
> contain the fixes that are both high impact and simple fixes, such that the
> current release-stopper list would then just block 5.10.3 or something.

I believe that all that have fixes have already been merged into

http://perl5.git.perl.org/perl.git/tree/maint-5.10

Dave's view is that he'd like to triage and fix these remaining bugs:

http://rt.perl.org/rt3//Public/Search/Simple.html?Query=MemberOf%3D66092

> Generally speaking, it should be fine to make any sort of release as long
> as it is known to not contain a regression from the immediately prior
> release.

I agree with the importance of not introducing regressions.

If a project has a reputation for making releases with regressions, then
it acts as a strong deterrent to upgrading from the existing system, that
is known to work. Upgrading might bring benefits, but it might also
introduce new bugs, and those might be worse than the bugs you already know
about and know how to work round.

Whereas if the reputation is one of (near) zero regressions, then it's an
incentive to upgrade - bugs will be fixed, but you won't be getting new bugs.

Nicholas Clark

Craig A. Berry

unread,
Jun 21, 2009, 3:33:54 AM6/21/09
to Jonathan Leto, perl5-...@perl.org
On Sat, Jun 20, 2009 at 3:05 PM, Jonathan Leto<jal...@gmail.com> wrote:

> As an exercise, I checked out the perl-5.10 git tag and then
> cherry-picked Rick Delaney's "Big slowdown in 5.10 @_ parameter
> passing " commit [1]. I ran all the tests and they pass on my machine.
> Obviously a bit more smoke testing is needed, but I took this as a
> good sign.


It's a good exercise. Would've been nice to have people doing that 17
months ago.

> This is in response to chromatic's comments [2]:
>
> Is pointing out that a patch for a known performance regression
> languishing, unreleased, in bleadperl for 17 months a problem? Is that
> criticism dismissible from everyone who isn't currently a Perl 5
> committer or the maintainer of a core module?

Pointing it out is not a problem. Suddenly suggesting that we
abandon public commitments and act on this one bug fix to the
exclusion of anything else that's happened in the interim is a bit
weird.

> I think that releasing Perl 5.10.1 with only this change is very
> valuable to the Perl community, so I went and tried to help it along.
>
> Is this a viable option?

It's a bit like walking into a party halfway through and saying, "I'd
like to propose the first toast." I'm sure people would like to
acknowledge the gesture, but it isn't the first because, well, it just
isn't. Anyone seriously wanting to see 5.10.1 out the door is
tracking the maint-5.10 branch, not the 5.10.0 tag.

Tracking maint-5.10 can mean a number of things. It obviously means
building it and running the test suite on your preferred platform.
Testing your favorite CPAN modules is a plus. Testing any darkpan
modules to which you have access is a double plus. Scouring eBay and
the alleys in your neighborhood for abandoned hardware on which to run
Perl tests is, well, a bit obsessive, but I've done it. Choose your
own level of eccentricity, technical proficiency, or whatever. But do
stick around.

Chromatic

unread,
Jun 21, 2009, 3:53:12 AM6/21/09
to perl5-...@perl.org, Craig A. Berry
On Sunday 21 June 2009 00:33:54 Craig A. Berry wrote:

> Anyone seriously wanting to see 5.10.1 out the door is
> tracking the maint-5.10 branch, not the 5.10.0 tag.

By this definition, the current approach which has failed to release 5.10 in
seventeen months (including addressing an important and embarrassing
regression detected seventeen months ago) is the only serious (and by
implication, correct) approach. By further implication, anyone not already
participating in the current process is not serious and does not care enough
about 5.10.1.

I prefer my well unpoisoned.

> Suddenly suggesting that we abandon public commitments
> and act on this one bug fix to the exclusion of anything else
> that's happened in the interim is a bit weird.

Is this the same public commitment not to release any code with important and
embarrassing regressions? What part of the current process has not abandoned
that commitment a very long time ago?

I also reject this false dilemma that ameliorating the damage of an important
and embarrassing regression precludes further development in the 5.10.x maint
series.

Every day that Perl 5.10 is the most recent, most modern Perl release
*increases* the number of people affected by the performance regression.

Every day that Perl 5.10 is the most recent, most modern Perl release (which
includes no warning that the smartmatch semantics will eventually change in
incompatible ways) further entrenches that broken behavior and ensures that an
eventual backwards-incompatible release will break *even more* code.

A *new* volunteer has already done the *very modest* work required to produce
a 5.10.1 which can address both of these problems *right now*. 5.10.1 could
be out in a week.

Which public commitments would releasing 5.10.1 with the perldelta warning and
the performance improvement violate?

-- c

Craig A. Berry

unread,
Jun 21, 2009, 4:13:57 AM6/21/09
to Michael G Schwern, Jonathan Leto, Perl 5 Porters
On Sat, Jun 20, 2009 at 9:57 PM, Michael G Schwern<sch...@pobox.com> wrote:
> Nicholas Clark wrote:

>> As is, currently, this is not the route that Dave wants to take. I'm not Dave.
>> It's not my call. If I understand Dave's view correctly, he'd like to triage
>> the following bugs before going to RC1:
>>
>>   http://rt.perl.org/rt3//Public/Search/Simple.html?Query=MemberOf%3D66092
>>
>>
>> So would definitely be helpful to getting Dave to ship 5.10.1 would be to
>> look through that list of bugs, and pick off any they feel able to make
>> progress on. Progress includes any of
>

> Hold on.  There's several assumptions here that got glossed over.  They're
> important.


>
> First is that there is THE "5.10.1" and that this quick release somehow
> challenges that.

> Second is the implication that a quick critical bugfix release will draw
> effort away from the larger release.

It already does and it already has, just by having this discussion
(again). For once we have something that is essentially a public
roadmap, though a bit informal. It would be nice to formalize it, put
it on the wiki, or something. But we have a plan for what 5.10.1
should be. Abandoning that now would just create confusion and churn.

Andy Armstrong

unread,
Jun 21, 2009, 5:15:10 AM6/21/09
to David Golden, Jonathan Leto, perl5-...@perl.org
On 21 Jun 2009, at 05:03, David Golden wrote:
> I'm not saying it's a cure for cancer or anything, just that I
> probably would never have considered it until the switch to git (or
> maybe SVK, but I like git better, now).


Same here. I've spent more time looking at the perl source in the last
couple of weeks than in the last 18 months - which is completely down
to git. It means I can work with the source as a first-class citizen
and submit changes to origin knowing they should be trivial to
integrate.

The fence is down but the herd will take a while to notice that
they're free.

--
Andy Armstrong, Hexten

Dave Mitchell

unread,
Jun 21, 2009, 9:08:53 AM6/21/09
to Michael G Schwern, Craig A. Berry, Jonathan Leto, Perl 5 Porters
I'm coming into this thread a bit late - I'm still about a week behind
on my p5p mailbox, and no-one thought to cc me on the thread, so I didn't
spot it earlier.

The situation with maint-5.10 is that

*) the dual-life module side of things is nearly sorted
*) smartmatch has been merged from bleed
*) there's still a big list of regressions from 5.8.x that need reviewing
and possibly fixing (note that I'm not saying they all have to be fixed
before 5.10.1; but at some point I need to review the list and decide
which ones must be fixed).
* I need to do some fixups to the release process to handle the New World
of git.

So all in all, I expect that the release of 5.10.1 is now a short number
of weeks away.

So...
I really can't see any point in releasing "5.10.0 + one bug fix" at this
point.


--
O Unicef Clearasil!
Gibberish and Drivel!
-- "Bored of the Rings"

David Golden

unread,
Jun 21, 2009, 9:24:50 AM6/21/09
to Perl 5 Porters
On Sun, Jun 21, 2009 at 9:08 AM, Dave Mitchell<da...@iabyn.com> wrote:
> So all in all, I expect that the release of 5.10.1 is now a short number
> of weeks away.
>
> So...
> I really can't see any point in releasing "5.10.0 + one bug fix" at this
> point.

While in general, I support a shorter release cycle for bug-fixes as I
mentioned in my post, I have to agree with Dave M. here.

Assuming that we really are a matter of weeks away, pushing a micro
release now will ultimately benefit few users, while it creates extra
work for OS packagers who will need to roll two sets of new perl
packages in short order.

I know I'm doing my part on the dual-life module side on M::B to get
it ready for release. I released M::B 0.33_02 and it went into blead.
I've finished the regression test of 0.33 to 0.33_02 on 3000+ modules
on CPAN with only 2 negative outcomes (and those from flaky modules
that might have spurious web failures). I'm about ready to roll
0.33_03, which has 15+ bugs closed in the M::B RT queue in the last
week, including all bugs marked "important" or above.

It sounds like Dave's big blockers are remaining dual-life modules --
Dave, run the tool again and name and shame people -- and bug list
triage.

http://rt.perl.org/rt3//Public/Search/Simple.html?Query=MemberOf%3D66092

So if people want to see 5.10.1 out the door, now's the time to go
look at the list, pick a bug, and opine on whether it needs to be
fixed or can sit and wait.

And if we *can* agree that a micro-release process makes sense going
forward, then the urgency to close out truly minor bugs in that list
should go down.

-- David

Gabor Szabo

unread,
Jun 21, 2009, 10:57:09 AM6/21/09
to Perl 5 Porters
I am really an outsider here without understanding the work that needs
to be done
for a release and without the time or knowledge to help but let me
give my point
of view.

I don't think downstream packagers will run and start packaging it unless
it turns out that the real 5.10.1 is not coming out in the next few weeks
for whatever reason or if they are close to freeze and they expect
that 5.10.1 will
come out after their dead-line.

I think a micro release with just one small bug fix would at least help people
figure out what is the price of that.

If the price is high it will hopefully point to the places where the
release process
needs to be improved.

If the price is already low then it might create an opportunity to
start releasing
more frequently.

It certainly will give an opportunity to the wider Perl community to test that
release and give feedback. As you have experienced 99.93 % of the users
will only touch a released version. That includes most of the CPAN authors as
well.

Gabor

David Golden

unread,
Jun 21, 2009, 11:25:10 AM6/21/09
to Gabor Szabo, Perl 5 Porters
On Sun, Jun 21, 2009 at 10:57 AM, Gabor Szabo<sza...@gmail.com> wrote:
> I think a micro release with just one small bug fix would at least help people
> figure out what is the price of that.
>
> If the price is high it will hopefully point to the places where the
> release process
> needs to be improved.

It certainly might be an opportunity for Dave M to delegate the
problem of figuring out the release process in the world of git.

> It certainly will give an opportunity to the wider Perl community to test that
> release and give feedback. As you have experienced 99.93 % of the users
> will only touch a released version. That includes most of the CPAN authors as
> well.

I hope that's less of an issue these days with CPAN Testers operating
at more scale. It's not automated, but if if Dave wants all of CPAN
(or a subset) smoked against a particular commit (at least on Linux),
I can probably turn that around in under a week. Or I can smoke two
separate commits and give the test grade deltas between them (doubles
the time, though). I suspect BinGOs and Andreas could be enlisted as
well for other operating systems.

-- David

Rafael Garcia-Suarez

unread,
Jun 21, 2009, 12:02:48 PM6/21/09
to Jonathan Leto, perl5-...@perl.org
2009/6/20 Jonathan Leto <jal...@gmail.com>:

> I think that releasing Perl 5.10.1 with only this change is very
> valuable to the Perl community, so I went and tried to help it along.
>
> Is this a viable option?

Not really, and I'll try to explain why in detail, and to propose an
alternative strategy.

First, the multiplication and rushing out of releases whenever a major
fix is committed in not desirable:

1. From a project management point of view, we're squeezing a release
between the last release point and the tip of the maintenance branch,
effectively multiplying tips, and thus multiplying effort. Why not put
innocent doc patch there too, for example? The slowness of release of
maint shall not be remedied to by multipling the number of maint
branches -- the number of maintainers is too low. This goes obvious
when said.

2. Perl 5 version numbers are not cheap. They correspond to more
subdirectories in @INC (and more stat calls to find modules), to more
versions to install for the CPAN authors and testers, to more space
and time on the smoke matrices, to more entries in Module::Corelist.
They also correspond to more hassle for OS packagers, with the
subdirectory layout changes (been there.)

3. Yes, there's been a long time between 5.10.0 and .1 : the first
maint release after a .0 seems to take more time because the code is
less stabilized. Also, a new pumpking started, and a new VCS was put
in place. We'll hopefully have in the future much closer releases,
like what Nicholas did for 5.8.x. Having a tool to help many people
cherry-picking patches to maint concurrently would also help
unbottlenecking the process.

However, we haven't failed at releasing quickly security patches in
the past, and we do have also a documented process to build perl with
a given set of patches, and make them reported by "perl -V". And if
you look at the perls distributed by RedHat, Debian, FreeBSD,
ActiveState, and a lot of other systems, you'll see that they have
many patches in there.

So, what we seem to need is, actually, a way to quickly bless,
distribute and advertise a small number of important patches.

I would suggest to maintain a list of important patches (with the
vendors). Git is a useful tool to release them. A ML could be created
to advertise them, but who cares about MLs nowadays? We would need to
advertise them as well 1. officially on dev.perl.org, 2. on use.perl /
perlbuzz / etc.

--
Every old idea will be proposed again with a different name and a different
presentation, regardless of whether it works.
-- RFC 1925

Ben Evans

unread,
Jun 21, 2009, 4:21:49 PM6/21/09
to Darren Duncan, perl5-...@perl.org
On Sat, Jun 20, 2009 at 10:39 PM, Darren Duncan <dar...@darrenduncan.net>wrote:

>
> Moreover, it will hopefully start a series of more frequent and simpler
> releases. Follow the Parrot/Rakudo mold, of just releasing something on X
> intervals, containing updates that are ready at the time.


Yes, but when you have zero production users (and mighty morphing APIs), all
release strategies are equally useful.

I personally find the Java version model of major, minor and update level to
work reasonably well. However, there have been a couple of instances I can
think of where breaking API changes (not just binary-incompatible but
actually source-level compile-time failures) were introduced at a minor
release. Both caused havoc at large shops I'm familiar with, and there was
an awful lot of cleanup which had to happen.

"Release to a fixed schedule, with whatever's ready" to my mind turns every
single minor release into that kind of Russian Roulette, and is something I
think should be avoided at all costs for a language which has a large,
complex and mature install base. It also does not address the need for
critical updates (eg last years US DST craziness) which may need to be made
available at very short notice.

I'm glad that the Parrot model works for them, but their situation is
completely different, and we have responsibilities to our users which should
be the paramount consideration when making these kinds of decisions.

Ben

David E. Wheeler

unread,
Jun 21, 2009, 11:26:18 PM6/21/09
to Perl 5 Porters
On Jun 21, 9:02 am, rgarciasua...@gmail.com (Rafael Garcia-Suarez)
wrote:

> First, the multiplication and rushing out of releases whenever a
major
> fix is committed in not desirable:
>
> 1. From a project management point of view, we're squeezing a release
> between the last release point and the tip of the maintenance branch,
> effectively multiplying tips, and thus multiplying effort. Why not
put
> innocent doc patch there too, for example? The slowness of release of
> maint shall not be remedied to by multipling the number of maint
> branches -- the number of maintainers is too low. This goes obvious
> when said.

Then we need more maintainers. The way to get more maintainers is to
release more often, with an automated release process. Sure there's a
chicken-and-the-egg problem here, but it starts to be solved by the
act of one egg. Surely there are enough eggheads on this list to make
it workable! ;-)

Also, why should a new minor release add a new tip? Are there not back
branches for the minor releases, with master/trunk/whatever designated
for 5.12? Why should a new release multiply the number of release
branches?

> 2. Perl 5 version numbers are not cheap. They correspond to more
> subdirectories in @INC (and more stat calls to find modules), to more
> versions to install for the CPAN authors and testers, to more space
> and time on the smoke matrices, to more entries in Module::Corelist.
> They also correspond to more hassle for OS packagers, with the
> subdirectory layout changes (been there.)

If minor releases are binary compatible, why should this be so? And
for those silly platforms that choose to do that, so what? I had 8
such directories for 5.8.8 and never complained, because things were
improving rapidly (every 3 months for a while!). I don't think most
folks really worry about the stat calls, that's hardly the slowest
thing in Perl. And I suspect that the CPAN testers will be the last to
complain if there are regular new releases of Perl with bug fixes.
Hell, a bunch of them run bleed anyway.

> 3. Yes, there's been a long time between 5.10.0 and .1 : the first
> maint release after a .0 seems to take more time because the code is
> less stabilized. Also, a new pumpking started, and a new VCS was put
> in place. We'll hopefully have in the future much closer releases,
> like what Nicholas did for 5.8.x.

Yay!

> Having a tool to help many people
> cherry-picking patches to maint concurrently would also help
> unbottlenecking the process.

Git should help with that.

> However, we haven't failed at releasing quickly security patches in
> the past, and we do have also a documented process to build perl with
> a given set of patches, and make them reported by "perl -V". And if
> you look at the perls distributed by RedHat, Debian, FreeBSD,
> ActiveState, and a lot of other systems, you'll see that they have
> many patches in there.
>
> So, what we seem to need is, actually, a way to quickly bless,
> distribute and advertise a small number of important patches.

Which no one will actually run. People run release, not patches.
Otherwise, why release at all?

> I would suggest to maintain a list of important patches (with the
> vendors). Git is a useful tool to release them. A ML could be created
> to advertise them, but who cares about MLs nowadays? We would need to
> advertise them as well 1. officially on dev.perl.org, 2. on
use.perl /
> perlbuzz / etc.

I think that this will work with only the geekiest of Perl hackers.
You know, those of us on Perl 5 porters already (okay, so I just re-
joined for the first time in years, sue me! ;-).

In all honesty, I'd love to see far more frequent releases. Any
technical barriers to such releases should be removed, IMHO.

Best,

David

Craig A. Berry

unread,
Jun 22, 2009, 12:22:12 AM6/22/09
to chromatic, perl5-...@perl.org
On Sun, Jun 21, 2009 at 2:53 AM, chromatic<chro...@wgz.org> wrote:
> On Sunday 21 June 2009 00:33:54 Craig A. Berry wrote:
>
>> Anyone seriously wanting to see 5.10.1 out the door is
>> tracking the maint-5.10 branch, not the 5.10.0 tag.
>
> By this definition, the current approach which has failed to release 5.10 in
> seventeen months (including addressing an important and embarrassing
> regression detected seventeen months ago) is the only serious (and by
> implication, correct) approach.

Approaches don't release software, people do. One of the reasons it's
important to track the actual maint-5.10 branch (or blead if you want
to stay a few days or a week ahead) is that you would then know what
people have really been doing and the immense progress that has been
made. You would know that the pace of 5.10.1 development has been
accelerating. You would know that new tools and processes have arisen
to meet new challenges in 5.10.x, such as the much larger number of
modules than there were in 5.8.x. You might be aware that there are
1300+ patches already applied since 5.10.0 to what will become 5.10.1,
and it might occur to you that for some people, the performance
regression you are so fond of is much less important than one or a
dozen or a hundred others that have already been applied, or maybe
even some that haven't been submitted yet.

One might become aware while tracking maint-5.10 that it is by
definition the development stream leading toward the next maintenance
release of the 5.10.x branch. Anyone who does any planning will have
been making plans based on that, whether that's packagers or just
people who want to see how their modules will work with the next
release. That is one of the public commitments I was referring to.

Eventually actual knowledge of Perl development processes might lead
you to some understanding of the relationships among branches, such as
the fact that many of the patches currently in the maint-5.10 branch
destined for 5.10.1 have also been pulled back to the 5.8.x branch and
released in 5.8.9. Which means that even if it were possible to
release 5.10.0 + 1 regression fix and call it 5.10.1, you would fix
one regression but potentially cause dozens or hundreds of others for
anyone upgrading from 5.8.9 to this hypothetical 5.10.1. I don't
think upgrading should cause you to lose bug fixes you previously had
-- that kind of negates the meaning of the word "upgrade."

> By further implication, anyone not already
> participating in the current process is not serious and does not care enough
> about 5.10.1.

Newcomers who have something to contribute are welcome, but there's no
greater insult to a newcomer than sending them off on some wild goose
chase that just wastes everyone's time. The pumpking has recently
reported status and confirmed plans, so that really settles what's
going to happen. If anyone wants to help, it's not especially hard to
find the many suggestions that have been made for how to do so.

> Every day that Perl 5.10 is the most recent, most modern Perl release
> *increases* the number of people affected by the performance regression.
>
> Every day that Perl 5.10 is the most recent, most modern Perl release (which
> includes no warning that the smartmatch semantics will eventually change in
> incompatible ways) further entrenches that broken behavior and ensures that an
> eventual backwards-incompatible release will break *even more* code.
>
> A *new* volunteer has already done the *very modest* work required to produce
> a 5.10.1 which can address both of these problems *right now*.  5.10.1 could
> be out in a week.

To his credit, Jonathan did not go as far as claiming to have produced
a new release, and I don't think you should put those words in his
mouth. He asked a serious question and got some serious answers. I
certainly hope he sticks around. You should know better than to think
that applying one patch is the same thing as producing a release.
Assumptions that some hypothetical quick, easy release could be done
without distracting from and further delaying the real 5.10.1 are just
wishful thinking.

> Which public commitments would releasing 5.10.1 with the perldelta warning and
> the performance improvement violate?

I've mentioned some already. You may have heard there is a list of
bugs that need looking at. The list has been posted numerous times.
Do you really need to see the URL again, or was that just a rhetorical
question?

David Nicol

unread,
Jun 22, 2009, 12:32:28 AM6/22/09
to perl5-...@perl.org
On Sun, Jun 21, 2009 at 11:02 AM, Rafael
Garcia-Suarez<rgarci...@gmail.com> wrote:
> So, what we seem to need is, actually, a way to quickly bless,
> distribute and advertise a small number of important patches.

At the risk of tainting the concept by agreeing with it, "so start
publishing patches" is my take on the discussion as well.

Many community-driven projects (well, at least the ones by Daniel
Bernstein that he now touches as little as possible; also the Linux
kernel) have a culture of releases augmented by a smorgasbord of
patches that offer some feature or other.

(Refraining from suggesting a bunch of steps I have no intention of
doing myself at this point.)

--
1996 IBM Model M, now with permanent marker dots on many keys. You?

Gabor Szabo

unread,
Jun 22, 2009, 1:14:44 AM6/22/09
to Perl 5 Porters
On Mon, Jun 22, 2009 at 7:32 AM, David Nicol<david...@gmail.com> wrote:
> On Sun, Jun 21, 2009 at 11:02 AM, Rafael
> Garcia-Suarez<rgarci...@gmail.com> wrote:
>> So, what we seem to need is, actually, a way to quickly bless,
>> distribute and advertise a small number of important patches.
>
> At the risk of tainting the concept by agreeing with it, "so start
> publishing patches" is my take on the discussion as well.
>
> Many community-driven projects (well, at least the ones by Daniel
> Bernstein that he now touches as little as possible; also the Linux
> kernel) have  a culture of releases augmented by a smorgasbord of
> patches that offer some feature or other.

Wait, why would releasing a patch be less of a work or less of a responsibility
than releasing a version including that patch?

Would that be less serious than a release?
Would that not require exactly the same amount of manual work?

Shouldn't that manual work be almost 0 for a group of people who
pride themselves knowing a language used in almost all the CM
systems for build and release automation?

Gabor

Aristotle Pagaltzis

unread,
Jun 23, 2009, 3:03:49 PM6/23/09
to perl5-...@perl.org
* Craig A. Berry <craig....@gmail.com> [2009-06-22 06:23]:

> Eventually actual knowledge of Perl development processes might
> lead you to some understanding of the relationships among
> branches, such as the fact that many of the patches currently
> in the maint-5.10 branch destined for 5.10.1 have also been
> pulled back to the 5.8.x branch and released in 5.8.9. Which
> means that even if it were possible to release 5.10.0 + 1
> regression fix and call it 5.10.1, you would fix one regression
> but potentially cause dozens or hundreds of others for anyone
> upgrading from 5.8.9 to this hypothetical 5.10.1. I don't think
> upgrading should cause you to lose bug fixes you previously had
> -- that kind of negates the meaning of the word "upgrade."

I cannot follow. Until such time as any 5.10.1 is released, the
upgrade path from 5.8.9 is 5.10.0. Surely this upgrade path also
causes loss of bug fixes that 5.8.9 contains? So why would
someone who has upgraded to 5.8.9 but is holding off on 5.10.0
not have the option of holding off on 5.10.1 also? If there is no
reason, then why would these users be of any concern to when
determining the admissible state of a 5.10.1 release? Surely the
users who are currently using unpatched 5.10.0 perls are of much
greater concern to that question?

Regards,
--
Aristotle Pagaltzis // <http://plasmasturm.org/>

Craig A. Berry

unread,
Jun 23, 2009, 4:00:28 PM6/23/09
to perl5-...@perl.org
On Tue, Jun 23, 2009 at 2:03 PM, Aristotle Pagaltzis<paga...@gmx.de> wrote:
> * Craig A. Berry <craig....@gmail.com> [2009-06-22 06:23]:
>> even if it were possible to release 5.10.0 + 1
>> regression fix and call it 5.10.1, you would fix one regression
>> but potentially cause dozens or hundreds of others for anyone
>> upgrading from 5.8.9 to this hypothetical 5.10.1. I don't think
>> upgrading should cause you to lose bug fixes you previously had
>> -- that kind of negates the meaning of the word "upgrade."
>
> I cannot follow. Until such time as any 5.10.1 is released, the
> upgrade path from 5.8.9 is 5.10.0. Surely this upgrade path also
> causes loss of bug fixes that 5.8.9 contains? So why would
> someone who has upgraded to 5.8.9 but is holding off on 5.10.0
> not have the option of holding off on 5.10.1 also? If there is no
> reason, then why would these users be of any concern to when
> determining the admissible state of a 5.10.1 release? Surely the
> users who are currently using unpatched 5.10.0 perls are of much
> greater concern to that question?

You're right to point out that there is already a less than ideal
situation for someone on 5.8.9 wanting to upgrade to 5.10.x. But at
least 5.8.9 came out *after* 5.10.0, so it doesn't utterly defy
expectation that it would have some things that were not in a 5.10.x
release yet. But if 5.10.1 were to come out now without many of the
optimizations and bug fixes in 5.8.9, I think that would really
confuse people. Even if they don't follow development closely enough
to regard what's merged into the maint-5.10 branch as an indication of
what's going to be in the next 5.10.x release, they would be rightly
disappointed if the newest version in the current branch were in some
ways an upgrade and other ways a downgrade compared to the older
branch.

David Nicol

unread,
Jun 23, 2009, 9:17:47 PM6/23/09
to perl5-...@perl.org
On Tue, Jun 23, 2009 at 3:00 PM, Craig A. Berry<craig....@gmail.com> wrote:
> You're right to point out that there is already a less than ideal
> situation for someone on 5.8.9 wanting to upgrade to 5.10.x.  But at
> least 5.8.9 came out *after* 5.10.0, so it doesn't utterly defy
> expectation that it would have some things that were not in a 5.10.x
> release yet.  But if 5.10.1 were to come out now without many of the
> optimizations and bug fixes in 5.8.9, I think that would really
> confuse people. Even if they don't follow development closely enough
> to regard what's merged into the maint-5.10 branch as an indication of
> what's going to be in the next 5.10.x release, they would be rightly
> disappointed if the newest version in the current branch were in some
> ways an upgrade and other ways a downgrade compared to the older
> branch.

Therefore, the correct way to release the improvements is in the form
of patches, in some kind of nifty patch applicability matrix. The
target audience of the release is distribution packagers, who should
not be in the least cowed by patches, or by git for that matter.

Having the p5p release lag releases from debian, redhat, activestate,
gentoo, etc makes perfect sense and could drive better testing of
release candidates.

David E. Wheeler

unread,
Jun 23, 2009, 10:27:40 PM6/23/09
to David Nicol, perl5-...@perl.org
On Jun 23, 2009, at 6:17 PM, David Nicol wrote:

> Therefore, the correct way to release the improvements is in the form
> of patches, in some kind of nifty patch applicability matrix. The
> target audience of the release is distribution packagers, who should
> not be in the least cowed by patches, or by git for that matter.

That leaves out those of us who detest packaging systems, and hate
having to maintain scripts that apply patches, constantly having to
decide what patches we need and what we don't for production.

> Having the p5p release lag releases from debian, redhat, activestate,
> gentoo, etc makes perfect sense and could drive better testing of
> release candidates.

I think that core Perl should be released far more often than it is.
If something breaks, people go back to the previous version and wait
for the next release in, say, just six weeks.

Best,

David

David Golden

unread,
Jun 23, 2009, 11:39:21 PM6/23/09
to p5p
On Tue, Jun 23, 2009 at 10:27 PM, David E. Wheeler<da...@kineticode.com> wrote:
> That leaves out those of us who detest packaging systems, and hate having to
> maintain scripts that apply patches, constantly having to decide what
> patches we need and what we don't for production.

Git being what it is, I almost hate to use the linux kernel as an
example, but there are some useful parallels.

Git is distributed -- anyone can maintain a git repository with
branches that make sense to them. Anyone can choose to build a perl
from anyone's git branch.

For example, in my git repo, I have "maint-5.6.2, maint-5.8.0, etc."
that take the tag and cherry pick the fixes to make the old perls
build on a modern linux system. They're not back in the main perl
repo -- I think someone objected to what I was proposing, but so what?
I'm not interested in fighting that battle. They're in my repo, they
work for my needs *AND* for anyone else who has the same problem.

So in my opinion, we need a way for people to publicize useful
branches they maintain and we need an easy way for an admin of average
skill to easily build and install a perl from an arbitrary git branch.

-- David

David E. Wheeler

unread,
Jun 24, 2009, 1:08:39 AM6/24/09
to David Golden, p5p
On Jun 23, 2009, at 8:39 PM, David Golden wrote:

> So in my opinion, we need a way for people to publicize useful
> branches they maintain and we need an easy way for an admin of average
> skill to easily build and install a perl from an arbitrary git branch.

This may well fracture Perl. I'm already looking to go with Perl5i
when Schwern & Co. decide to "release." Do you want more of those?

Best,

David

Chromatic

unread,
Jun 24, 2009, 1:17:20 AM6/24/09
to perl5-...@perl.org, David Golden
On Tuesday 23 June 2009 20:39:21 David Golden wrote:

> So in my opinion, we need a way for people to publicize useful
> branches they maintain and we need an easy way for an admin of average
> skill to easily build and install a perl from an arbitrary git branch.

That sounds like an affordance for an admin of average skill (is that code for
"doesn't follow p5p closely"?) to believe that a big wad of code vetted for
stability, utility, and correctness even less often than bleadperl has the
patina of support, or at least official tolerence.

The existence of people willing to tolerate those kinds of antics to get
useful features and necessary patches indicates something very important about
the release process... especially considering the kind of abuse Red Hat
received from p5p for pulling a patch which turns out to have caused a
performance problem in a small but measurable subset of significant programs.

(Careful readers have noted more than a hint of irony in the previous
sentence.)

-- c

Rafael Garcia-Suarez

unread,
Jun 24, 2009, 2:12:10 AM6/24/09
to David E. Wheeler, perl5-...@perl.org
2009/6/24 David E. Wheeler <da...@kineticode.com>:

> On Jun 23, 2009, at 6:17 PM, David Nicol wrote:
>
>> Therefore, the correct way to release the improvements is in the form
>> of patches, in some kind of nifty patch applicability matrix.  The
>> target audience of the release is distribution packagers, who should
>> not be in the least cowed by patches, or by git for that matter.
>
> That leaves out those of us who detest packaging systems, and hate having to
> maintain scripts that apply patches, constantly having to decide what
> patches we need and what we don't for production.

To me, detesting packaging systems is a bit like detesting version
control systems... and I know the guts of rpm enough to know how
hateful they can be.

>> Having the p5p release lag releases from debian, redhat, activestate,
>> gentoo, etc makes perfect sense and could drive better testing of
>> release candidates.
>
> I think that core Perl should be released far more often than it is. If
> something breaks, people go back to the previous version and wait for the
> next release in, say, just six weeks.

Going back is not easy, unless you never install any modules, and
don't use any program that embeds perl or that is linked with
libperl.so (like mod_perl, or things like the vim editor on my ubuntu
system). Perl is actually pretty low on the dependency chain.

David E. Wheeler

unread,
Jun 24, 2009, 2:35:19 AM6/24/09
to Rafael Garcia-Suarez, perl5-...@perl.org
On Jun 23, 2009, at 11:12 PM, Rafael Garcia-Suarez wrote:

> To me, detesting packaging systems is a bit like detesting version
> control systems... and I know the guts of rpm enough to know how
> hateful they can be.

I'm not talking about the source code for packaging systems; I likely
know less than you do. What I know is that they are always behind the
release curve, often integrate unreleased patches (!), and don't use
stuff I've compiled and installed myself as dependencies. And the time
they save me compared to compiling is negligible compared with the
time I typically spend configuring the software to work once it's
installed.

> Going back is not easy, unless you never install any modules, and
> don't use any program that embeds perl or that is linked with
> libperl.so (like mod_perl, or things like the vim editor on my ubuntu
> system). Perl is actually pretty low on the dependency chain.

Only a fool would upgrade a production box to a new release and
completely replace the existing version in the process. One typically
installs it in a completely separate root, and if it fails, you delete
that version without any effect on the installed version. Any admin of
average skill can do this, and should.

Best,

David

Rafael Garcia-Suarez

unread,
Jun 24, 2009, 5:17:27 AM6/24/09
to Ovid, chromatic, perl5-...@perl.org, David Golden
2009/6/24 Ovid <publiust...@yahoo.com>:
>
> I want to encourage people to experiment.  Perl really, really needs some
> help and barring Perl 6 being released and adopted tomorrow, Perl 5 is the
> pony to bet on right now.  Rafael holds the keys to the main branch of Perl
> right now and he's taking a conservative approach, as is his right.  Thus, I
> don't see anything changing there.  We need a good, stable fork where we can
> try to make Perl a modern language.  Maybe's it's Schwern's fork.  Maybe it's
> Kurila (I honestly don't know as I haven't paid enough attention).

Seriously, I don't see the point of forking. You'd want to fragment even more
the little resources we have for Perl 5 ? If someone sees me as an impediment
and wants the patch pumpkin, then ask.

--
The truth is that we live out our lives putting off all that can be put off;
perhaps we all know deep down that we are immortal and that sooner or later
all men will do and know all things.
-- Borges

Nicholas Clark

unread,
Jun 24, 2009, 5:17:01 AM6/24/09