sustaining Perl 5 maintenance
Nicholas Clark
to Léon saying to me something close to:
And there are enough are working on Perl 5?
and my answer was plain
No
Specifically, we have a problem. I may seem to have been going on about
this for years (maybe 2 or 3) but that doesn't make me feel that I'm wrong.
Specifically, maintaining Perl 5 is work, not fun.
We don't have enough people doing it currently, and everyone I've asked in
the past year or two if they wanted core commit has politely declined.
"Patches welcome" isn't helping simply because there aren't enough of us
reviewing and applying/correcting/rejecting-with-feedback the patches we
get. Right now I think it's mainly Rafael doing it.
TPF grants won't help cure this. Dave Rolsky has a hole in one:
My big reason for not wanting a grant as they stand is that it would
provide pressure to do work without providing any extra time for that
work.
http://use.perl.org/comments.pl?sid=38694&cid=61289
and
I don't want one. It's not enough money for the hassle, and it would
just cause me more stress to take the grant. That would suck the fun
out of doing the work.
http://use.perl.org/comments.pl?sid=38694&cid=61256
and clearly our summariser has the same issue that money does not create
time:
(I'm way behind on sumamrising)
http://www.xray.mpe.mpg.de/mailing-lists/perl5-porters/2008-02/msg00800.html
I'm not certain what the steady state workload is to stand still (ignore
new development), but let's try to work it out. We get just over 10 bugs a
week:
$ grep '^Created this week' 2007-??.mbox 2008-01.mbox | perl -lnwe '$w++; /: (\d+)/; $b += $1; END {print "$b $w ", $b/$w}'
504 49 10.2857142857143
So, say 10.5 a week, and 20 minutes on average just to assess, answer,
reply to the bug, that's
*** 3.5 hours a week.
Bugs can take anything from 0 hours to a number of days to fix. Even if bug
reports come with patches, so in theory it's 0, in practice people send
patches that are mangled, or that don't pass their own tests, or, sometimes
don't even compile. It's all work, and all mundane work.
So, if on average a bug takes half a day to deal with
*** 5.5 days a week
Rafael estimates that patches (not sure if that included bugs - most
patches aren't in RT) steady state would take a day a week to deal with
*** 1 day a week
I estimate that keeping on top of blead->maint integrations takes about a
day a week on average
*** 1 day a week
which has just got us to 8 (working) days a week TO STAND STILL. No new
development there.
I think we need help. Full time help. I think we need more than volunteers,
and we need more than TPF (as is, with the current grant scheme) can
provide.
Specifically, I feel that we need a full time junior whose job it is to
keep on top of bug reports, create (best case) fixes or (worst case) TODO
tests, validate patches compile and pass tests, and find the simple things
rewarding because there are still a learning experience. This poses a
problem, as Richard Clamp commented before I even mentioned it - a junior
would need access to a senior to mentor. (As well as a manager, which need
not be the same person). I'd hope that we can address this by having
enthusiastic buy-in from many of the current committers (and other
experienced people on here and on IRC), but it would probably work best if
such a person were co-located within an existing firm using Perl, as their
form of sponsorship.
Note, this is not me looking for a job doing this, as I've tried it (that
TPF grant in 2006) and didn't find it at all fun or rewarding.
I'm specifically not talking about starting a support business to compete
with ActiveState. It's a free market - there's nothing stopping anyone else
doing this, and the fact that no-one has makes me wonder if the marketplace
is not of the size where it's viable for two to survive. It scares me a lot
to try to offer "priority support" on the basis of an ad hoc group of
semi-volunteers, because it's really hard to see how one can offer any
guarantee of timeliness in bug fixing. Particularly there is the crunch
dilemma of does one offer a quick "work around" to get the client up and
running again, and then find oneself stuck supporting it for ages.
I feel that attempting to do this would digresses us from achieving
something here, as I can't see in practise what it offers that ActiveState
doesn't already.
I'm thinking instead of having a plan in place such that we can give a good
reason for firms and individuals to donate (via a specific TPF fund), much
like was done with great success in 2001. It's a non-commercial thing -
your benefits are at worst "feel good", at best, increased visibility that
may well help your recruiting, without any formal SLA, or contractual
expectation. We're really not in a position to offer any sort of
commercial-grade support service.
Also, I'm pretty sure that there isn't really work for much more than one
full time person, so trying to create a business with a whole team isn't
going to work, unless the business development person manages to also bring
in a whole raft of new bugs. :-/
So I'm proposing that we work out what it would take to hire a full time
"bug wrangler" - someone to deal with all the bugs, at least as first line,
it frees up the more skilled volunteers to deal with assessing
* escalated bugs
* proposed fixes to bugs from the bug wrangler
* proposed fixes to bugs supplied with the bug report
* third party patches
The bug wrangler certainly doesn't have a monopoly on applying (or rejecting)
patches, so I just don't see their lack of years of core experience as an
issue. In fact, they can probably start out without even having commit rights
(although the move to git would let them have full commit on a staging branch
that it would be trivial to merge through to blead)
This isn't a job suitable for a summer intern - bugs come 12 months a year,
and the sort of person I propose we recruit (smart but reasonably
inexperienced) is likely to take a couple of months of mentoring investment
before they are independently useful - investment which would only go to
waste on an intern who leaves.
I don't see it as a job for volunteers either - however hard we try on
recruiting, I can't see us getting enough volunteers to generate 40 hours
labour a week, and multiple volunteers are going to be less efficient than
one continuous worker. Multiple people have overhead in communication;
people who only have a few hours of an evening free can't usefully start
work on a bug that might take more than that evening to complete - do they
then try to hand it over to someone else, or does it drop until the next
evening they have free, maybe a week later, by which time they've forgotten
it all?
Yes, better tools will help with managing patches, and ongoing development,
d but that isn't the problem I'm talking about addressing here. It's the
daily grind of answering bug reports - reading e-mail, evaluating and
understanding the problem, triaging, and replying as appropriate.
Of course, I don't object to new tools and more volunteers. And I'd love to
be proved wrong and have everything drastically improve. But I just don't
see that future panning out.
Nicholas Clark
Andy Armstrong
> Of course, I don't object to new tools and more volunteers. And I'd
> love to
> be proved wrong and have everything drastically improve. But I just
> don't
> see that future panning out.
The thing that's stopped me getting more actively involved is
uncertainty about whether I can make the time to learn enough of core
to be a useful contributor.
From my external (possibly fantasy) perspective: I'm fine with C and
Perl and imagine that if I knew core better I could usefully
contribute by picking off, say, a bug a week. But I don't know core at
all well and would worry that one bug a week wouldn't get me
sufficiently engaged to develop the depth of knowledge I imagine one
would need to make efficient progress on that bug.
I think that's a component of the problem - it's a big lump of code to
get up to speed on. If you intersect the sets 'perl-fu', 'c-fu' and
'has time to learn core' you probably end up with a pretty small list
of candidates.
Maybe I should a) get parallel testing working and b) start picking
off bugs and find out what that feels like in practice :)
--
Andy Armstrong, Hexten
Nicholas Clark
> On 15 Apr 2008, at 21:14, Nicholas Clark wrote:
> >Of course, I don't object to new tools and more volunteers. And I'd
> >love to
> >be proved wrong and have everything drastically improve. But I just
> >don't
> >see that future panning out.
>
>
> The thing that's stopped me getting more actively involved is
> uncertainty about whether I can make the time to learn enough of core
> to be a useful contributor.
Says the man who already did dtrace.
> From my external (possibly fantasy) perspective: I'm fine with C and
> Perl and imagine that if I knew core better I could usefully
> contribute by picking off, say, a bug a week. But I don't know core at
> all well and would worry that one bug a week wouldn't get me
> sufficiently engaged to develop the depth of knowledge I imagine one
> would need to make efficient progress on that bug.
Possibly true, but there are also plenty of suggestions in perltodo:
http://mirrors.develooper.com/perl/APC/perl-current/pod/perltodo.pod
which are grouped as
=head1 Tasks that only need Perl knowledge
=head1 Tasks that need a little sysadmin-type knowledge
=head1 Tasks that need a little C knowledge
=head1 Tasks that need a knowledge of XS
=head1 Tasks that need a knowledge of the interpreter
=head1 Big projects
> I think that's a component of the problem - it's a big lump of code to
> get up to speed on. If you intersect the sets 'perl-fu', 'c-fu' and
> 'has time to learn core' you probably end up with a pretty small list
> of candidates.
People don't need c-fu, or time to learn the core. There are bugs outstanding
on pure-perl modules like File:::Find and the Pod processors.
However, if you want to get an idea of what it would take to get the bit
beyond Perl-fu and C-fu, I'm probably not the best person to ask. Steve
Peters springs to mind, as he did exactly this quite recently.
> Maybe I should a) get parallel testing working and b) start picking
> off bugs and find out what that feels like in practice :)
Yes please. Much that I want it, I've not even started on trying to implement
parallel testing, for the exact mirror of your concerns about learning the
core - I doubted that it was a good use of my time learning the nuances of
the guts of Test::Harness 3.
Nicholas Clark
Nicholas Clark
> On 15 Apr 2008, at 21:43, Nicholas Clark wrote:
> >http://mirrors.develooper.com/perl/APC/perl-current/pod/perltodo.pod
> >
> >which are grouped as
> >
> >=head1 Tasks that only need Perl knowledge
> >=head1 Tasks that need a little sysadmin-type knowledge
> >=head1 Tasks that need a little C knowledge
> >=head1 Tasks that need a knowledge of XS
> >=head1 Tasks that need a knowledge of the interpreter
> >=head1 Big projects
>
It's easy for me. I WTFM (well, strictly, the most recent re-write, including
the re-ordering to achieve that grouping)
It's also why I'm really pleased that David is featuring a random TODO
inline in the summary each week. It's much more likely to get read than
just a link to a file somewhere else.
Although so far (2 weeks?) it has only generated one clarification question,
and no code. But that's more than we had before.
Nicholas Clark
Andy Armstrong
> http://mirrors.develooper.com/perl/APC/perl-current/pod/perltodo.pod
>
> which are grouped as
>
> =head1 Tasks that only need Perl knowledge
> =head1 Tasks that need a little sysadmin-type knowledge
> =head1 Tasks that need a little C knowledge
> =head1 Tasks that need a knowledge of XS
> =head1 Tasks that need a knowledge of the interpreter
> =head1 Big projects
Ah. I really must get more in the habit of RTFM. Thanks Nick :)
>> Maybe I should a) get parallel testing working and b) start picking
>> off bugs and find out what that feels like in practice :)
>
> Yes please. Much that I want it, I've not even started on trying to
> implement
> parallel testing, for the exact mirror of your concerns about
> learning the
> core - I doubted that it was a good use of my time learning the
> nuances of
> the guts of Test::Harness 3.
Thanks to Oslo I know how I'm going to do it now. All that remains is
a bit of typing.
--
Andy Armstrong, Hexten
David Landgren
> It's easy for me. I WTFM (well, strictly, the most recent re-write, including
> the re-ordering to achieve that grouping)
>
> It's also why I'm really pleased that David is featuring a random TODO
> inline in the summary each week. It's much more likely to get read than
> just a link to a file somewhere else.
Yes, I finally managed to credit you for the idea this week.
> Although so far (2 weeks?) it has only generated one clarification question,
> and no code. But that's more than we had before.
3 so far, not counting this week's summary just out the door.
David
Eric Wilhelm
# on Tuesday 15 April 2008 13:14:
>Also, I'm pretty sure that there isn't really work for much more than
> one full time person, so trying to create a business with a whole
> team isn't going to work, unless the business development person
> manages to also bring in a whole raft of new bugs. :-/
Well, a for-profit business offering priority support doesn't have to
fix bugs to make money -- and it is most likely not their primary
motivator. If bugs in perl don't suddenly become *much* more painful
to the users (you know, the sort of pain you pay dearly to get rid of),
I think it will be hard to make it fly as anything other than a
non-profit. (Barring enough momentum and idealism -- but that's not a
startup.)
Or, to put it another way: from your analysis, simply *answering* each
incoming bug costs something like $150 (assuming that your bug
secretary gets ~$62k/yr.)
Now, if we could only get a nickel per week out of our 25,000 biggest
users (without spending three nickels to deliver the cash...)
--Eric
--
"It is a mistake to allow any mechanical object to realize that you are
in a hurry."
--Ralph's Observation
---------------------------------------------------
http://scratchcomputing.com
---------------------------------------------------
Richard Dice
Thanks very much for your work putting together this email.
You and I have talked about these things before so it's no surprise
for me to hear this from you and probably not much of my reply is
going to be a surprise either. But it's good to share this stuff more
broadly so I'm glad this is out in the open.
I agree that small grants are not going to cut it, for the reasons
stated. Small amounts of money do not create time. Large amounts of
money do, because then it becomes the kind of paid work that people
can arrange their lives around.
I would like for TPF to be able to provide this level of funding to
one or more people, to work full time on p5 core. As you pointed out,
one person full time is sufficient to stand still. (But stand still
competently and comfortably.) More than one is what is needed for
moving forward.
Note that my desire for TPF to be able to do this isn't specifically
because I think TPF is the only way to do it, but merely as an
observation on that no-one else currently is and it is fully
consistent with our mission. If someone else did it then I would be
wonderfully happy.
My major disagreement is merely one of tactics. I'm not convinced
that TPF would be able to raise sufficient donations to make this
happen. Philanthropy from individuals tends to the small amounts, and
the amounts needed for this would not be small. Philanthropy from
companies is notoriously hard to get. And as you (correctly) point
out, it could be difficult to sell a differentiated product and/or
service based simply on p5 core development, otherwise ActiveState
would be in that game.
This leaves two things -- grants from major charitable institutions,
and strategic investments from corporations. Either way, making a
pitch to either of these kinds of organizations is going to consume
serious resources, equivalent to... a full-time job. It's called
business development, and it's generally an executive-level position.
So, we've just succeeded in talking ourselves from problem A into
problem B, both of which are similarly insolvable. :-)
So...
I am attempting to bootstrap the business development capabilities of
TPF. These thoughts have been on my mind, growing gradually more and
more formed for years now. I have been meaning to write them up into
official position-paper type things to put on the TPF web site. But
to do so properly would require a weekend of work, I'm I've got just
as few round tuits as anyone else. (I'm already putting in so much
time on TPF stuff that my wife is Seriously Unhappy with me.)
But this at least is a taste of where my thinking is at.
Cheers,
- Richard
David Nicol
project teams
and would like to be included in the p5p-as-business discussion
(Well what's stopping you? You are a famously brash loudmouth.)
Ah. So I am. I see two problems, one is income model, the other is
fair distribution. Recent inclusion of how-long-it-took-me reports on p5p
indicates that tracking of one "grain" for each "hour" put to effort
approved by the approval governance committee, which is what I've been thinking
about setting up, could reasonably be tested on p5p. So when income comes
in, it goes out divided among the grains.
That still doesn't address the income model issue though. Is the bottom line
from perlmonks' banner ads and donation acceptance program public knowledge?
Styling the organization of participants as a consulting company that does
contract perl work (Authors' Guild, anyone?) and considers paying into the
maintenance pool part of the work, or even participants receive grains for
their work for the clients, and grains provide residual income into the future,
may be a little pyramidish but not really. The income model would be
licensing a certification, which could be relied on to signify value worth the
premium paid to hire contractors from participating consulting shops.
Do we have a p5p-biz list to move the discussion to?
Yitzchak Scott-Thoennes
> That still doesn't address the income model issue though. Is the bottom
> line from perlmonks' banner ads and donation acceptance program public
> knowledge?
Pair provides perlmonks 3 servers and bandwidth at no cost.
The pair banner ads are in recognition of this; they don't
generate any revenue. (Though there may have been some
exploration of banner ad revenue in the distant past.)
Nicholas Clark
> I've been thinking a lot about contractual frameworks for distributed
> project teams
> and would like to be included in the p5p-as-business discussion
You don't appear to have read my message.
On Tue, Apr 15, 2008 at 09:14:44PM +0100, Nicholas Clark wrote:
> I'm specifically not talking about starting a support business to compete
> with ActiveState. It's a free market - there's nothing stopping anyone else
> doing this
> I'm thinking instead of having a plan in place such that we can give a good
> reason for firms and individuals to donate (via a specific TPF fund), much
> like was done with great success in 2001. It's a non-commercial thing -
^^^^^^^^^^^^^^^^^^^^
> your benefits are at worst "feel good", at best, increased visibility that
> may well help your recruiting, without any formal SLA, or contractual
> expectation. We're really not in a position to offer any sort of
> commercial-grade support service.
On Tue, Apr 15, 2008 at 09:07:28PM -0500, David Nicol wrote:
> Do we have a p5p-biz list to move the discussion to?
Please start one and take your "as a business" discussion there.
Meanwhile, here, feedback on the the actual contents of my message is on
topic.
Nicholas Clark
Nicholas Clark
> # from Nicholas Clark
> # on Tuesday 15 April 2008 13:14:
>
> >Also, I'm pretty sure that there isn't really work for much more than
> > one full time person, so trying to create a business with a whole
> > team isn't going to work, unless the business development person
> > manages to also bring in a whole raft of new bugs. :-/
>
> Well, a for-profit business offering priority support doesn't have to
> fix bugs to make money -- and it is most likely not their primary
> motivator. If bugs in perl don't suddenly become *much* more painful
> to the users (you know, the sort of pain you pay dearly to get rid of),
> I think it will be hard to make it fly as anything other than a
> non-profit. (Barring enough momentum and idealism -- but that's not a
> startup.)
Good point, which has come up in private conversations, but I failed to
correlate with this - Perl simply isn't broken enough. Most things work too
well, hence no-one finds that they need to fix their itch, so in turn, they
don't get sucked into core development generally.
Maybe we need to start adding bugs, somewhat like a protection racket.
"Your program works very nicely. It would be a shame if something went
wrong with it, wouldn't it? ..."
> Or, to put it another way: from your analysis, simply *answering* each
> incoming bug costs something like $150 (assuming that your bug
> secretary gets ~$62k/yr.)
Thanks for doing the maths. Yes, about $60K was the hypothetical salary in
a conversation with Richard Dice.
Maybe we should call this "the sixty four thousand dollar plan"
But yes, I guess it means that each and every bug anyone submits has a true
cost of over $100 to answer. I know how long some of my bug reports (and
fixes) have taken, and I think the submitter would be be shocked at the figure
if I were in a position (and had a desire) to invoice them.
> Now, if we could only get a nickel per week out of our 25,000 biggest
> users (without spending three nickels to deliver the cash...)
Yes. Exactly. But how?
Getting $10,000ish out of our biggest 6 bug-hitting corporate users would
seem easier. But, as Richard Dice observes, still not easy.
And this isn't $64K for new features. This is $64K just for bug reports.
Given the enthusiasm we see for
http://mirrors.develooper.com/perl/APC/perl-current/pod/perltodo.pod
(either adding to it with "this would be useful because..." or
"here's a patch" or even "I see that this is in TODO. Is it likely to be
done soon, because ...")
I have trouble envisaging a viable business for pay-for-improvements to Perl
After all, if a market exists, ActiveState are already in a prime position to
tap it. They have the skilled staff. And if anyone here wants something done,
I could point them at a couple of (other) competent people who could do it.
Nicholas Clark
Nicholas Clark
> I agree that small grants are not going to cut it, for the reasons
> stated. Small amounts of money do not create time. Large amounts of
> money do, because then it becomes the kind of paid work that people
> can arrange their lives around.
Yes, this is key.
> I would like for TPF to be able to provide this level of funding to
> one or more people, to work full time on p5 core. As you pointed out,
> one person full time is sufficient to stand still. (But stand still
> competently and comfortably.) More than one is what is needed for
> moving forward.
I agree, but I feel that if we can get just one person full time it will help
a lot. As is, with zero people full time, our outstanding bug count is only
going backwards slowly. We still have volunteers, and by allowing them to
concentrate more on non-trivial (and non RTFM) bugs, and bigger picture
improvements, we will more forward.
> Note that my desire for TPF to be able to do this isn't specifically
> because I think TPF is the only way to do it, but merely as an
> observation on that no-one else currently is and it is fully
> consistent with our mission. If someone else did it then I would be
> wonderfully happy.
Yes, me too. I really don't mind where the money comes from, and in fact I
don't even care if it is money. If some wonderful philanthropic (or self-
interested*) firm chose to donate a member of staff's time, that would work
just as well.
> My major disagreement is merely one of tactics. I'm not convinced
> So, we've just succeeded in talking ourselves from problem A into
> problem B, both of which are similarly insolvable. :-)
Well, possibly. I'm no expert in problem B, and wasn't proposing to become
one.
But right now I'd welcome feedback on on whether this even is a solution for
problem A (sustaining maintenance) because I'm not omniscient, and there may
be flaws in my plan that I'd not thought of.
(and Richard has already given feedback on drafts, so everyone else - that
last paragraph was aimed at you)
Nicholas Clark
* It was noted by others on IRC that historically no Perl using firms seem to
feel that it is in their (commercial) interest to hire core perl developers
to work on the core, on the basis that they have sufficient hardware costs
that incremental software wins are worth it. It surprises me that *all*
firms think the same - some places use an awful lot of hardware. After all,
Google even found it worth it to go one step beyond and develop its own
hardware.
Bram
> inline in the summary each week. It's much more likely to get read than
> just a link to a file somewhere else.
>
> Although so far (2 weeks?) it has only generated one clarification
> question,
> and no code. But that's more than we had before.
What about also sending a weekly message to p5p with a random TODO? (or
perhaps with a random TODO from each group)
If it is in the weekly-summary of p5p then it is also easy to miss...
The main question/problem:
Who is more likely to read the (entire) summary? And who is more likely to
work on the TODO?
Someone that is reading the messages/is active on p5p or someone that is
only reading the summary (the perl5-summary list that is)?
I tend to read every message on p5p but I don't tend to fully read the
weekly-summary (I usually do take a quick look at the titles to see if
there are things I missed).
The TODO's in the summary went unnoticed by me...
So who else has missed them?
(Of course I now know that the summary includes a random TODO and now I
will look for it every week)
Kind regards,
Bram
H.Merijn Brand
> > It's also why I'm really pleased that David is featuring a random TODO
> > inline in the summary each week. It's much more likely to get read than
> > just a link to a file somewhere else.
> >
> > Although so far (2 weeks?) it has only generated one clarification
> > question,
> > and no code. But that's more than we had before.
>
> What about also sending a weekly message to p5p with a random TODO? (or
> perhaps with a random TODO from each group)
:)
> If it is in the weekly-summary of p5p then it is also easy to miss...
>
> The main question/problem:
>
> Who is more likely to read the (entire) summary?
I do, and I'm grateful to grinder and the funders for reviving the
summary!
> And who is more likely to work on the TODO?
> Someone that is reading the messages/is active on p5p or someone that is
> only reading the summary (the perl5-summary list that is)?
>
> I tend to read every message on p5p but I don't tend to fully read the
> weekly-summary (I usually do take a quick look at the titles to see if
> there are things I missed).
> The TODO's in the summary went unnoticed by me...
> So who else has missed them?
>
> (Of course I now know that the summary includes a random TODO and now I
> will look for it every week)
--
H.Merijn Brand Amsterdam Perl Mongers (http://amsterdam.pm.org/)
using & porting perl 5.6.2, 5.8.x, 5.10.x on HP-UX 10.20, 11.00, 11.11,
& 11.23, SuSE 10.1 & 10.2, AIX 5.2, and Cygwin. http://qa.perl.org
http://mirrors.develooper.com/hpux/ http://www.test-smoke.org
http://www.goldmark.org/jeff/stupid-disclaimers/
David Nicol
> On Tue, Apr 15, 2008 at 04:39:43PM -0400, Richard Dice wrote:
>
> > Note that my desire for TPF to be able to do this isn't specifically
> > because I think TPF is the only way to do it, but merely as an
> > observation on that no-one else currently is and it is fully
> > consistent with our mission. If someone else did it then I would be
> > wonderfully happy.
>
> Yes, me too. I really don't mind where the money comes from, and in fact I
> don't even care if it is money.
Please reply off-list to be added to a mailing list which will be created for
the purpose of exploring funding schemes. I will create it on receipt of two
subscription requests. Probably on google-groups.
Nicholas Clark
> Thanks for doing the maths. Yes, about $60K was the hypothetical salary in
> a conversation with Richard Dice.
>
> Maybe we should call this "the sixty four thousand dollar plan"
That was the guessed salary for a keen junior, for the plan I was suggesting
that no-one has yet said "mmm, plan!" or "will never fly on technical grounds
because..."
(yes, I'm not yet interested in "how do we get the money", given that in 2001
and 2002, just after the dot.com, TPF managed to raise large sums against
less constrained deliverables than this)
But if anyone competent (many patches already in core, if not commit rights)
wants to say "I can stay sane fielding bugs, not need managing, side-step
the whole recruitment process, but cost more" then we have an alternative
plan, with cost, which we could consider.
Nicholas Clark
Chromatic
> Good point, which has come up in private conversations, but I failed to
> correlate with this - Perl simply isn't broken enough. Most things work too
> well, hence no-one finds that they need to fix their itch, so in turn, they
> don't get sucked into core development generally.
> Maybe we need to start adding bugs, somewhat like a protection racket.
... or stop supporting I can't even count how many versions of Perl. How much
time does any given pumpking spend shuffling patches between blead, 5.10, and
5.8?
-- c
Nicholas Clark
> I estimate that keeping on top of blead->maint integrations takes about a
> day a week on average
>
> *** 1 day a week
1 day. In the original message.
Which is a lot less than the 5 days a week I estimate that bugs take.
This isn't going to help with the core problem.
Nicholas Clark
Eric Wilhelm
# on Wednesday 16 April 2008 09:27:
>> Maybe we should call this "the sixty four thousand dollar plan"
>
>That was the guessed salary for a keen junior, for the plan I was
> suggesting that no-one has yet said "mmm, plan!" or "will never fly
> on technical grounds because..."
I can't exactly speak to the technical grounds, but in what way can one
challenge the estimates and the hypothesis that a *suitable* employee
(or person taking-on employee-like responsibilities with employee-like
compensation) could handle the workload and improve the situation? On
technical grounds?
>But if anyone competent (many patches already in core, if not commit
> rights) wants to say "I can stay sane fielding bugs, not need
> managing, side-step the whole recruitment process, but cost more"
> then we have an alternative plan, with cost, which we could consider.
Yes, it really does depend (heavily) on the person. Perhaps the next
step is to put out some feelers (take nominations?) for candidates and
try to find pledges for the funds. Maybe a 3-month contract to start?
How would the interview process work? And, is the silence == approval?
Tels
On Wednesday 16 April 2008 09:37:55 Nicholas Clark wrote:
> On Tue, Apr 15, 2008 at 04:06:00PM -0700, Eric Wilhelm wrote:
> > # from Nicholas Clark
> >
> > # on Tuesday 15 April 2008 13:14:
> > >Also, I'm pretty sure that there isn't really work for much more
> > > than one full time person, so trying to create a business with a
> > > whole team isn't going to work, unless the business development
> > > person manages to also bring in a whole raft of new bugs. :-/
> >
> > Well, a for-profit business offering priority support doesn't have
> > to fix bugs to make money -- and it is most likely not their
> > primary motivator. If bugs in perl don't suddenly become *much*
> > more painful to the users (you know, the sort of pain you pay
> > dearly to get rid of), I think it will be hard to make it fly as
> > anything other than a non-profit. (Barring enough momentum and
> > idealism -- but that's not a startup.)
>
> Good point, which has come up in private
conversathttp://jappy.dataup.de/?id=18139ions, but I failed
> to correlate with this - Perl simply isn't broken enough. Most things
> work too well, hence no-one finds that they need to fix their itch,
> so in turn, they don't get sucked into core development generally.
>
> Maybe we need to start adding bugs, somewhat like a protection
> racket.
>
> "Your program works very nicely. It would be a shame if something
> went wrong with it, wouldn't it? ..."
>
> > Or, to put it another way: from your analysis, simply *answering*
> > each incoming bug costs something like $150 (assuming that your bug
> > secretary gets ~$62k/yr.)
>
> Thanks for doing the maths. Yes, about $60K was the hypothetical
> salary in a conversation with Richard Dice.
>
> Maybe we should call this "the sixty four thousand dollar plan"
>
> But yes, I guess it means that each and every bug anyone submits has
> a true cost of over $100 to answer. I know how long some of my bug
> reports (and fixes) have taken, and I think the submitter would be be
> shocked at the figure if I were in a position (and had a desire) to
> invoice them.
And that is just the cost to let the user file the bug report and
acknoledge we got it. FIxing it, OTOH, ....
All the best,
Tels
--
Signed on Wed Apr 16 19:33:37 2008 with key 0x93B84C15.
View my photo gallery: http://bloodgate.com/photos
PGP key on http://bloodgate.com/tels.asc or per email.
"People who are rather more than six feet tall and nearly as broad
across the shoulders often have uneventful journeys. People jump out
at them from behind rocks then say things like, 'Oh. Sorry. I thought
you were someone else.'"
-- Terry Pratchett
Dave Mitchell
> That was the guessed salary for a keen junior, for the plan I was suggesting
> that no-one has yet said "mmm, plan!" or "will never fly on technical grounds
> because..."
Okay, some thoughts about the junior post:
What I worry about is that because perl is so mature, a lot of the bugs
that get reported are subtle, and (a) are hard to diagnose, and (b) are
even harder to fix. For example, a reported "premature free" error may
turn out, after a day's hard debugging, to be just another symptom of
the fact that the stack isn't refcounted; and the fix is to alter PUSH/POP
macros in pp.h to do refcounting, then "simply" go, line-by-line through
the 24K lines of pp*.c, making sure that each op is fixed to correctly
handle refcounting under all circumstances (eg not to just abandon the top
N items on the stack). At which point of course, you jinstead just
chalk it up to experience and the bugs database grows by 1.
I would expect the junior not to be able to make many inroads against that
sort of bug. On the other hand, just having someone filter away the simple
bugs would still be useful, but whether it's still a full-time job, I
don't know.
An alternative would just to get lots of p5pers to each commit to doing
a couple of hours of bug reviewing / fixing per week, in a slightly more
organised fashion than at present, so that at least all bug reports get
answered, and the simple ones get processed. Whether this would work in
practice, I don't know; but from a personal psychological perspective, I
know I'd be more likely to do this if I had publicly committed to spending
two hours each tues, rather than just a vague good-will intent to look at
the bugs DB from time to time (and usually getting distracted in the
meantime).
[ Hmmmm: maybe I should publicly commit to doing maint-5.10 integrations
too ;-) ]
> But if anyone competent (many patches already in core, if not commit rights)
> wants to say "I can stay sane fielding bugs, not need managing, side-step
> the whole recruitment process, but cost more" then we have an alternative
> plan, with cost, which we could consider.
Well... I could probably answer yes to the list above, but only if the role
was part-time (eg 2 days per week).
--
Thank God I'm an atheist.....
Sven Dowideit
(the other 2-3 days) might be easier too.
I found trying to learn enough to add dtrace probes pretty daunting
(especially due to the pain of debugging all the nested preprocessor
non-functions), and very rewarding, only pausing it because I ran out of
time :/.
I recal noticing, but not having time to del with an offer from someone
to teach Perl core innards to anyone interested in helping maintain Perl
- perhaps all these things could come together?
Still looking to learn more - btw Git helps alot there, the rsync thing
was a really difficult work practice to learn.
Sven
--
Professional Wiki Innovation and Support
Sven Dowideit - http://DistributedINFORMATION.com
A WikiRing Partner http://wikiring.com
Nicholas Clark
> On Wed, Apr 16, 2008 at 05:27:31PM +0100, Nicholas Clark wrote:
> > That was the guessed salary for a keen junior, for the plan I was suggesting
> > that no-one has yet said "mmm, plan!" or "will never fly on technical grounds
> > because..."
> turn out, after a day's hard debugging, to be just another symptom of
^^^
> I would expect the junior not to be able to make many inroads against that
> sort of bug. On the other hand, just having someone filter away the simple
> bugs would still be useful, but whether it's still a full-time job, I
> don't know.
I'm not sure. However, as well as the 11 new bugs each week, there are 1800
outstanding bugs that can be looked at, which would be 36 per week for a year,
or 1 an hour. *That* would be somewhat soul destroying, but I'd be confident
that there's enough work there to fill up any slack time.
Also, all the core's documentation could be read, with code examples verified
as working (preferably by adding tests), and obvious insanities spotted, even
if it's only flagged up for someone more knowledgeable to fix, or someone with
better English to redraft.
We also have areas of the core without code coverage in the regression tests -
it would be useful for a junior to identify those, and do his/her best at
trying to figure out how to test them. A particular example is that I see that
there are a lot of "TODO"s in the warnings tests.
> An alternative would just to get lots of p5pers to each commit to doing
> a couple of hours of bug reviewing / fixing per week, in a slightly more
> organised fashion than at present, so that at least all bug reports get
> answered, and the simple ones get processed. Whether this would work in
> practice, I don't know; but from a personal psychological perspective, I
I don't think it would, for a few reasons. You've already stated one above -
"day's hard debugging". Any bug with a cause that couldn't be identified
within a couple of hours would go unanswered.
Also, a conversation last night reminded me of the experiences of a firm,
which are relevant:
Said firm, which might prefer to remain nameless, had its development team
in the head office in London, but ran websites for customers worldwide. It
decided that to better support its sales staff in New York, it would have
some of the developers work a late shift in London, from 2pm to 10pm, to
increase the overlap with New York, and give non-zero overlap with
(potential) customers on the Pacific Coast.
All well and good, but developers on late shift didn't spend most of their
time asking sales questions - in fact, they were really there in case, so
spent most of their time working on regular projects, including fixing bugs.
What often happened would be that a European customer found a bug, and it
would be assigned to a late shift developer when or soon after he/she
arrived at work. He/she would work on it throughout the evening, but would
not complete it at 10pm, so would leave to go home with it unresolved. The
account manager would come back in to work the next morning, would ask the
status, and then (because it was perceived to be urgent, and nearly fixed),
and a different (day-shift) programmer was assigned to carry on. However,
quite often, it took the new programmer a while to come up to speed on the
bug, such that he/she was just about at the point to make progress, when the
original late-shift programmer arrived again around 2pm.
In the end they decided that treating bugs this way was not effective - bugs
were either sufficiently serious that on-call people should deal with them
non-stop until they were solved, or they were not, in which case they were
dealt with routinely.
Yes, management may be partly to blame, but the lesson I took home from this
was that tag-bug-wrangling is not efficient - it's somewhat like expecting
9 women to produce a baby in 1 month.
Hence I'm not convinced that a lot of people doing fixing/reviewing
(say at least 20 new people, each doing 2 hours every other week) will help
that much.
(new people, because I'm assuming that any "existing" person who spends more
time on bug reviewing/fixing will stop doing something else useful Perl-
related that we will miss. And for volunteers, time is usually the
constraint, not money)
> know I'd be more likely to do this if I had publicly committed to spending
> two hours each tues, rather than just a vague good-will intent to look at
> the bugs DB from time to time (and usually getting distracted in the
> meantime).
> [ Hmmmm: maybe I should publicly commit to doing maint-5.10 integrations
> too ;-) ]
I've learned enough to know that things I commit to more rapidly stop being
fun, and guilt is not as good a motivation as enthusiasm. This might just be
me.
> > But if anyone competent (many patches already in core, if not commit rights)
> > wants to say "I can stay sane fielding bugs, not need managing, side-step
> > the whole recruitment process, but cost more" then we have an alternative
> > plan, with cost, which we could consider.
>
> Well... I could probably answer yes to the list above, but only if the role
> was part-time (eg 2 days per week).
Mmm, tempting. But then are we expanding the plan to 0.4 * Dave + 1.0 * $other
(or maybe 0.8 or 0.6 * $other)? Or are you comfortable that you can stay sane
for 2 days a week trying to answer 5 bugs a day?
Nicholas Clark
H.Merijn Brand
wrote:
> > know I'd be more likely to do this if I had publicly committed to spending
> > two hours each tues, rather than just a vague good-will intent to look at
> > the bugs DB from time to time (and usually getting distracted in the
> > meantime).
> > [ Hmmmm: maybe I should publicly commit to doing maint-5.10 integrations
> > too ;-) ]
>
> I've learned enough to know that things I commit to more rapidly stop being
> fun, and guilt is not as good a motivation as enthusiasm. This might just be
> me.
No, here too. Motivation (and the time available to spend on FUN parts
of problems to gain motivation) are essential.
Dave Mitchell
> > I would expect the junior not to be able to make many inroads against that
> > sort of bug. On the other hand, just having someone filter away the simple
> > bugs would still be useful, but whether it's still a full-time job, I
> > don't know.
>
> I'm not sure. However, as well as the 11 new bugs each week, there are 1800
> outstanding bugs that can be looked at, which would be 36 per week for a year,
> or 1 an hour. *That* would be somewhat soul destroying, but I'd be confident
> that there's enough work there to fill up any slack time.
That's very true - on refection I agree there's lots of work for a junior
to do, certianly full-time at first. I just also think that there's a lot
of stuff that only a senior person can do too.
> > An alternative would just to get lots of p5pers to each commit to doing
> > a couple of hours of bug reviewing / fixing per week, in a slightly more
> > organised fashion than at present, so that at least all bug reports get
> > answered, and the simple ones get processed. Whether this would work in
> > practice, I don't know; but from a personal psychological perspective, I
>
> I don't think it would, for a few reasons. You've already stated one above -
> "day's hard debugging". Any bug with a cause that couldn't be identified
> within a couple of hours would go unanswered.
I was probably being excessively starry-eyed when I wrote that :-)
> > Well... I could probably answer yes to the list above, but only if the role
> > was part-time (eg 2 days per week).
>
> Mmm, tempting. But then are we expanding the plan to 0.4 * Dave + 1.0 * $other
> (or maybe 0.8 or 0.6 * $other)? Or are you comfortable that you can stay sane
> for 2 days a week trying to answer 5 bugs a day?
I would probably remain sane if it was a mixture of mundane and
interesting bugs. It would certainly be less tedious than some of the paid
work I do :-)
I think there's certainly scope for both both a junior and senior.
If there was only funding for one, then I'm not sure which is more
important: the junior would be better at reducing the number of bug
reports, while the senior would be better at reducing the number of
bugs.
--
"I do not resent criticism, even when, for the sake of emphasis,
it parts for the time with reality".
-- Winston Churchill, House of Commons, 22nd Jan 1941.
Tels
On Monday 21 April 2008 16:16:02 Dave Mitchell wrote:
> On Fri, Apr 18, 2008 at 02:15:28PM +0100, Nicholas Clark wrote:
> > > I would expect the junior not to be able to make many inroads
> > > against that sort of bug. On the other hand, just having someone
> > > filter away the simple bugs would still be useful, but whether
> > > it's still a full-time job, I don't know.
[snip]
> > Mmm, tempting. But then are we expanding the plan to 0.4 * Dave +
> > 1.0 * $other (or maybe 0.8 or 0.6 * $other)? Or are you comfortable
> > that you can stay sane for 2 days a week trying to answer 5 bugs a
> > day?
>
> I would probably remain sane if it was a mixture of mundane and
> interesting bugs. It would certainly be less tedious than some of the
> paid work I do :-)
>
> I think there's certainly scope for both both a junior and senior.
> If there was only funding for one, then I'm not sure which is more
> important: the junior would be better at reducing the number of bug
> reports, while the senior would be better at reducing the number of
> bugs.
I'd then choose the junior, in the hope he first reduced the number of
bug reports, and then becomes senior enough to actually start reducing
the bugs :)
All the best,
Tels
--
Signed on Mon Apr 21 19:05:49 2008 with key 0x93B84C15.
View my photo gallery: http://bloodgate.com/photos
PGP key on http://bloodgate.com/tels.asc or per email.
"God is dead and I don't feel all too well either...."
-- Ralph Moonen
Chris Prather
>
> I think there's certainly scope for both both a junior and senior.
> If there was only funding for one, then I'm not sure which is more
> important: the junior would be better at reducing the number of bug
> reports, while the senior would be better at reducing the number of
> bugs.
>
>
I would expect that eventually the Junior would be good at the latter (for
some subset of bugs) as well.
-Chris
Sven Dowideit
To keep this discussion going towards something actionable:
The Senior's required skill set is relatively obvious - a current Perl
developer that has enough experience to guide the Junior, and to be able
to work on the entire suite of Bugs
The Junior on the other hand - What would you list in the 'job ad'?
Perl, C, Preprocessor macros, cross platform attitude ....
It might even make sense to consider making the Junior role a
Scholarship sort of thing - to bootstrap interested developers to the
experience level that will grow the Developer community.
I for example, am interested in doing more for Perl, but the sheer
immensity (and obfuscation of the macros) of the Codebase, and the time
pressures of finding and doing paid work make it slow going. Both these
2 poor excuses can be mitigated through the use of a Perl Developer
Scholarship, leading to a hopefully growing pool of Perl developers.
Sven
ava...@gmail.com
>
> To keep this discussion going towards something actionable:
>
> The Senior's required skill set is relatively obvious - a current Perl
> developer that has enough experience to guide the Junior, and to be able
> to work on the entire suite of Bugs
>
> The Junior on the other hand - What would you list in the 'job ad'?
>
> Perl, C, Preprocessor macros, cross platform attitude ....
I think interest is the most important part. I didn't know C when I
started contributing to the core and certainly didn't know the nuances
of the preprocessor or common pitfalls of writing portable code.
Although I'm by no means an active contributor or an important one for
that matter I've managed to get some patches accepted and I understand
the rest well enough to fix the occasional bug.
Reini Urban
> On Wed, Apr 16, 2008 at 08:37:55AM +0100, Nicholas Clark wrote:
> > Thanks for doing the maths. Yes, about $60K was the hypothetical salary in
> > a conversation with Richard Dice.
> >
> > Maybe we should call this "the sixty four thousand dollar plan"
>
> That was the guessed salary for a keen junior, for the plan I was suggesting
> that no-one has yet said "mmm, plan!" or "will never fly on technical grounds
> because..."
>
> (yes, I'm not yet interested in "how do we get the money", given that in 2001
> and 2002, just after the dot.com, TPF managed to raise large sums against
> less constrained deliverables than this)
Can we now safely assume that Bram is our new keen junior?
If so, good work so far!
--
Reini
Nicholas Clark
> On 4/23/08, Sven Dowideit <SvenDo...@home.org.au> wrote:
> >
> > To keep this discussion going towards something actionable:
> >
> > The Senior's required skill set is relatively obvious - a current Perl
> > developer that has enough experience to guide the Junior, and to be able
> > to work on the entire suite of Bugs
On a strict reading, I don't think that anyone in the world qualifies for
the Senior, because no-one (dead or alive) knows all parts of the core well
enough to fix bugs on them. (Specifically the regexp engine, the lexer and
the IO system spring to mind as strange black boxes.)
But I like your description, and I think that it's about right.
> > The Junior on the other hand - What would you list in the 'job ad'?
> >
> > Perl, C, Preprocessor macros, cross platform attitude ....
>
> I think interest is the most important part. I didn't know C when I
> started contributing to the core and certainly didn't know the nuances
> of the preprocessor or common pitfalls of writing portable code.
>
> Although I'm by no means an active contributor or an important one for
> that matter I've managed to get some patches accepted and I understand
> the rest well enough to fix the occasional bug.
Yes, I think that interest is key, although I think that some C is going to
be essential. I'd not like to try to teach someone C at the same time as
trying to teach them how C is used in the perl core.
Found as a side effect of something else, this blog entry about Subversion
has an interesting concept:
It may end up becoming a mainly "corporate" open source project (that
is, all development funded by corporations that depend on it), but
that's a fine way for a piece of mature software to settle down.
http://blog.red-bean.com/sussman/?p=90
I guess with this I'm hoping that we can transition Perl 5 towards being a
"corporate" open source project, given that Perl 5 certainly seems to be
nicely settled down. (And with its tentacles in far more places than is
obvious from above)
Nicholas Clark
David Nicol
> I'd not like to try to teach someone C at the same time as
> trying to teach them how C is used in the perl core.
Although they would be free of preconceptions. I imagine the Junior, after
graduating from the perl-5 maintenance project, encountering some fresh C
program on a new assignment and asking, "Where are all the macros? Why
isn't that abstracted?"
Mark Mielke
Most portable toolkits heavily use macros? Glib is the first example
that comes to mind. Perhaps Hello World avoids the use of macros.
Eventually all portable/maintainable/complex C programs make heavy use
of macros. I don't think Perl is unique, although it may be a good example.
Cheers,
mark
--
Mark Mielke <ma...@mielke.cc>