Blocker Definition and Focus?
Damon Sicore
Team,
I have a few bugzilla queries that I think people may be interested
in, and I think these may generate a bit of conversation. But first,
I want to present the criteria for blocker bugs as we've discussed in
the past:
- To be blocking, a bug must define an issue for which we would hold
shipment of the release, otherwise it will be marked [wanted-
firefox3]/[wanted-1.9]; those bugs can be renominated or elevated in
the future, but once marked blocking, a bug should never be un-
blocked without a serious change in foundational assumptions. To
refine this criteria of "holding a release":
* Must be a security issue, or
* Must be an issue that prevents users from browsing the web on
a daily basis, or
* A top crasher, or
* A memory leak, or
* A performance issue, or
* Is a regression from 2.0, or
* Is functionality in support of a P1 PRD item.
- We won't block on "meta" or tracking bugs, only the dependencies
which are actually blocking.
- We won't block on unconfirmed bugs.
- Bugs marked fixed can still be marked as blockers so that if they
reopen due to regressions we can track them.
- Nominations that come with descriptions of why you feel it should
block (or be wanted) and the impact of a potential fix are easier for
us to work with.
Now, with this criteria in mind, if we take a look at the work that
has occurred over the past 30 days, in bugzilla, there are some
interesting discussion points:
* Over the past 7 days (as of this email), 15 1.9 platform blockers
have been closed (See: http://tinyurl.com/2utwz2 ).
* Over the past 7 days, 20 non-blockers (1.9) have been closed.
(See: http://tinyurl.com/2pymfg ).
* Over the past 30 days, 74 blockers closed (See: http://
tinyurl.com/33ua3w ).
* Over the past 30 days, 145 non-blockers closed (See: http://
tinyurl.com/2wyr9a ).
These numbers bring these questions to mind:
1) Are we truly focusing on what we are defining as blockers today?
2) Is our definition/interpretation of what bugs should be blocking
accurate, or should our definition be changed?
3) With blocker lists that are not dropping, seemingly indicating
that we are never going to ship as we'll never reduce our blocker
lists to converge, why are we closing so many non-blockers in
comparison to closing blockers?
4) When we look at our review queues, are we reviewing blockers
first? And, even further, beta blockers before blockers?
Assuming we keep the definition of blockers at the beginning of this
email, I think there is a proper time to work on blockers versus non-
blockers, and at this point in our release cycle (trying to get beta
out the door), I would think that we'd be almost focusing exclusively
on blockers (beta blockers to be more exact). While true that non-
blockers are essential--we can't ship a product without them--if we
don't get the beta out the door, we will be missing an opportunity to
ship two betas this year. I think we all agree this is not what we
want.
Thoughts?
Sincerely,
Damon
Boris Zbarsky
> To refine this criteria of "holding a release":
> * Must be a security issue, or
> * Must be an issue that prevents users from browsing the web on a
> daily basis, or
> * A top crasher, or
> * A memory leak, or
> * A performance issue, or
> * Is a regression from 2.0, or
> * Is functionality in support of a P1 PRD item.
Satisfying one of those bullet points is not enough, right? For example, we
could have a regression from 2.0 that is not a blocker.
> - Bugs marked fixed can still be marked as blockers so that if they
> reopen due to regressions we can track them.
Though in practice people sometimes remove the blocker nomination once the bug
is fixed, presumably to help with blocker triage load.
> * Over the past 7 days (as of this email), 15 1.9 platform blockers
> have been closed (See: http://tinyurl.com/2utwz2 ).
>
> * Over the past 7 days, 20 non-blockers (1.9) have been closed. (See:
> http://tinyurl.com/2pymfg ).
Though of the 20 non-blockers, 2 are currently nominated as blockers. 3 more
were nominated and the nomination got removed when they got fixed. Doesn't
change the numbers that much, really.
The other consideration here is that some bugs never get nominated as blockers
(though they perhaps should be) if they get filed (especially with a patch), and
then fixed in short order.
All that said,
> 1) Are we truly focusing on what we are defining as blockers today?
Speaking for myself, probably not as much as I could. At least in part this has
to do with my answer to question 3 below.
> 3) With blocker lists that are not dropping, seemingly indicating that
> we are never going to ship as we'll never reduce our blocker lists to
> converge, why are we closing so many non-blockers in comparison to
> closing blockers?
At least in part, your average blocker bug is hard to fix and takes time
(otherwise it would have been fixed back when it was filed). So even if one
spends most of one's time on blockers the raw count might not quite reflect that.
As a corollary, if one has a short chunk of time to work between other
commitments, it's hard to make progress on a blocker and sometimes easy to fix
something else. The options then are do nothing or fix something else.
> I would think that we'd be almost focusing exclusively on
> blockers (beta blockers to be more exact).
It looks like the list of beta blockers is pretty well-owned. Are there
particular ones that could use help?
Can't say that for the overall blocker list, of course...
-Boris
Boris Zbarsky
> I have a few bugzilla queries that I think people may be interested in,
> and I think these may generate a bit of conversation.
I should note that I'm glad we're having this discussion, by the way!
-Boris
Benjamin Smedberg
> * Over the past 7 days, 20 non-blockers (1.9) have been closed. (See:
> http://tinyurl.com/2pymfg ).
> 3) With blocker lists that are not dropping, seemingly indicating that
> we are never going to ship as we'll never reduce our blocker lists to
> converge, why are we closing so many non-blockers in comparison to
> closing blockers?
> Assuming we keep the definition of blockers at the beginning of this
> email, I think there is a proper time to work on blockers versus
> non-blockers, and at this point in our release cycle (trying to get beta
> out the door), I would think that we'd be almost focusing exclusively on
> blockers (beta blockers to be more exact). While true that non-blockers
> are essential--we can't ship a product without them--if we don't get the
> beta out the door, we will be missing an opportunity to ship two betas
> this year. I think we all agree this is not what we want.
To break this down a little bit (I only see 12 bugs now, probably due
to recent sheriff backouts):
The PRBool rewrites (398599) are automated refactorings detecting invalid
use of the PRBool type. This is partly to aid in moz2 work and partly to fix
logical inconsistencies in our code: because Taras shouldn't be working on
blockers in general, this sounds like a good use of his time.
A couple of my build changes (398566, 398730) were written specifically for
moz2 and landed on trunk because it was safe to do so.
A couple of dead-code-removal bugs (399035, 398465) were trivial to write
and review.
A set of bugs on the list (397434, 399406, 398510) would have been blockers
but it was easier to just fix them than wait for blocker triage.
Are you suggesting that some of these bugs shouldn't have landed? I fully
agree that core hackers (employees and non-employees) should be prioritizing
blockers over non-blockers; but I don't think this data implies we're doing
otherwise. And I don't think we should stop accepting these kinds of changes
if they are ready.
--BDS
Robert O'Callahan
Damon Sicore wrote:
> * Over the past 7 days (as of this email), 15 1.9 platform blockers
> have been closed (See: http://tinyurl.com/2utwz2 ).
Note that a number of other blockers were closed with resolutions
other than FIXED. E.g. a few of my blockers were closed WORKSFORME,
typically because they were fixed by ongoing work but we didn't
identify the particular checkin with certainty.
> * Over the past 7 days, 20 non-blockers (1.9) have been closed.
> (See: http://tinyurl.com/2pymfg ).
FWIW, my bug on that list was FIXED by a checkin for a blocker.
16 of my 20 currently assigned blockers are stalled, mostly waiting
for review or dependent on bugs waiting for review, but some waiting
for (re)landing and also some I can't reproduce or need additional
feedback on. The other 4 only became blockers yesterday. Clearly I
need more bugs on my plate (dangerous thing to say); in layout I think
we need to work on assigning more of the 72 unowned bugs. This will
help us identify who's doomed and rebalance work appropriately. It
will also focus people's minds on whether bugs really need to be
blockers. Then instead of saying "oh my, 150 layout blockers" we can
say "David's really doomed and needs help or retriaging with his
bugs".
Rob
Robert Kaiser
> * Over the past 7 days (as of this email), 15 1.9 platform blockers have
> been closed (See: http://tinyurl.com/2utwz2 ).
>
> * Over the past 7 days, 20 non-blockers (1.9) have been closed. (See:
> http://tinyurl.com/2pymfg ).
>
> * Over the past 30 days, 74 blockers closed (See:
I think plain bug counts often give a wrong impression about what's
actually going on.
This is among other things because
- one major blocker being fixed is weighted equally there as each of a
couple of one- or few-line correctness fixes one stumbled across while
investigating this major bug
- random volunteer contributors being likely to fix smaller non-blocker
issues (though looking at the queries, those are mostly absent here)
- when someone stumbles upon a new crasher or such and has a fix
available fast, he probably just asks for approval instead of blocking,
so bad but easy-to-fix stuff ends up being not marked a blocker, while
big and hard work without immediate patches probably ends up as one
There might be other reasons as well, but those already show that such
counts tend to not clearly depict the focus or non-focus on blockers.
Robert Kaiser
Robert O'Callahan
On Oct 17, 2:12 pm, Damon Sicore <dsic...@mozilla.com> wrote:
> * Over the past 7 days (as of this email), 15 1.9 platform blockers
> have been closed (See: http://tinyurl.com/2utwz2).
Some others got resolved in other ways; a few of mine got marked
WORKSFORME (typically fixed by ongoing work on other blockers, but we
didn't identify the fixing checkin with 100% certainty).
> * Over the past 7 days, 20 non-blockers (1.9) have been closed.
> (See: http://tinyurl.com/2pymfg).
FWIW my bug in there was FIXED by a checkin for a blocker bug.
16 out of 20 of my currently assigned blocking+ bugs are waiting for
review, waiting for landing, waiting for feedback, or I cannot
reproduce, or depend on another such bug (some are really dups, others
need additional work once the dependee bug lands). The other 4 are
bugs that only became blocking+ yesterday so I haven't worked on them
yet. So obviously I need to grab more bugs. I think the way forward is
to actually assign as many of those bugs as possible to real people;
that will help us see who's genuinely doomed and rebalance workload,
and also concentrate our minds on whether bugs should really be
blockers.
Rob
Robert O'Callahan
> A few random notes...
Wow, Google Groups held on to that for about five hours!
Rob