bugzilla.mozilla.org workflow changes
Gervase Markham
optimum Bugzilla workflow:
http://wiki.mozilla.org/BugzillaWorkflowImprovements
This included the clear definition of a workflow:
http://steelgryphon.com/testcases/bugzilla-workflow-9.png
which I hope will be useful to new QA contributors. Among other things,
the new workflow had a READY state, into which bugs should be put by
triagers and QA when they were ready to be fixed, and from which
developers could take bugs to fix.
Having reviewed the discussion from back then:
http://groups.google.com/group/mozilla.dev.planning/browse_thread/thread/80a8f074ab137f10/f9a60aaf914bcd7e?q=%22READY+state%22+mozilla&lnk=ol&
I still think we should make the following changes:
- Create the new READY state, and encourage triagers to manually move
ready bugs into it.
- Abolish the REOPENED state (automatically converting all existing
REOPENED bugs to NEW) as it adds no useful information.
- Publish a tweaked version of mconnor's diagram so people know how
things should work, and as a guide for new QA community members.
Do people think there is still value in this?
Gerv
Jean-Marc Desperrier
> Among other things, the new workflow had a READY state, into which bugs
> should be put by triagers and QA when they were ready to be fixed, and
> from which developers could take bugs to fix.
>
> I still think we should make the following changes:
>
> - Create the new READY state, and encourage triagers to manually move
> ready bugs into it.
>
> - Abolish the REOPENED state (automatically converting all existing
> REOPENED bugs to NEW) as it adds no useful information.
>
> - Publish a tweaked version of mconnor's diagram so people know how
> things should work, and as a guide for new QA community members.
>
> Do people think there is still value in this?
I'd go further. There could be a lot of value in creating new states
between "assigned" and "resolved fixed".
I'm thinking of :
- Assigned : Changed to mean that the bug is waiting until the dev it's
affected to *actually* starts working on it
- Under Work : the dev is currently busy investigating the bug/writing
code to solve it
- Patched : There's a patch attached that solves the bug as far as the
affected dev is concerned, but that has not yet been validated.
And then a report of the bugs that have stayed more than a few weeks in
the "Patched" state, or several months in the "Assigned" state would be
*very* useful.
Benjamin Smedberg
> This included the clear definition of a workflow:
> http://steelgryphon.com/testcases/bugzilla-workflow-9.png
> which I hope will be useful to new QA contributors. Among other things,
> the new workflow had a READY state, into which bugs should be put by
> triagers and QA when they were ready to be fixed, and from which
> developers could take bugs to fix.
Have developers indicated that they are actually going to use this new
state? I'm wary of adding additional metadata to all our bugs unless there
is a clear indication that it's going to help someone in a practical way.
READY is only better than a little whiteboard note if it's going to help
create effective queries, and we'd all pay a price to keep READY and NEW
separate.
It's like ASSIGNED: while there may be some theoretical difference between
ASSIGNED and NEW, I haven't seen most developers use the ASSIGNED state.
I tend to think that in most of the modules I own READY would just be
another thing for people to argue over, with little practical benefit. On
the other hand, I think we could probably ignore it and it would cause only
minor inconvenience.
--BDS
L. David Baron
> - Create the new READY state, and encourage triagers to manually move
> ready bugs into it.
Don't we already have such a distinction: UNCONFIRMED vs. NEW?
What does this *additional* distinction add?
I'd love to have a way to un-confirm a bug, though. (That is, move
it from NEW to UNCONFIRMED, because it shouldn't have been
confirmed.)
> - Abolish the REOPENED state (automatically converting all existing
> REOPENED bugs to NEW) as it adds no useful information.
Sounds reasonable to me.
-David
--
L. David Baron http://dbaron.org/
Mozilla Corporation http://www.mozilla.com/
Martijn
> On Tuesday 2009-03-31 10:24 +0100, Gervase Markham wrote:
>> ready bugs into it.
>
> What does this *additional* distinction add?
I have the same feeling about this.
Afaics, it would only cause a lot of work where qa/triagers needs to
go through all the NEW bugs and mark them as READY when appropriate.
> I'd love to have a way to un-confirm a bug, though. (That is, move
> it from NEW to UNCONFIRMED, because it shouldn't have been
> confirmed.)
>
>> - Abolish the REOPENED state (automatically converting all existing
>> REOPENED bugs to NEW) as it adds no useful information.
I guess you could do that, but I don't really see the harm of REOPENED bugs.
Regards,
Martijn
> Sounds reasonable to me.
>
> -David
>
> --
> L. David Baron http://dbaron.org/
> Mozilla Corporation http://www.mozilla.com/
> _______________________________________________
> dev-planning mailing list
> dev-pl...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-planning
>
--
Martijn Wargers - Help Mozilla!
http://quality.mozilla.org/
http://wiki.mozilla.org/Mozilla_QA_Community
irc://irc.mozilla.org/qa - /nick mw22
Joshua Cranmer
> On Tuesday 2009-03-31 10:24 +0100, Gervase Markham wrote:
>> - Create the new READY state, and encourage triagers to manually move
>> ready bugs into it.
>
> Don't we already have such a distinction: UNCONFIRMED vs. NEW?
> What does this *additional* distinction add?
At this stage, I'm going to say that adding a new READY state will
likely cause some procedural furor. I can sometimes see the need for a
distinction between bugs that are not sufficiently reproducible to be
fixable but clearly exist among a large sector of the user population
(e.g., <https://bugzilla.mozilla.org/show_bug.cgi?id=451118>) and the
bugs whose existence may be in question or bugs whose fixes are, in
principle, known.
That said, the more useful place to stick the state, in my eyes, is
between UNCO and NEW, not between NEW and ASSI as is the current proposal.
> I'd love to have a way to un-confirm a bug, though. (That is, move
> it from NEW to UNCONFIRMED, because it shouldn't have been
> confirmed.)
This, on the other hand, I would gladly welcome. Some of the components
I triage have large numbers of NEW bugs which were confirmed in days
when the bar was rather low.
>> - Abolish the REOPENED state (automatically converting all existing
>> REOPENED bugs to NEW) as it adds no useful information.
>
> Sounds reasonable to me.
+<however much my vote counts for>
Aakash Desai
As for a guide to go from state to state for a bug, there's one posted here: http://quality.mozilla.org/bugs-life-walkthrough . It's a beginner's guide, so there's still some kinks (i.e. screenshots, specific reasons for going from one state to the other) that need to be ironed out or put into an Advanced version of the document.
Lastly, there's still some problems with the workflow diagram linked to in Gervase's e-mail. For the lazy, the link is http://steelgryphon.com/testcases/bugzilla-workflow-9.png . Here are my gripes on it:
- WontFix is not just for enhancements
- If a bug is found under inappropriate conditions, then the bug is Invalid, not WorksForMe. If it's not found in the scenario that the bug reporter listed, then it's a WorksForMe.
Thanks,
Aakash
----- Original Message -----
From: "L. David Baron" <dba...@dbaron.org>
To: dev-pl...@lists.mozilla.org
Sent: Tuesday, March 31, 2009 8:00:09 AM GMT -08:00 US/Canada Pacific
Subject: Re: bugzilla.mozilla.org workflow changes
On Tuesday 2009-03-31 10:24 +0100, Gervase Markham wrote:
> - Create the new READY state, and encourage triagers to manually move
> ready bugs into it.
Don't we already have such a distinction: UNCONFIRMED vs. NEW?
What does this *additional* distinction add?
I'd love to have a way to un-confirm a bug, though. (That is, move
it from NEW to UNCONFIRMED, because it shouldn't have been
confirmed.)
> - Abolish the REOPENED state (automatically converting all existing
> REOPENED bugs to NEW) as it adds no useful information.
Sounds reasonable to me.
-David
Mike Shaver
> That said, the more useful place to stick the state, in my eyes, is between
> UNCO and NEW, not between NEW and ASSI as is the current proposal.
I propose a zero-net-complexity rule for our bugzilla. If you add a
state, you have to remove one. REOPENED and ASSIGNED are both pretty
much useless to us, not carrying their weight in terms of clutter or
complexity. WONTFIX/INVALID might be foldable too, since it's not
like WONTFIX keeps us from having people re-ask, and I don't think we
ever really care about distinguishing "what you're asking for is not
going to happen" and "what you asked doesn't make sense or is a bug in
some other piece of software or wtf are you trying to say etc." beyond
what would be expressed in the closing comment.
(I'd like to say that if you add a flag you have to remove one, but
when all we have is flags/keywords/whiteboard-tribal-knowledge, every
problem looks like a [?-+ parity:investigate]. See also the co-opting
of priority field for shade-of-blocker, squatting a useful management
and planning tool for module owners and managers.)
See also: QA contact and Hardware fields, IMO. If we didn't have
bugzilla right now, this is definitely not the system we'd build.
Here are the things I care about as an engineering/product manager:
- what bugs have had work done on them?
- what bugs are assigned to which people (loading)? what reviews?
- what bugs block the given release?
- what is the rate of addition to the blocker list? what is the rate
of fix (to trunk, and to a given release stream)? (likewise for
security and crash bugs, and untriaged/new bugs)
- what bugs have patches? what bugs have test cases?
- what open bugs haven't been touched in a while?
- what bugs are being reported against which newly-released versions
or milestones?
As a developer, also I care about:
- what bugs need a comment from me?
- what bugs of mine are ready to land on which branches?
- what bugs have an updated patch for re-review?
- where do I put a comment in a bug so that it gets pulled into
release notes or a changelog?
- what bugs have clear regression ranges?
- in what *@(#&*@*#&@ component should I file this bug I just found?
- why can I still not watch components?
As a user and product advocate, I care about:
- are you really telling me that there's nowhere to file a bug about a
layout problem in Firefox without clicking through "Other Products"
and looking through the list of Java crypto software and webtools?
- how to not be mortified when someone asks where to file a bug, and
what all the twiddles mean
- how to do anything but nod sadly when someone says "I didn't have
time to figure out how to file that bug"
I don't think these states are really going to help me answer many of
these things, so I'm loath to take much additional complexity or time
investment for it.
Mike Shaver
> I'm not sure that I agree with abolishing the Re-Opened state on a semantic level. The Re-Opened state is used as a marker to community members (whether they're involved or not involved with fixing the bug) that this bug already has had a patch created and was reviewed before being submitted. Now, from what I can gather, a bug that goes to Re-Opened status usually means that it failed on a specific platform, certain use case failed or it broke another function. On the other hand, the New state means that the bug has been unconfirmed and it's either in the process of being assigned to a developer (semantics again because bugs stay in the New state even after they're assigned) and/or a patch has not been created for it. I think moving a bug back to the New state should be possible, but the Re-opened state does have a lot of use to it to people not involved with the bug.
I don't think REOPENED tells you much that you don't have to read the
bug to confirm, at which point the state isn't useful. At most,
REOPENEDness should be a keyword like "regression" so that the 0.3% of
people who would query on it can do so without adding more state-space
for the rest of the world.
> As for a guide to go from state to state for a bug, there's one posted here: http://quality.mozilla.org/bugs-life-walkthrough . It's a beginner's guide, so there's still some kinks (i.e. screenshots, specific reasons for going from one state to the other) that need to be ironed out or put into an Advanced version of the document.
If the beginner's guide doesn't fit in 500 words and a 800x600 image,
we should streamline the process until it does, IMO.
> - If a bug is found under inappropriate conditions, then the bug is Invalid, not WorksForMe. If it's not found in the scenario that the bug reporter listed, then it's a WorksForMe.
Most people don't care about that, though. They care about:
- we don't know wtf about this bug
- this bug will be fixed
- when (as in which release stream, as well as date)
- this bug won't be fixed (why not is a detail)
- this bug has been fixed in (pick-N release streams/versions/dates)
It's only to the twisted mind of a developer that "INVALID" is a form
"resolved"; likewise with WONTFIX, I think.
Mike
Joshua Cranmer
> On Tue, Mar 31, 2009 at 11:36 AM, Joshua Cranmer <Pidg...@verizon.net> wrote:
>> That said, the more useful place to stick the state, in my eyes, is between
>> UNCO and NEW, not between NEW and ASSI as is the current proposal.
For clarification: I'm not exactly in favor of adding a new state, but
I'm not altogether opposed to one in theory.
> See also: QA contact and Hardware fields, IMO. If we didn't have
> bugzilla right now, this is definitely not the system we'd build.
Is there a way to watch components without utilizing QA contact? If
there is, feel free to make that information more widely visible and
then get rid of it.
> As a developer, also I care about:
>
> - what bugs need a comment from me?
> - what bugs of mine are ready to land on which branches?
> - what bugs have an updated patch for re-review?
> - where do I put a comment in a bug so that it gets pulled into
> release notes or a changelog?
> - what bugs have clear regression ranges?
> - in what *@(#&*@*#&@ component should I file this bug I just found?
> - why can I still not watch components?
As a potential developer:
- which bugs would people like to me fix?
- which bugs are people not working on?
> I don't think these states are really going to help me answer many of
> these things, so I'm loath to take much additional complexity or time
> investment for it.
I find the ASSI state useful, in that I can distinguish between "I'll
look at fixing this bug if I have time" and "I'm going to commit to
fixing this bug." On the other hand, REOP is not very useful for any of
my bug grepping strategies.
Aakash Desai
bug to confirm, at which point the state isn't useful. At most,
REOPENEDness should be a keyword like "regression" so that the 0.3% of
people who would query on it can do so without adding more state-space
for the rest of the world.
Adding it as a keyword is fine and dandy for me as long as it's not completely removed from the system. Re-Openedness is something that's going to be seen in the wild when working on cross platform projects and it should be taken into account, no matter how good we are at handling them or how "small" of a number they begin to show up in our system.
> If the beginner's guide doesn't fit in 500 words and a 800x600 image,
we should streamline the process until it does, IMO.
We should really ask if all beginner's guides need to fit into a 500 word document or something hard-set like that. For me, if we're using something as complex as this: http://www.bugzilla.org/docs/tip/en/html/lifecycle.html ... sometimes writing a bit more is necessary to really explain it. Now, how the reader sees the information splayed out onto the screen is probably where you're getting at. In that light, yeah sure, I did say there were some kinks that needed to be ironed out :). We don't want to scare away potential volunteers, but having that information up in that form, for now, is a better alternative than not having it up at all.
> Most people don't care about that, though. They care about:
- we don't know wtf about this bug
- this bug will be fixed
- when (as in which release stream, as well as date)
- this bug won't be fixed (why not is a detail)
- this bug has been fixed in (pick-N release streams/versions/dates)
It's only to the twisted mind of a developer that "INVALID" is a form
"resolved"; likewise with WONTFIX, I think.
Developers aren't the only ones reading and changing the states on these bugs. There's a whole host of people external to Mozilla's community that read through this bug system (or we hope read through it) that don't understand how it works. If we start blurring the lines of what each state means, so it's easier for a developer to read, then we're not doing our job in trying to get more of the community involved.
Thanks,
Aakash
----- Original Message -----
From: "Mike Shaver" <mike....@gmail.com>
To: "Aakash Desai" <ade...@mozilla.com>
Cc: "L. David Baron" <dba...@dbaron.org>, dev-pl...@lists.mozilla.org
Sent: Tuesday, March 31, 2009 9:14:57 AM GMT -08:00 US/Canada Pacific
Subject: Re: bugzilla.mozilla.org workflow changes
Michael Connor
On 31-Mar-09, at 11:57 AM, Aakash Desai wrote:
> Lastly, there's still some problems with the workflow diagram linked
> to in Gervase's e-mail. For the lazy, the link is http://steelgryphon.com/testcases/bugzilla-workflow-9.png
> . Here are my gripes on it:
>
> - WontFix is not just for enhancements
It doesn't cover every single possible workflow, but it's somewhat
idealized. WONTFIX vs. INVALID is often a grey area, IMO.
> - If a bug is found under inappropriate conditions, then the bug is
> Invalid, not WorksForMe. If it's not found in the scenario that the
> bug reporter listed, then it's a WorksForMe.
The assumption made in the "can it be reproduced under appropriate
conditions?" decision point is that you follow the STR, and can't repro.
-- Mike
Serge Gautherie
> I'd love to have a way to un-confirm a bug, though. (That is, move
> it from NEW to UNCONFIRMED, because it shouldn't have been
> confirmed.)
I concur:
NEW would then mean "ready" somehow;
UNCONFIRMED would mean "needs triage", as it already does.
Boris Zbarsky
> Developers aren't the only ones reading and changing the states on these bugs. There's a whole host of people external to Mozilla's community that read through this bug system (or we hope read through it) that don't understand how it works. If we start blurring the lines of what each state means, so it's easier for a developer to read, then we're not doing our job in trying to get more of the community involved.
Shaver's point was that non-developers are the ones who don't care about
WONTFIX vs INVALID. The care about whether the behavior will change or not.
-Boris
Serge Gautherie
> It's like ASSIGNED: while there may be some theoretical difference between
> ASSIGNED and NEW, I haven't seen most developers use the ASSIGNED state.
I'm sad about that:
ASSIGNED should mean someone is actually (planning on) working on it.
Whereas "Assigned To" (+ NEW) is more like asking someone to, or that he
may/will but not yet.
Helps to see (in the graphs) which/how_many bugs are (supposedly)
currently taken care of (among all the open bugs).
Serge Gautherie
> - Abolish the REOPENED state (automatically converting all existing
> REOPENED bugs to NEW) as it adds no useful information.
I don't know. I usualy see two typical cases:
1) bugs which were (thought) fixed:
Reopen is usually a backout or a regression or needs additional changes.
This REOPENED I see as a kind of a priority todo.
2) bugs which were thought invalid/duplicates/etc:
These I wish they could go back (directly) to their previous state
instead of REOPENED.
Igor Bukanov
That is a good point. So what about renaming NEW to simply OPEN? This
would be especially relevant given that now REOPEN means to transition
into that state.
Igor
Aakash Desai
idealized. WONTFIX vs. INVALID is often a grey area, IMO.
That's great that there's an understanding of the issue known behind the diagram, but it's still community facing. The gray area doesn't belong to people that are read through this diagram for the first time. Those things are learned when they're dealing with bugzilla in the wild. The purpose of any set of documentation is create clarity out of an unclear situation and allow the reader to ask questions that build their knowledge of the system. It's not supposed to force the reader to ask questions about how the system should work as its base.
> The assumption made in the "can it be reproduced under appropriate
conditions?" decision point is that you follow the STR, and can't repro.
Following the STR and not being able to reproduce the problem can occur from misunderstandings of how people comprehend the steps. That definitely doesn't inherently make a bug invalid.
-- Aakash
----- Original Message -----
From: "Michael Connor" <mco...@mozilla.com>
To: "Aakash Desai" <ade...@mozilla.com>
Cc: "L. David Baron" <dba...@dbaron.org>, dev-pl...@lists.mozilla.org
Sent: Tuesday, March 31, 2009 9:51:54 AM GMT -08:00 US/Canada Pacific
Subject: Re: bugzilla.mozilla.org workflow changes
Frédéric Buclin
> I'd love to have a way to un-confirm a bug, though. (That is, move
> it from NEW to UNCONFIRMED, because it shouldn't have been
> confirmed.)
You can already do that since Bugzilla 3.2. b.m.o has this NEW -> UNCO
bug status transition disabled.
LpSolit
Frédéric Buclin
> Is there a way to watch components without utilizing QA contact? If
> there is, feel free to make that information more widely visible and
> then get rid of it.
No, you currently cannot watch components. Which is why each component
has its own default QA contact.
LpSolit
Boris Zbarsky
> You can already do that since Bugzilla 3.2. b.m.o has this NEW -> UNCO
> bug status transition disabled.
Uh... how hard would it be to ... undo that? With a vengeance.
-Boris
Aakash Desai
-- Aakash
----- Original Message -----
From: "Boris Zbarsky" <bzba...@mit.edu>
To: dev-pl...@lists.mozilla.org
Sent: Tuesday, March 31, 2009 9:54:52 AM GMT -08:00 US/Canada Pacific
Subject: Re: bugzilla.mozilla.org workflow changes
-Boris
Benjamin Smedberg
> Benjamin Smedberg wrote:
>
>> It's like ASSIGNED: while there may be some theoretical difference
>> between
>> ASSIGNED and NEW, I haven't seen most developers use the ASSIGNED state.
>
> I'm sad about that:
> ASSIGNED should mean someone is actually (planning on) working on it.
> Whereas "Assigned To" (+ NEW) is more like asking someone to, or that he
> may/will but not yet.
I think the answer to that is/should be: if you're not actively working on
it, don't assign it to yourself. If you want to track bugs you're interested
in perhaps working on in the future, use personal bug tags.
--BDS
L. David Baron
Frédéric Buclin
>> You can already do that since Bugzilla 3.2. b.m.o has this NEW -> UNCO
>> bug status transition disabled.
>
> Uh... how hard would it be to ... undo that? With a vengeance.
It's trivial. The checkbox for the NEW -> UNCO transition is currently
unchecked. Just enable it and you can use it immediately.
LpSolit
Michael Connor
On 31-Mar-09, at 12:09 PM, Mike Shaver wrote:
> On Tue, Mar 31, 2009 at 11:36 AM, Joshua Cranmer <Pidg...@verizon.net
> > wrote:
>> That said, the more useful place to stick the state, in my eyes, is
>> between
>> UNCO and NEW, not between NEW and ASSI as is the current proposal.
>
> I propose a zero-net-complexity rule for our bugzilla. If you add a
> state, you have to remove one. REOPENED and ASSIGNED are both pretty
> much useless to us, not carrying their weight in terms of clutter or
> complexity. WONTFIX/INVALID might be foldable too, since it's not
> like WONTFIX keeps us from having people re-ask, and I don't think we
> ever really care about distinguishing "what you're asking for is not
> going to happen" and "what you asked doesn't make sense or is a bug in
> some other piece of software or wtf are you trying to say etc." beyond
> what would be expressed in the closing comment.
I agree with this sentiment. ASSIGNED and REOPENED can both die, IMO.
That said, I think we have three states we care about:
* Someone reported a bug
* We can reproduce this bug following STR, and more investigation is
warranted.
* We understand this bug and have determined we should fix it, and
have the information needed to fix it.
> (I'd like to say that if you add a flag you have to remove one, but
> when all we have is flags/keywords/whiteboard-tribal-knowledge, every
> problem looks like a [?-+ parity:investigate]. See also the co-opting
> of priority field for shade-of-blocker, squatting a useful management
> and planning tool for module owners and managers.)
Yes, we need a better flag system. Working with the tools we have,
etc. Insane pile-o-flags was a lot worse.
> As a developer, also I care about:
>
> - what bugs have clear regression ranges?
Breaking this out, what you really care about is probably more like:
- what bugs have the information needed to just fix the bug.
This is some combination of testcase ready, regression window found,
> As a user and product advocate, I care about:
> - are you really telling me that there's nowhere to file a bug about a
> layout problem in Firefox without clicking through "Other Products"
> and looking through the list of Java crypto software and webtools?
> - how to not be mortified when someone asks where to file a bug, and
> what all the twiddles mean
> - how to do anything but nod sadly when someone says "I didn't have
> time to figure out how to file that bug"
These changes are explicitly designed to solve these problems. The
intent is that the initial bug filing experience, for someone without
bmo privs, can be made simple:
What are you reporting a bug in? Firefox
Can you give us a short summary of the problem? The foozit is broken.
Please give details on how to reproduce this problem? When I click on
the foozit on http://example.com, the foozit blorbles.
That gets dumped to Firefox::General (or, since stuff legitimately
lives there, Firefox::Needs Triage). Then we have a first pass of
triage, which is very fast, basically:
* Is this an RFE or a bug?
** If it's a bug, can I reproduce given the STR?
* If yes, confirm and move to the correct component, unless it's an
obvious dupe.
* If no, give people information on support/filing better bugs/etc and
close the bug.
Basically, this is splitting the traditionally overloaded UNCO vs. NEW
status change into two steps: something anyone can do and then
something requiring deeper knowledge, and removing the need for users
to deal with anything more complicated than telling us their problem.
-- Mike
Michael Connor
On 31-Mar-09, at 1:03 PM, Aakash Desai wrote:
>> It doesn't cover every single possible workflow, but it's somewhat
> idealized. WONTFIX vs. INVALID is often a grey area, IMO.
>
> That's great that there's an understanding of the issue known behind
> the diagram, but it's still community facing. The gray area doesn't
> belong to people that are read through this diagram for the first
> time. Those things are learned when they're dealing with bugzilla in
> the wild. The purpose of any set of documentation is create clarity
> out of an unclear situation and allow the reader to ask questions
> that build their knowledge of the system. It's not supposed to force
> the reader to ask questions about how the system should work as its
> base.
Shaver already mentioned this, as a "we should just collapse those two
together" idea. Basically, the idea is to change process to be
simpler, and avoid grey areas. This was a proposal for how we should
be doing things, rather than documentation on what we do now.
>> The assumption made in the "can it be reproduced under appropriate
> conditions?" decision point is that you follow the STR, and can't
> repro.
>
> Following the STR and not being able to reproduce the problem can
> occur from misunderstandings of how people comprehend the steps.
> That definitely doesn't inherently make a bug invalid.
We have historically had a very low rate of return on digging into
poorly-filed bugs. The intent is "try the STR on the same platform,
if it doesn't work, resolve and move on" as the approach to triage.
Common bugs will be filed again, with better STR. Trust me on that one.
-- Mike
Mike Shaver
> Re-Openedness is something that's going to be seen in the wild when working on cross platform projects and it should be taken into account
I don't know what cross-platformness has to do with it, since REOPENED
doesn't mean "regressed something", it means "was backed out" or "fix
didn't work".
> no matter how good we are at handling them or how "small" of a number they begin to show up in our system.
No, the amount of cost (complexity, confusion, visual noise) we
(users, developers, administrators) should bear is definitely related
to how common the cases are in which the capability gives us value.
>> If the beginner's guide doesn't fit in 500 words and a 800x600 image,
> we should streamline the process until it does, IMO.
>
> We should really ask if all beginner's guides need to fit into a 500 word document or something hard-set like that. For me, if we're using something as complex as this: http://www.bugzilla.org/docs/tip/en/html/lifecycle.html ... sometimes writing a bit more is necessary to really explain it.
That's exactly my point: our process is too complicated.
>> Most people don't care about that, though. They care about:
> - we don't know wtf about this bug
> - this bug will be fixed
> - when (as in which release stream, as well as date)
> - this bug won't be fixed (why not is a detail)
> - this bug has been fixed in (pick-N release streams/versions/dates)
> It's only to the twisted mind of a developer that "INVALID" is a form
> "resolved"; likewise with WONTFIX, I think.
>
> Developers aren't the only ones reading and changing the states on these bugs.
I know -- those are the things that bug reporters and followers-on care about.
> There's a whole host of people external to Mozilla's community that read through this bug system (or we hope read through it) that don't understand how it works. If we start blurring the lines of what each state means, so it's easier for a developer to read, then we're not doing our job in trying to get more of the community involved.
People report bugs in order to have action taken on them. They care
about understanding what action is being taken, and what outcome
results, but they also very much care that the action _is_ taken,
which means they have a vestd interest in developers being able to
work with the bugs effectively and avoid things being dropped or
delayed unnecessarily.
It's not like bugzilla is currently optimized for our developers at
the expense of our users -- it's not really optimal for anyone.
Mike
Michael Connor
On 31-Mar-09, at 11:00 AM, L. David Baron wrote:
> On Tuesday 2009-03-31 10:24 +0100, Gervase Markham wrote:
>> - Create the new READY state, and encourage triagers to manually move
>> ready bugs into it.
>
> Don't we already have such a distinction: UNCONFIRMED vs. NEW?
> What does this *additional* distinction add?
I think the problem is that this is overloaded. NEW in layout means a
lot of things (it is a bug under the spec + has a testcase + lunar
alignment is positive, etc), and effectively creates a barrier to
entry for anyone touching layout bugs. Of course, in my ideal
workflow, UNCO bugs are in a big bucket away from components we're
using for actual work. Only when we can reproduce issues does it get
onto developer/QA radar.
A major goal is that UNCO->NEW is stage one of a filtering process,
and our overtaxed QA people (i.e. the people who create testcases and
find regression windows) can save time by only focusing on bugs people
can actually reproduce.
-- Mike
Frédéric Buclin
> It's not like bugzilla is currently optimized for our developers at
> the expense of our users -- it's not really optimal for anyone.
Do you have concrete proposals? And if possible which is not limited to
Fx teams.
LpSolit
Michael Connor
On 31-Mar-09, at 1:49 PM, Mike Shaver wrote:
> On Tue, Mar 31, 2009 at 12:51 PM, Aakash Desai <ade...@mozilla.com>
> wrote:
>> Re-Openedness is something that's going to be seen in the wild when
>> working on cross platform projects and it should be taken into
>> account
>
> I don't know what cross-platformness has to do with it, since REOPENED
> doesn't mean "regressed something", it means "was backed out" or "fix
> didn't work".
Or "no, it shouldn't be INVALID/WONTFIX/etc." It really only means
"this was closed, now it isn't" which isn't of a lot of use.
-- Mike
Mike Shaver
Phil Ringnalda
> I propose a zero-net-complexity rule for our bugzilla. If you add a
> state, you have to remove one. REOPENED and ASSIGNED are both pretty
> much useless to us
Not that I expect *anything* to happen from all this after all the
previous iterations of sound-n-fury, but, please don't remove ASSI until
after Tb3 ships. We're overloading it so that
blockingTb3/assignee=driver/NEW means that driver is assigned the task
of finding an owner and blockingTb3/assignee=driver/ASSI means that
driver happens to be the person who is fixing the bug.
Reed Loden
Mike Shaver <mike....@gmail.com> wrote:
> I propose a zero-net-complexity rule for our bugzilla. If you add a
> state, you have to remove one. REOPENED and ASSIGNED are both pretty
> much useless to us
Not sure who the "us" is, but I personally use ASSIGNED all the time to
track bugs I'm actively working on versus ones that are just assigned
to me to get to sometime. I would be very unhappy if it just suddenly
disappeared, as I would have to resort to using status whiteboard
entries, which aren't very easy to maintain.
~reed
--
Reed Loden - <re...@reedloden.com>
timeless
> Don't rub it it. :(
Sorry, I just had to rub it *in* :(
Michael Connor
I already intend to do something here, FWIW. I think we can kill
REOPENED for sure, and probably ASSIGNED, since it's generally
underused.
If we added READY as we removed ASSIGNED/REOPENED, you could use that
in place of ASSIGNED, since something without an owner isn't ready...
-- Mike
Frédéric Buclin
> I already intend to do something here, FWIW. I think we can kill
> REOPENED for sure, and probably ASSIGNED, since it's generally underused.
We also use the ASSIGNED status in the Bugzilla product, for the same
reason as described by reed. I would hate to see it going away.
LpSolit
Michael Connor
I don't think that we should leave statuses that aren't widely used
for the sake of a small minority within Mozilla. The idea of
"accepting" a bug is really strange workflow, and opaque to most
people, and just because someone is using it doesn't mean it actually
makes sense for the installation. If the bug isn't something you're
going to do, reassign to nobody@. Isn't "keep track of bugs I
probably will fix eventually" what personal tagging is supposed to
solve, as Benjamin suggests?
In terms of making it easier for people to understand what's going on,
having ambiguous "assigned to me, but I'm not actually doing anything
with it" states is really not a good approach.
-- Mike
Phil Ringnalda
> If we added READY as we removed ASSIGNED/REOPENED, you could use that in
> place of ASSIGNED, since something without an owner isn't ready...
Lol whut? If it isn't READY unless it has an owner, then just pretend
that READY is spelled A-S-S-I-G-N-E-D and we can all go home.
If instead what you mean is that there are four available states,
NEW+nobody@, NEW+somebody@, READY+nobody@, READY+somebody@, then yeah,
we'd do fine with an ASSI => READY migration, we'd just be slightly
perverting the new meaning of NEW by having them assigned (which is
actually fairly reasonable, since NEW and assigned to the QA person who
will then change it to READY and assigned to nobody@ once they write a
testcase might make this worth doing, and NEW and assigned to the
manager who is looking for a victim is an obvious extension of that),
and anyway when you take away people's "my NEW and my ASSI" distinction,
they'll just use "my NEW and my READY" instead, so that's fine.
Frank Wein
> It's like ASSIGNED: while there may be some theoretical difference between
> ASSIGNED and NEW, I haven't seen most developers use the ASSIGNED state.
The meaning of ASSIGNED also changed a bit in the last years (as far as
I know). A few years ago every component had a real person as default
assignee. These days almost every component has nob...@mozilla.org as
assignee, so there's probably not much need for the ASSIGNED state anymore.
Frank
Michael Connor
Yeah, that's basically it. If people really want to preserve the
"it's assigned to me, kinda" workflow. I'd like to just break people
out of that though.
-- Mike
Mike Shaver
Proposals I make would definitely be biased towards facilitating the
scale, pace, and level of interconnectedness of Firefox/Gecko
development, and towards addressing the needs of Firefox/Gecko.
Bugzilla might let us customize things on a per-product level such
that other products in b.m.o can have a workflow and set of tools that
suits their needs better where they differ? (Thunderbird might be
able to have a state for "owner needed", f.e.!)
Mike
GPHemsley
> 2009/3/31 Frédéric Buclin <LpSo...@gmail.com>:
Indeed, this discussion seems to be heavily biased towards Firefox/
Gecko, and forgetting all the many different pieces of software that
use BMO for their bug tracking. The software with smaller communities
and fewer filed bugs (e.g. Bespin) are better able to use all of these
status names. I use ASSIGNED to categorize bugs that are actually the
responsibility of a particular person. Otherwise, I just ignore
whomever the bug is assigned to (usually nobody), because the bug is
not actually "assigned".
And it's true that REOPENED is (should be) used for bugs that were
originally thought to be RESOLVED, but later turned out that they
weren't, for reasons that have already been mentioned: fix backed out,
fix didn't work, etc. As for READY, in the course of reading this
discussion, I've already forgotten what it means. Ready for what?
Triage? Testcase? Fix to be written? Fix to be applied? In my opinion,
READY is a bad name for a new status.
If you want to combine WONTFIX and INVALID, I suppose I could live
with that, even though specific situations can help to differentiate
which would be appropriate. In that case, I'd recommend rolling them
into NOTABUG.
I probably had more things to say, but it took me a while to get
through these 40-something posts, so I've forgotten the rest. I'll
just wait for a reply to the ones I've already mentioned.
And they key idea here is: don't forget the little people and how they
work. Enable/disable status types on a component-by-component basis,
if you have to.
Gordon
Michael Connor
On 31-Mar-09, at 7:59 PM, GPHemsley wrote:
> Indeed, this discussion seems to be heavily biased towards Firefox/
> Gecko, and forgetting all the many different pieces of software that
> use BMO for their bug tracking. The software with smaller communities
> and fewer filed bugs (e.g. Bespin) are better able to use all of these
> status names. I use ASSIGNED to categorize bugs that are actually the
> responsibility of a particular person. Otherwise, I just ignore
> whomever the bug is assigned to (usually nobody), because the bug is
> not actually "assigned".
Yes, and it's not intuitive, or obvious, or transparent. It's opaque
and confusing.
Having an assignee who's not actually assigned to or working on the
bug is one of the weirdest legacies of Netscape-era development
practices. No one's actually given a great explanation of why this is
actually necessary, vs. just not assigning bugs to yourself if you're
not working on them.
> And it's true that REOPENED is (should be) used for bugs that were
> originally thought to be RESOLVED, but later turned out that they
> weren't, for reasons that have already been mentioned: fix backed out,
> fix didn't work, etc.
And what's the value of that status? Once someone changes the owner,
or changes the status, that metadata is gone. If it's worth
preserving, it should be prominent somewhere, regardless of what the
status gets changed to, surely?
> As for READY, in the course of reading this
> discussion, I've already forgotten what it means. Ready for what?
> Triage? Testcase? Fix to be written? Fix to be applied? In my opinion,
> READY is a bad name for a new status.
Ready to fix, i.e. can be picked up and fixed by anyone with the
skills necessary.
> And they key idea here is: don't forget the little people and how they
> work. Enable/disable status types on a component-by-component basis,
> if you have to.
If the "little people" want to use a confusing, counter-intuitive
workflow, they can run their own bugzilla instance, IMO. Or the
Bugzilla people can add per-component workflows. We should not allow
the needs of the many to continue to suffer because some people are
resistant to change. We need to solve problems because the old
process simply doesn't scale to meet our needs, and makes it hard to
grow to meet our volume.
-- Mike
Shawn Wilsher
> If the "little people" want to use a confusing, counter-intuitive
> workflow, they can run their own bugzilla instance, IMO. Or the Bugzilla
> people can add per-component workflows. We should not allow the needs of
> the many to continue to suffer because some people are resistant to
> change. We need to solve problems because the old process simply doesn't
> scale to meet our needs, and makes it hard to grow to meet our volume.
"So say we all."
/sdwilsh
Michael Connor
things around:
Problems:
* The current workflow is not always obvious, especially bug statuses.
* Assigned To vs. ASSIGNED is confusing. For some people, Assigned To
+ NEW doesn't actually mean they're doing anything with the bug.
Around 80% of FIXED bugs go directly from NEW to RESOLVED, so this is
an increasingly minor view, and a legacy of the old system of default
assignees per component.
* REOPENED is a strange state which doesn't have a lot of perceived
value.
* NEW is overloaded, and there is no status showing when a bug is
ready to be fixed by a developer.
Objectives
* Make it easier to get people with less technical/domain expertise
helping with triage.
* Focus domain expert QA on bugs that can be reproduced.
* Make it easier for developers to pick up a bug and fix it.
* (Needs Feedback): Make it clear when work is actually happening.
Draft thinking:
New flow:
UNCONFIRMED
All bugs go into a single bucket, first pass triage makes sure bug is
actually a bug, not an obvious dupe, and can be reproduced. RFE's
will just get confirmed.
NEW
Bugs that need testcases, regression windows, evaluation against
standards, etc. Basically "yes, this is bug, but it's not clear
whether/how to fix it"
READY
Bugs that have the requirements in place to start work.
INPROGRESS
The bug owner is working on a fix.
RESOLVED
The fix has landed and the bug is resolved.
GPHemsley
At no point did I intend to imply that Bugzilla or BMO should not
change to increase productivity and streamline and simply the
interface. I just want to make sure that these decisions are made with
the fact in mind that Firefox and Gecko are not the only pieces of
software that are using the interface.
I don't think that there's any doubt that turning on the ability to
change bugs from NEW back to UNCONFIRMED will help everybody. I think
there should also be the option to bypass the REOPENED status
(assuming it's kept), especially from the dead-end (rather than finish-
line) resolutions, like INVALID or WONTFIX.
And it probably would be good to tie the ASSIGNED status and assignee
field together, and eliminate that annoying and distracting
"nob...@mozilla.org" assignee. Perhaps it would be good to give
resolutions to the REOPENED status, too? Such as BACKEDOUT or WRONGFIX
or something?
As for assigning a bug to yourself just to keep track of it, isn't
that what CCing is for?
The fact that I needed you to explain to me what "READY" meant means
that it's not a good a name. Perhaps READYTOFIX is a better choice, if
you're going to go with that option? I think it's a good idea to
indicate the different steps in a bug's life, as there are certainly
some that are missing.
But I don't think it would be useful to do a one-for-one switch, or to
worry about whether everyone would use every possible status.
Different people or components will do different things, whatever's
best for them. That's what I meant when I said not to forget about the
little people.
Gordon