OK, so let's completely rejig the Bugzilla workflow instead :-)) Let me try and synthesize where we have got to so far. Mike suggested the following names and states, with other people's suggested names for each state in brackets afterwards. This fits pretty well with what Max (who does this for a living) suggested too. I've made a few tweaks and expanded it a bit.
UNCONFIRMED Most bugs go into a single bucket, first pass triage makes sure bug is actually a bug, not an obvious dupe, and can be reproduced. RFEs will just get confirmed. Trusted contributors bypass this state.
CONFIRMED (OPEN; formerly NEW) Bugs that need testcases, regression windows, evaluation against standards, etc. Basically "yes, this is bug, but it's not clear whether/how to fix it".
READYTOFIX (READY) Bugs that have the requirements in place to start work.
INPROGRESS (ACCEPTED, FIXINPROGRESS; created by renaming ASSIGNED) The bug owner is working on a fix.
RESOLVED The fix has landed.
VERIFIED QA has confirmed that the fix has solved the problem.
We also currently have a CLOSED state. Hardly anyone uses it - only 1629 bugs (0.3%), and none in the most recent 150,000, are in that state. We should move them all back to VERIFIED and eliminate it, along with REOPENED.
Transitions: transition between all states would be allowed, with the exception that VERIFIED can only be reached from RESOLVED, and INPROGRESS and READYTOFIX can't go (directly) back to UNCONFIRMED.
An additional option would be to start using underscores - READY_TO_FIX, IN_PROGRESS - for readability.
> READYTOFIX (READY) > Bugs that have the requirements in place to start work.
> INPROGRESS (ACCEPTED, FIXINPROGRESS; created by renaming ASSIGNED) > The bug owner is working on a fix.
My concern with having both of these states is that we've reproduced, under another name, the has-an-assignee/is-in-an-assigned-state repetition. Or to put it another way, what does it mean if a bug is READY but has an assignee, or INPROGRESS but does not have one?
Should these two states be merged?
Other than that, I think the proposal is excellent. :-P
> We also currently have a CLOSED state. Hardly anyone uses it - only 1629 > bugs (0.3%), and none in the most recent 150,000, are in that state. We > should move them all back to VERIFIED and eliminate it, along with > REOPENED.
Ah - just noticed that it's disabled (in that no transitions to it are valid) on b.m.o. anyway. I don't know if it's still worth removing it. And perhaps this is the way to eliminate REOPENED.
On Wed, Apr 1, 2009 at 3:19 PM, Gervase Markham <g...@mozilla.org> wrote: > OK, so let's completely rejig the Bugzilla workflow instead :-)) Let me try > and synthesize where we have got to so far. Mike suggested the following > names and states, with other people's suggested names for each state in > brackets afterwards. This fits pretty well with what Max (who does this for > a living) suggested too. I've made a few tweaks and expanded it a bit.
> UNCONFIRMED > Most bugs go into a single bucket, first pass triage makes sure bug > is actually a bug, not an obvious dupe, and can be reproduced. RFEs > will just get confirmed. Trusted contributors bypass this state.
> CONFIRMED (OPEN; formerly NEW) > Bugs that need testcases, regression windows, evaluation against > standards, etc. Basically "yes, this is bug, but it's not clear > whether/how to fix it".
That's not my understanding of what the NEW state is. A NEW bug can very well have a testcase and a regression window, etc. And it might also be very well known how it should be fixed.
> READYTOFIX (READY) > Bugs that have the requirements in place to start work. > INPROGRESS (ACCEPTED, FIXINPROGRESS; created by renaming ASSIGNED) > The bug owner is working on a fix.
> RESOLVED > The fix has landed.
> VERIFIED > QA has confirmed that the fix has solved the problem.
> We also currently have a CLOSED state. Hardly anyone uses it - only 1629 > bugs (0.3%), and none in the most recent 150,000, are in that state. We > should move them all back to VERIFIED and eliminate it, along with REOPENED.
> Transitions: transition between all states would be allowed, with the > exception that VERIFIED can only be reached from RESOLVED, and INPROGRESS > and READYTOFIX can't go (directly) back to UNCONFIRMED.
> An additional option would be to start using underscores - READY_TO_FIX, > IN_PROGRESS - for readability.
> OK, so let's completely rejig the Bugzilla workflow instead :-)) Let me > try and synthesize where we have got to so far. Mike suggested the > following names and states, with other people's suggested names for each > state in brackets afterwards. This fits pretty well with what Max (who > does this for a living) suggested too. I've made a few tweaks and > expanded it a bit.
> UNCONFIRMED > Most bugs go into a single bucket, first pass triage makes sure bug > is actually a bug, not an obvious dupe, and can be reproduced. RFEs > will just get confirmed. Trusted contributors bypass this state.
> CONFIRMED (OPEN; formerly NEW) > Bugs that need testcases, regression windows, evaluation against > standards, etc. Basically "yes, this is bug, but it's not clear > whether/how to fix it".
> READYTOFIX (READY) > Bugs that have the requirements in place to start work.
> INPROGRESS (ACCEPTED, FIXINPROGRESS; created by renaming ASSIGNED) > The bug owner is working on a fix.
> RESOLVED > The fix has landed.
> VERIFIED > QA has confirmed that the fix has solved the problem.
> We also currently have a CLOSED state. Hardly anyone uses it - only 1629 > bugs (0.3%), and none in the most recent 150,000, are in that state. We > should move them all back to VERIFIED and eliminate it, along with > REOPENED.
> Transitions: transition between all states would be allowed, with the > exception that VERIFIED can only be reached from RESOLVED, and > INPROGRESS and READYTOFIX can't go (directly) back to UNCONFIRMED.
> An additional option would be to start using underscores - READY_TO_FIX, > IN_PROGRESS - for readability.
> Gerv
Now I'm happy again. This is precisely what should be done. (Though the descriptions may be changed, this is the perfect list of statuses.) I'm not sure about the underscores, but I won't care, as long as these new names are used.
Is it possible to disable changing the assignee from nobody@ to something else without changing the status to INPROGRESS? That would prevent them from going out of sync when somebody changes the assignee, and would eliminate the need to worry about it.
Also, I agree with the proposal to disable REOPENED in the way that CLOSED has been. Then you can take your time in transitioning bugs to the new workflow system.
Gervase Markham wrote: > UNCONFIRMED > Most bugs go into a single bucket, first pass triage makes sure bug > is actually a bug, not an obvious dupe, and can be reproduced. RFEs > will just get confirmed. Trusted contributors bypass this state.
> CONFIRMED (OPEN; formerly NEW) > Bugs that need testcases, regression windows, evaluation against > standards, etc. Basically "yes, this is bug, but it's not clear > whether/how to fix it".
> READYTOFIX (READY) > Bugs that have the requirements in place to start work.
1. What percentage of bugs require the 'confirm -> ready' step?
Regression windows are only for regression bugs. (ask with keyword regressionwindow-wanted)
Testcases are primarily for bugs about parsing or displaying user input files (xul, html, css, ecmascript, etc.); many bugs are not (UX, performance, etc.). (ask with keyword testcase-wanted)
'evaluation against standards' is primarily for bugs about parsing or displaying user input files (standard html, css, ecmascript, etc.); many bugs are not (UX, performance, etc.).
2. Is 'evaluation against standards' usually something that triage/QA can do, or does it require developer/manager expertise?
(Similarly, in general RESOLVED INVALID is something triage can do, but only the lead developer/manager can decide RESOLVED WONTFIX.)
3. Can a triager advance a bug straight to 'READYTOFIX' if it doesn't have any of the above requirements? Would 'canconfirm' privilege disallow this?
> Testcases are primarily for bugs about parsing or displaying user input > files (xul, html, css, ecmascript, etc.); many bugs are not (UX, > performance, etc.). (ask with keyword testcase-wanted)
This isn't right. In general, a bug should still have a litmus test or automated test. API bugs (which I see a lot of), need these types of tests.
> On Wed, Apr 1, 2009 at 3:19 PM, Gervase Markham <g...@mozilla.org> > wrote: >> OK, so let's completely rejig the Bugzilla workflow instead :-)) >> Let me try >> and synthesize where we have got to so far. Mike suggested the >> following >> names and states, with other people's suggested names for each >> state in >> brackets afterwards. This fits pretty well with what Max (who does >> this for >> a living) suggested too. I've made a few tweaks and expanded it a >> bit.
>> UNCONFIRMED >> Most bugs go into a single bucket, first pass triage makes sure bug >> is actually a bug, not an obvious dupe, and can be reproduced. RFEs >> will just get confirmed. Trusted contributors bypass this state.
>> CONFIRMED (OPEN; formerly NEW) >> Bugs that need testcases, regression windows, evaluation against >> standards, etc. Basically "yes, this is bug, but it's not clear >> whether/how to fix it".
> That's not my understanding of what the NEW state is. A NEW bug can > very well have a testcase and a regression window, etc. > And it might also be very well known how it should be fixed.
Yes, that's what I meant all those times I said "NEW is overloaded and should be split up." We're proposing replacing NEW with CONFIRMED and READYTOFIX because those are two distinct states that currently both fall under NEW.
> We also currently have a CLOSED state. Hardly anyone uses it - only 1629 > bugs (0.3%), and none in the most recent 150,000, are in that state. We > should move them all back to VERIFIED and eliminate it, along with > REOPENED.
Ok, maybe I missed it. But what happens when something is RESOLVED (i.e. the fix has landed) and then we discover that the fix is incorrect (tests fail or what have you). Then we'd move it to what state?
That's what we would normally use REOPENED for (which is not a good name), but it seems that we should have a state for "things that were attempted but failed to work". Or we should at teh very least call out what that transition looks like.
On Wed, Apr 1, 2009 at 9:40 PM, Clint Talbert <ctalb...@mozilla.com> wrote: > Ok, maybe I missed it. But what happens when something is RESOLVED (i.e. the > fix has landed) and then we discover that the fix is incorrect (tests fail > or what have you). Then we'd move it to what state?
> That's what we would normally use REOPENED for (which is not a good name), > but it seems that we should have a state for "things that were attempted but > failed to work". Or we should at teh very least call out what that > transition looks like.
There are lots of bugs that will have things that failed to work, on the developer's own machine or on the try server or on a parallel development clone. I don't think that the case where they've been committed to m-c and then backed out of there warrants a special state. I think that transition should look like RESOLVED -> NEW with a comment detailing the reason.
(I also thing that the single-resolved-state pigeonholing hurts us, but one miracle at a time. :) )
On Apr 1, 9:19 am, Gervase Markham <g...@mozilla.org> wrote:
> OK, so let's completely rejig the Bugzilla workflow instead :-))
Hi,
Things like REOPENED are actually pretty useful to tell when a bug wasn't actually fixed. I strongly suggest sitting down with some of the people who live in bugzilla daily, doing triage, before making any changes. Last time people changed bugzilla pretty much all the changes made bugzilla harder to use for those who used it most.
> On 4/1/09 6:19 AM, Gervase Markham wrote: >> We also currently have a CLOSED state. Hardly anyone uses it - only 1629 >> bugs (0.3%), and none in the most recent 150,000, are in that state. We >> should move them all back to VERIFIED and eliminate it, along with >> REOPENED.
> Ok, maybe I missed it. But what happens when something is RESOLVED (i.e. > the fix has landed) and then we discover that the fix is incorrect > (tests fail or what have you). Then we'd move it to what state?
READYTOFIX, presumably. Or, if it turns out the failure is so bad that it requires an entirely different approach and a total rethink, CONFIRMED.
> That's what we would normally use REOPENED for (which is not a good > name), but it seems that we should have a state for "things that were > attempted but failed to work".
Well, we have the "regression" keyword. And normally this sort of bug is already on someone's radar - the person who tried to fix it first time.
> Things like REOPENED are actually pretty useful to tell when a bug > wasn't actually fixed. I strongly suggest sitting down with some of > the people who live in bugzilla daily, doing triage, before making any > changes.
That's what we're doing here, isn't it?
I suggested adding a single new state; the impetus to rejig the entire workflow came from others, notably mconnor.
I agree that people should temper the forcefulness with which they make their remarks about how the workflow should work based on how much time they spend working in or with Bugzilla.
> Last time people changed bugzilla
Bugzilla changes all the time. Can you be more specific as to which change you mean?
> pretty much all the > changes made bugzilla harder to use for those who used it most.
Gervase Markham wrote: > CONFIRMED (OPEN; formerly NEW) > Bugs that need testcases, regression windows, evaluation against > standards, etc. Basically "yes, this is bug, but it's not clear > whether/how to fix it".
> READYTOFIX (READY) > Bugs that have the requirements in place to start work.
I still don't fully grasp the difference between those two for the vast number of things I'm working on. Would it make sense for CONFIRMED to actually mean what READYTOFIX means here and instead have a NEEDINFO state (like Novell/openSUSE have) which is usually set by developers (potentially also triagers though) who see they need additional info to be able to work on this?
On Thu, Apr 2, 2009 at 12:49 PM, Jonathan Watt <jw...@jwatt.org> wrote: > On 4/1/09 3:19 PM, Gervase Markham wrote: >> An additional option would be to start using underscores - READY_TO_FIX, >> IN_PROGRESS - for readability.
> Please! Anything to make bugzilla look a little more approachable and user > friendly to the newb.
Personally, I find those states confusing and I think it would make bugzilla look less user friendly. Or were you talking about the underscores?
> On Thu, Apr 2, 2009 at 12:49 PM, Jonathan Watt <jw...@jwatt.org> wrote: >> On 4/1/09 3:19 PM, Gervase Markham wrote: >>> An additional option would be to start using underscores - READY_TO_FIX, >>> IN_PROGRESS - for readability. >> Please! Anything to make bugzilla look a little more approachable and user >> friendly to the newb.
> Personally, I find those states confusing and I think it would make > bugzilla look less user friendly. > Or were you talking about the underscores?
I'm talking about the underscores, as was Gerv in that sentence.
> On 02/04/09 04:26, Stuart Parmenter wrote: >> Things like REOPENED are actually pretty useful to tell when a bug >> wasn't actually fixed. I strongly suggest sitting down with some of >> the people who live in bugzilla daily, doing triage, before making any >> changes.
> That's what we're doing here, isn't it?
Goodness, that all sounds rather combative, rereading it. Sorry :-)
Let me try again: I certainly hope to make sure that most (or, hopefully, all) people using bugzilla.mozilla.org regularly think changes we make are improvements.
> I still don't fully grasp the difference between those two for the vast > number of things I'm working on. Would it make sense for CONFIRMED to > actually mean what READYTOFIX means here and instead have a NEEDINFO > state (like Novell/openSUSE have) which is usually set by developers > (potentially also triagers though) who see they need additional info to > be able to work on this?
I think the difference between the two workflows is the question of on whom the responsibility seems to fall to investigate whether the bug is ready.
In the CONFIRMED/READYTOFIX workflow, QA will make sure a CONFIRMED bug has everything it needs (testcase, steps to reproduce etc.), and then push it to READYTOFIX for a developer to pick up.
In the CONFIRMED/NEEDINFO workflow, the bug goes straight to CONFIRMED, and then a developer has to investigate whether they can actually fix it and, if not, push it back to NEEDINFO for a QA person to handle.
The former puts more work on QA, the latter on developers.
If I'm right, then if we are attempting to optimize for developer time, we should prefer a workflow which clearly indicates that the former is what's wanted.
On Thu, Apr 2, 2009 at 10:42 AM, Gervase Markham <g...@mozilla.org> wrote: > Let me try again: I certainly hope to make sure that most (or, hopefully, > all) people using bugzilla.mozilla.org regularly think changes we make are > improvements.
What proportion of changes (weight it however you'd like) are discussed with b.m.o's heaviest users, such as release-drivers and full-time developers, before implementation in our system?
> On Thu, Apr 2, 2009 at 10:42 AM, Gervase Markham<g...@mozilla.org> wrote: >> Let me try again: I certainly hope to make sure that most (or, hopefully, >> all) people using bugzilla.mozilla.org regularly think changes we make are >> improvements.
> What proportion of changes (weight it however you'd like) are > discussed with b.m.o's heaviest users, such as release-drivers and > full-time developers, before implementation in our system?
Let me rephrase and expand upon that slightly :-) The below is not meant to be patronizing.
Bugzilla is under development; bugzilla.mozilla.org will, from time to time, upgrade to a more recent version. This upgrade work is done generally by justdave and mkanat, under the auspices of Mozilla IT, and has little to do with me. The changes that this process brings with it may be viewed in different ways by different users. The Mozilla community can affect these changes before the fact in the normal open source way, by getting involved in the Bugzilla development process (filing bugs, giving feedback etc.). Clearly, as a major user of Bugzilla, Mozilla community feedback is given weight.
But those aren't the changes I meant. Separately from the above, Bugzilla can be customised on a per-instance basis. Such a change may well modify or revert a change from the category above ;-) Some of those customisations and changes come out of processes instigated by me (e.g. the recent Bugzilla component and product reorganization). If we change workflow, that would be another change in this category. For such changes, I personally hope to make sure that all or most b.m.o. users consider them improvements. Which is what I was trying to say above.
So the answer to your question, for the changes I was talking about, is 100% - if the aforesaid release drivers and full-time developers are participating in forums such as mozilla.dev.planning. But that's obviously not all the changes that might be made, so the other answer would be a lot less than 100%. Many more changes are made by the Bugzilla development process than are made by our local customizations.
