Triage workflow proposal
Michael Connor
we handle triage into a separate thread, since I think what makes the
most sense here requires code changes, while the post-confirmation
workflow can be done via the admin interface.
Assertions:
* Most details we ask for are irrelevant to users, and there is a
considerable amount of complexity overhead that, often as not, gets
ignored or set incorrectly.
* Half of all bugs filed are dupes.
* Many more are INVALID or WORKSFORME.
* We struggle to keep up with triage, and reducing overhead/
duplication of effort are important goals in getting on top of things.
* If a request for info in an UNCO bug is not answered within two
weeks, the odds of a response fall to almost nil.
Concept:
* Reduce no-privs bug filing to be a simple form reporting a bug in a
software product and push all UNCO bugs into a component per product.
** This means Summary and Description only. We'll still capture the
other info, but not in a user-visible way.
** I would actually have these bugs live in a Triage product, so that
flags can't be set until the bug is at least CONFIRMED.
* Add a NEEDINFO status for bugs requiring more information to
confirm. Added comments would automatically bump this back to UNCO.
* NEEDINFO bugs get auto-resolved to RESOLVED INCOMPLETE after two
weeks of inactivity.
* Triagers ignore NEEDINFO, and only work through UNCO bugs.
* Bugs that can be reproduced and aren't obviously DUPLICATE or
INVALID get marked CONFIRMED and moved to the correct component for
deeper QA.
I would, for example, mass-bump everything currently UNCO in Firefox
to NEEDINFO once we have this, since some anecdotal triage has shown
me that most of these are either dupes or obsoleted by new code
changes. Other module owners would be free to make their own calls on
how to deal with the legacy UNCO bugs.
Caveats:
* This is assumed to be lossy. Some number of real bugs will get
resolved prematurely due to lack of info. But we currently have 15k
UNCO bugs in Core/Toolkit/Firefox, some dating to 2002, so I think
there's little risk of making the situation worse by going faster.
Most/all important bugs get multiple reports anyway (if only one
person cares enough to file it, it's predominantly an edge case or
incredibly trivial).
* We'll need to invest a bit in the automatic status change on comment
and the auto-resolver (automatic nightly process, I would imagine).
Thoughts? Flames?
-- Mike
AC
> * Reduce no-privs bug filing to be a simple form reporting a bug in a
> software product and push all UNCO bugs into a component per product.
> ** This means Summary and Description only. We'll still capture the
> other info, but not in a user-visible way.
Is this plan limited to the browser products only, not all products?
Useragent only indicates the user's browser version, irrelevant for bugs
filed by Thunderbird or platform users. Does it provide specific
enough info for all variants of unix/linux/etc. and their window
managers? How about fennec platforms?
Is this replacing the guided bug form for the browser products?
https://bugzilla.mozilla.org/enter_bug.cgi?product=Firefox&format=guided
Simplifying as much as hendrix (the feedback web form that feeds into
the mozilla.feedback newsgroups) might be too much.
http://hendrix.mozilla.org/
(One difference might be getting a working reporter email address.)
The URL, reproduciblity, Steps to reproduce, Actual results, and
Expected results still seem helpful for guiding people to write a useful
bug report. Will users be asked to search for duplicates first?
Gervase Markham
Thanks for this - splitting it out makes sense.
On 06/04/09 19:51, Michael Connor wrote:
> So, rather than block on "one big plan" I'd like to split off the way we
> handle triage into a separate thread, since I think what makes the most
> sense here requires code changes, while the post-confirmation workflow
> can be done via the admin interface.
OK. Once we've defined perfection, we'll also want to think about how
close we can get to it without code changes.
> * Most details we ask for are irrelevant to users, and there is a
> considerable amount of complexity overhead that, often as not, gets
> ignored or set incorrectly.
I think we need to distinguish a bit more here. If there's stuff we
almost always need, and which is usually not provided without prompting,
it makes sense to ask for it up front rather than have an
almost-inevitable trip through NEEDINFO.
Before we get to filing, we need to divert broken web pages to Reporter,
and do some dupe-finding. The Launchpad model is quite good. It takes a
"bug summary", does dupe-finding work with it, and then gives you the
form to file the rest of the bug. There's a SoC project to implement a
dupe-finding algorithm.
Anyway, looking at the fields in Bugzilla Helper, I think we should
still ask for the following:
* Product
* Build ID or equivalent (for non-browsers)
* Summary
* Steps To Reproduce (What did you do?)
* Actual Results (What did you expect?)
* Expected Results (What actually happened?)
* Security flag (I checked; a significant number of security-sensitive
bugs do come in through the Bugzilla Helper)
I assert that the formal breakdown of the description into those three
pieces produces better bug reports, and the trade-off in additional
complexity for reporters is more than offset by the reduced number of
bugs which will need a pass through NEEDINFO and reduced QA time in
writing up what's needed and putting them into and out of that state.
Get the following automatically from the user agent:
* OS
* Platform
* Build ID (for browsers)
And not ask for the following:
* Component
* Severity
* Reproducibility
* URL (hopefully they'll give it if it's relevant)
> * Half of all bugs filed are dupes.
> * Many more are INVALID or WORKSFORME.
> * We struggle to keep up with triage, and reducing overhead/duplication
> of effort are important goals in getting on top of things.
> * If a request for info in an UNCO bug is not answered within two weeks,
> the odds of a response fall to almost nil.
All of these I can believe. I assume you have stats? :-)
> * Reduce no-privs bug filing to be a simple form reporting a bug in a
> software product and push all UNCO bugs into a component per product.
> ** This means Summary and Description only. We'll still capture the
> other info, but not in a user-visible way.
See above. And of course we can't capture some other info
programmatically for non-browser products.
> ** I would actually have these bugs live in a Triage product, so that
> flags can't be set until the bug is at least CONFIRMED.
The fact that moving bugs between products in Bugzilla is not as slick
as it might be may mean we have to use social controls here.
> * Add a NEEDINFO status for bugs requiring more information to confirm.
> Added comments would automatically bump this back to UNCO.
Interesting idea. This could get messed up by someone doing a mass
change or something, though.
> * NEEDINFO bugs get auto-resolved to RESOLVED INCOMPLETE after two weeks
> of inactivity.
Is there any risk in making it four weeks? People go on holidays for two
weeks. If they are in a separate state, we don't need to be
extra-aggressive.
> * This is assumed to be lossy. Some number of real bugs will get
> resolved prematurely due to lack of info.
Yes. I think getting over any fear of this happening is vital.
Gerv
David Tenser
> Assertions:
>
> * Half of all bugs filed are dupes.
> * Many more are INVALID or WORKSFORME.
That leaves us with 0 actual bugs, yay! ;)
>
> Concept:
>
> * Reduce no-privs bug filing to be a simple form reporting a bug in a
> software product and push all UNCO bugs into a component per product.
> ** This means Summary and Description only. We'll still capture the
> other info, but not in a user-visible way.
I'll echo Gerv's thoughts here. Not asking for steps to reproduce a bug
will probably just result in lower quality bug reports, which wouldn't
improve the current triage workflow since most bugs would be marked as
NEEDINFO.
It might be a good idea to ask for the general type of problem being
reported, e.g.:
* Crash -> automatically set the severity
* Stops responding / freeze -> automatically set severity
* Other type of problem -> do nothing (default triage component/severity)
In other words, there might be a way to make it easier for users to
classify/categorize the type of problem they're experiencing without
actually asking for components, severity, and platform.
> * Add a NEEDINFO status for bugs requiring more information to confirm.
> Added comments would automatically bump this back to UNCO.
> * NEEDINFO bugs get auto-resolved to RESOLVED INCOMPLETE after two weeks
> of inactivity.
Two weeks feels a bit too aggressive; I'd rather choose something like
two months unless we can demonstrate that a deadline of two weeks does a
better job of improving the triage workflow with an acceptable level of
actual bug "dataloss."
> Thoughts? Flames?
Overall, I really like these ideas and think they can help a great deal.
- David
GPHemsley
> The URL, reproduciblity, Steps to reproduce, Actual results, and
> Expected results still seem helpful for guiding people to write a useful
> bug report. Will users be asked to search for duplicates first?
Didn't Bugzilla used to have that feature? What happened to it?
Zack Weinberg
I don't think users should be asked to search for duplicates. It's not
a thing that can be done reliably without domain knowledge, so it's
just another incomprehensible hoop we are making people jump through.
zw
John J. Barton
I think asking users to search for duplicates is a waste of everyone's
time. The vocabulary for describing problems is too large and the
bugzilla engine is too slow. Instead, I think everyone would benefit
from guiding people to a common vocabulary in their reports. Ask bug
reporters to give the exact text of error messages, exact titles on
buttons they used, and develop some documentation pages with some common
vocabulary. The bug report UI could have screen shots of the UI with
"tell me where it hurts" so users walk through the steps to reproduce
the problem.
jjb
Joshua Cranmer
> On 09-04-06 20.51, Michael Connor wrote:
>> Assertions:
>>
>> * Half of all bugs filed are dupes.
>> * Many more are INVALID or WORKSFORME.
>
> That leaves us with 0 actual bugs, yay! ;)
The actual statistics would come out to be somewhere around, "half of
all bugs filed are not bugs, most of which are dupes."
>> * Reduce no-privs bug filing to be a simple form reporting a bug in a
>> software product and push all UNCO bugs into a component per product.
>> ** This means Summary and Description only. We'll still capture the
>> other info, but not in a user-visible way.
>
> I'll echo Gerv's thoughts here. Not asking for steps to reproduce a bug
> will probably just result in lower quality bug reports, which wouldn't
> improve the current triage workflow since most bugs would be marked as
> NEEDINFO.
Pretty much every new bug I've seen filed by non-expert users has had to
have a fair amount of QA to be able to produce basic steps to reproduce,
since (at least for TB reports), the average STR look like:
"Open window. Type X in. Fail." A lot of the basic steps generally
involve the same basic questions, so a much more guided bug entry form
would be wonderful.
>> * Add a NEEDINFO status for bugs requiring more information to confirm.
>> Added comments would automatically bump this back to UNCO.
>> * NEEDINFO bugs get auto-resolved to RESOLVED INCOMPLETE after two weeks
>> of inactivity.
>
> Two weeks feels a bit too aggressive; I'd rather choose something like
> two months unless we can demonstrate that a deadline of two weeks does a
> better job of improving the triage workflow with an acceptable level of
> actual bug "dataloss."
Judging from my experience and passive observation with closemes in
Thunderbird, most people will either be responding to the bug within 48
hours of the request for information, 24 hours after being closed, or
never. We generally hold to a rule of 2-3 weeks before resolving bugs as
INCO with the closeme system. Two weeks seems a bit short on time--I'm
more comfortable with three--but two months (eight weeks) seems way too
long.
Mike Shaver
<johnj...@johnjbarton.com> wrote:
>> Didn't Bugzilla used to have that feature? What happened to it?
>
> I think asking users to search for duplicates is a waste of everyone's time.
> The vocabulary for describing problems is too large and the bugzilla engine
> is too slow.
Yeah, I agree. I think we could do keyword matching on our top N
keywords (updated daily to a static JSON file, and matched on the
client) to help people find high-confidence dups, but making them run
and understand the search is a huge fail sandwich IMO.
If people _do_ identify likely dups from the client-side keyword
matching, we should just add the "maybe dup of: bug 1234 (print menu),
bug 12345 (print preview), bug 123456 (print crash)" to the tail of
their submission to ease the life of triagers. False dups are worse
than missed dups, I would say.
Mike
Boris Zbarsky
> False dups are worse than missed dups, I would say.
Fully agreed, especially when people dup to what are essentially metabugs.
I've said this often in performance bugs: don't dup unless you have data
(e.g. a profile) indicating that it's the same issue, or unless the url
and steps to reproduce are the same. If you think it's the same thing,
mark dependent; when the other bug is fixed this one can be retested.
-Boris
chris hofmann
John J. Barton wrote:
> GPHemsley wrote:
>> On Apr 6, 8:47 pm, AC <u...@domain.invalid> wrote:
>>> The URL, reproduciblity, Steps to reproduce, Actual results, and
>>> Expected results still seem helpful for guiding people to write a
>>> useful
>>> bug report. Will users be asked to search for duplicates first?
>>
>> Didn't Bugzilla used to have that feature? What happened to it?
>
I think you have described two or maybe three different problems here.
> I think asking users to search for duplicates is a waste of everyone's
> time. The vocabulary for describing problems is too large and the
> bugzilla engine is too slow.
"users" (bug triagers) search for an duplicates now. that's how bugs
get marked dulicate.
I'm sure there are some simple searching tricks that triage people are
using to find dupes quickly, and probably some more complex ones as
well. We should get some people to speak up with these tricks, then
see if there are some ways to automated them.
think about a process like this, where the first round of the bug entry
is more like a "preview mode"
1. user submits a bug just like in the process that is used now.
1a. user enters bug information
1b. hits submit
2. apply some simple automated search tricks on title and/or body of the
submission.
search might be limited to last thousand or several thousand bugs
to help speed things up.
a corpus of older frequent duped bugs could be added in.
then show:
3a. a list of 4 or 5 possible recent dupes to check out and add more
information too
3b. the body of their bug summission
3c. a [go ahead and submit] button
if user didn't see anything in the titles or a deeper probe into the
recent dup links they would go ahead with step 3c and the bug gets
submitted.
> Instead, I think everyone would benefit from guiding people to a
> common vocabulary in their reports. Ask bug reporters to give the
> exact text of error messages, exact titles on buttons they used, and
> develop some documentation pages with some common vocabulary. The bug
> report UI could have screen shots of the UI with "tell me where it
> hurts" so users walk through the steps to reproduce the problem.
Rather than "Instead of" I think this point is "In addtion to." If
the vocabulary going into step 1 and 2 above gets more concise and
focused the search results get better and the automation for finding
dupes gets faster and more efficient.
The 3 step process above is definitely not going to find all the dupes,
or maybe even a high pct.
The key parts to this are that the annoyance factor for filing a bug
needs to be keep small, and needs to be improved over time. What if
this caught only 10-20% of incoming dupes? That might be a big win for
bug duping triage people and free up more time to work on the harder
duping and triaging problems.
-chofmann
>
> jjb
>
> _______________________________________________
> dev-planning mailing list
> dev-pl...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-planning
Robert Kaiser
> Get the following automatically from the user agent:
> * OS
> * Platform
> * Build ID (for browsers)
How does this work for e.g. Thunderbird, where people don't file the bug
using the product itself?
>> * Add a NEEDINFO status for bugs requiring more information to confirm.
>> Added comments would automatically bump this back to UNCO.
>
> Interesting idea. This could get messed up by someone doing a mass
> change or something, though.
I think Novell/openSuSE, who have that state, actually have a checkbox
under the comment field that says "this comment provides the info
requested" or so, which when checked makes it go back to NEW in their
case, IIRC (most intelligent would be if it would go back to whatever
state the bug was in before it was NEEDINFO). We probably could use a
solution like that.
Robert Kaiser
Robert Kaiser
> * Reduce no-privs bug filing to be a simple form reporting a bug in a
> software product and push all UNCO bugs into a component per product.
This sounds like a good idea, but please have a pref in user settings to
make them use the normal bug filing form. Again and again we have people
who're even working on bug and doing good stuff but they don't even have
canconfirm yet. They should have an option to at least file bugs against
the right components and with the right settings, even though it
shouldn't be the default.
> I would, for example, mass-bump everything currently UNCO in Firefox to
> NEEDINFO once we have this, since some anecdotal triage has shown me
> that most of these are either dupes or obsoleted by new code changes.
I would go as far as even thinking about 1) doing this for all of BMO
and 2) doing the same for very old NEW bugs (at least if they are from
before the point where we made it the default that unprivileged users
could only file UNCO bugs).
Robert Kaiser
Mark Banner
> Gervase Markham wrote:
>> Get the following automatically from the user agent:
>> * OS
>> * Platform
>> * Build ID (for browsers)
>
> How does this work for e.g. Thunderbird, where people don't file the bug
> using the product itself?
Simple: it doesn't. Could we do a if filing in xxx product ask for
version from About -> Help (or wherever).
I'm pretty sure that the first question most frequently asked for bugs
filed in Thunderbird is "What version are you using?".
Mostly the build identifier field is a) missed off b) wrong (e.g.
Firefox's), c) completely misunderstood. Sometimes it is actually right,
but I think that's a rare occasion.
Standard8
Michael Connor
On 7-Apr-09, at 8:55 AM, David Tenser wrote:
> On 09-04-06 20.51, Michael Connor wrote:
>> Assertions:
>>
>> * Half of all bugs filed are dupes.
>> * Many more are INVALID or WORKSFORME.
>
> That leaves us with 0 actual bugs, yay! ;)
>
>>
>> Concept:
>>
>> * Reduce no-privs bug filing to be a simple form reporting a bug in a
>> software product and push all UNCO bugs into a component per product.
>> ** This means Summary and Description only. We'll still capture the
>> other info, but not in a user-visible way.
>
> I'll echo Gerv's thoughts here. Not asking for steps to reproduce a
> bug will probably just result in lower quality bug reports, which
> wouldn't improve the current triage workflow since most bugs would
> be marked as NEEDINFO.
I didn't say we wouldn't ask for this, but from reading a bunch of
UNCO bugs people seem to get annoyed by some of these questions (i.e.
"Expected Results: Not crash you @#%#$# jerks!") We'll provide
instructions, but it seems like more often than not the guided form
results in weird data instead of simply "tell us exactly how to
reproduce this bug, and provide links to any pages you're talking
about."
> It might be a good idea to ask for the general type of problem being
> reported, e.g.:
>
> * Crash -> automatically set the severity
> * Stops responding / freeze -> automatically set severity
> * Other type of problem -> do nothing (default triage component/
> severity)
I don't think this is helpful enough to make users process and answer
it. I'm trying to lower the bar to reporting info, and the more
fields you add, the less likely a user will fill them all in.
> In other words, there might be a way to make it easier for users to
> classify/categorize the type of problem they're experiencing without
> actually asking for components, severity, and platform.
>
>> * Add a NEEDINFO status for bugs requiring more information to
>> confirm.
>> Added comments would automatically bump this back to UNCO.
>> * NEEDINFO bugs get auto-resolved to RESOLVED INCOMPLETE after two
>> weeks
>> of inactivity.
>
> Two weeks feels a bit too aggressive; I'd rather choose something
> like two months unless we can demonstrate that a deadline of two
> weeks does a better job of improving the triage workflow with an
> acceptable level of actual bug "dataloss."
We're not going to get data on this without doing this, of course,
since we're talking about a rather different process than we've
followed in the past. I'm not sure why you'd ask for proof of
something that we can't actually prove without doing it. We can make
educated guesses, but we can't demonstrate something we've never done
will work a certain way. We can always tweak upwards if people are
hitting this too often and filing new bugs.
One key advantage of two weeks is that the reporter will get mail
while it's still closer to their memory. If we waited two months,
odds are that the user has already given up. I don't think the
timeframe matters much other than this, since NEEDINFO means we're not
looking at the bug again until we get details from someone.
-- Mike
Michael Connor
On 7-Apr-09, at 3:16 PM, Mark Banner wrote:
> On 7/4/09 19:32, Robert Kaiser wrote:
>> Gervase Markham wrote:
>>> Get the following automatically from the user agent:
>>> * OS
>>> * Platform
>>> * Build ID (for browsers)
>>
>> How does this work for e.g. Thunderbird, where people don't file
>> the bug
>> using the product itself?
>
> Simple: it doesn't. Could we do a if filing in xxx product ask for
> version from About -> Help (or wherever).
Yeah, Version/OS is probably easier to expose for those products, and
we can at least pre-select OS in most cases.
-- Mike
Michael Connor
On 7-Apr-09, at 3:16 PM, Mark Banner wrote:
> On 7/4/09 19:32, Robert Kaiser wrote:
>> Gervase Markham wrote:
>>> Get the following automatically from the user agent:
>>> * OS
>>> * Platform
>>> * Build ID (for browsers)
>>
>> How does this work for e.g. Thunderbird, where people don't file
>> the bug
>> using the product itself?
>
> Simple: it doesn't. Could we do a if filing in xxx product ask for
> version from About -> Help (or wherever).
Yeah, Version/OS is probably easier to expose for those products, and
Michael Connor
On 7-Apr-09, at 2:38 PM, Robert Kaiser wrote:
> Michael Connor wrote:
>> * Reduce no-privs bug filing to be a simple form reporting a bug in a
>> software product and push all UNCO bugs into a component per product.
>
> This sounds like a good idea, but please have a pref in user
> settings to make them use the normal bug filing form. Again and
> again we have people who're even working on bug and doing good stuff
> but they don't even have canconfirm yet. They should have an option
> to at least file bugs against the right components and with the
> right settings, even though it shouldn't be the default.
I'd rather just give them canconfirm, is there a reason someone who
can figure out our component setup and bugzilla in general can't have
canconfirm? "has half a clue" is enough for me.
>> I would, for example, mass-bump everything currently UNCO in
>> Firefox to
>> NEEDINFO once we have this, since some anecdotal triage has shown me
>> that most of these are either dupes or obsoleted by new code changes.
>
> I would go as far as even thinking about 1) doing this for all of
> BMO and 2) doing the same for very old NEW bugs (at least if they
> are from before the point where we made it the default that
> unprivileged users could only file UNCO bugs).
I'm willing to be the bad guy for bugs under my module, less so for
the rest of bmo. Plus, I think that some components like Layout
probably would want a chance to shuffle bugs around into the new
buckets.
-- Mike
Frédéric Buclin
> Is this replacing the guided bug form for the browser products?
> https://bugzilla.mozilla.org/enter_bug.cgi?product=Firefox&format=guided
I don't see why you would replace this guided form, as it exists
precisely to help "powerless" users. This page must be improved if
needed, rather than implementing something new.
Also, note that Bugzilla 3.4 has an "easier to use" enter bug form:
https://landfill.bugzilla.org/bugzilla-tip/enter_bug.cgi?product=WorldControl
If you don't have a Bugzilla account on landfill, I can blog about it
and paste a screenshot or two, if you want to.
LpSolit
Robert Kaiser
>
> On 7-Apr-09, at 2:38 PM, Robert Kaiser wrote:
>
>> Michael Connor wrote:
>>> * Reduce no-privs bug filing to be a simple form reporting a bug in a
>>> software product and push all UNCO bugs into a component per product.
>>
>> This sounds like a good idea, but please have a pref in user settings
>> to make them use the normal bug filing form. Again and again we have
>> people who're even working on bug and doing good stuff but they don't
>> even have canconfirm yet. They should have an option to at least file
>> bugs against the right components and with the right settings, even
>> though it shouldn't be the default.
>
> I'd rather just give them canconfirm, is there a reason someone who can
> figure out our component setup and bugzilla in general can't have
> canconfirm? "has half a clue" is enough for me.
The problem here is that it takes some time to even get canconfirm, and
usually it needs some showing of well-done bugs as well. Having already
well-educated people that just happen to have a new account on our
bugzilla (they may e.g. have been working with other projects and bug
reporting system a lot before coming to us) doesn't sound ideal, and
those people we are concerned with when doing the simpler bug entry form
are usually people who don't even look into their prefs a lot.
>>> I would, for example, mass-bump everything currently UNCO in Firefox to
>>> NEEDINFO once we have this, since some anecdotal triage has shown me
>>> that most of these are either dupes or obsoleted by new code changes.
>>
>> I would go as far as even thinking about 1) doing this for all of BMO
>> and 2) doing the same for very old NEW bugs (at least if they are from
>> before the point where we made it the default that unprivileged users
>> could only file UNCO bugs).
>
> I'm willing to be the bad guy for bugs under my module, less so for the
> rest of bmo. Plus, I think that some components like Layout probably
> would want a chance to shuffle bugs around into the new buckets.
I think that we just will have a huge pile of dead bodies lurking around
if we don't go and do this for at least Core, Toolkit, SeaMonkey, and
MailNews Core as well as Firefox, and also include very old NEW bugs
(probably judging by when the last comment on the bug was made).
Robert Kaiser
Clint Talbert
> David Tenser wrote:
>> On 09-04-06 20.51, Michael Connor wrote:
>>> * Reduce no-privs bug filing to be a simple form reporting a bug in a
>>> software product and push all UNCO bugs into a component per product.
>>> ** This means Summary and Description only. We'll still capture the
>>> other info, but not in a user-visible way.
>>
>> I'll echo Gerv's thoughts here. Not asking for steps to reproduce a
>> bug will probably just result in lower quality bug reports, which
>> wouldn't improve the current triage workflow since most bugs would be
>> marked as NEEDINFO.
>
> Pretty much every new bug I've seen filed by non-expert users has had to
> have a fair amount of QA to be able to produce basic steps to reproduce,
> since (at least for TB reports), the average STR look like:
> "Open window. Type X in. Fail." A lot of the basic steps generally
> involve the same basic questions, so a much more guided bug entry form
> would be wonderful.
>
of the time the directed questions on the simple form do give us a
decent set of STR/Expected Behavior/Actual Behavior.
I'd be loath to ask much more than that. I think that guessing version
of product is going to be wrong enough of the time that it makes sense
to ask for it. Both in terms of non-browser products and in terms of
people using version X to file a bug for version Y (because version Y is
crashing or what have you). Perhaps that can be guessed and pre-filled
though so that people don't have to change it. I don't see this really
working in our favor now though and it's the way it currently works IIRC.
Also, I'd recommend we no longer ask new users to find dupes, and we
lose the big display of "bugs filed today" on the page. Another
suggestion we are often asking in triage is "did you try this in safe
mode/ new profile". Should we have that on the form? Or should we have
a place where people can at least list the extensions they had? This
might be too much information, but it is something we ask over and over
in triage.
>>> * Add a NEEDINFO status for bugs requiring more information to confirm.
>>> Added comments would automatically bump this back to UNCO.
>>> * NEEDINFO bugs get auto-resolved to RESOLVED INCOMPLETE after two weeks
>>> of inactivity.
>>
>> Two weeks feels a bit too aggressive; I'd rather choose something like
>> two months unless we can demonstrate that a deadline of two weeks does
>> a better job of improving the triage workflow with an acceptable level
>> of actual bug "dataloss."
>
> Judging from my experience and passive observation with closemes in
> Thunderbird, most people will either be responding to the bug within 48
> hours of the request for information, 24 hours after being closed, or
> never. We generally hold to a rule of 2-3 weeks before resolving bugs as
> INCO with the closeme system. Two weeks seems a bit short on time--I'm
> more comfortable with three--but two months (eight weeks) seems way too
> long.
I like this idea, I recommend that we make the auto-close occur after 3
weeks, which is what we do now with CLOSEME bugs, and this way it won't
be a huge "policy" shift in the way we do triage.
Great thoughts. Thanks Mike and everyone working on these threads.
Clint
Gervase Markham
> Yeah, I agree. I think we could do keyword matching on our top N
> keywords (updated daily to a static JSON file, and matched on the
> client) to help people find high-confidence dups, but making them run
> and understand the search is a huge fail sandwich IMO.
This year, I am mentoring a SoC project for automatic duplicate
detection, server-side. I anticipate doing something launchpad-like -
asking the user "here are five bugs - is yours one of these?".
Gerv
Gervase Markham
> Gervase Markham wrote:
>> Get the following automatically from the user agent:
>> * OS
>> * Platform
>> * Build ID (for browsers)
>
> How does this work for e.g. Thunderbird, where people don't file the bug
> using the product itself?
Most ordinary people have one computer; we assume the OS and Platform of
the machine they filed on are the same as that of the software they are
filing the bug on, unless they tell us otherwise.
For Build ID, as you can see, it says "for browsers". We'll need to ask
for software like Thunderbird, although hopefully we can find a better
name for it.
> I think Novell/openSuSE, who have that state, actually have a checkbox
> under the comment field that says "this comment provides the info
> requested" or so, which when checked makes it go back to NEW in their
> case, IIRC (most intelligent would be if it would go back to whatever
> state the bug was in before it was NEEDINFO). We probably could use a
> solution like that.
I like that. :-)
Gerv
Gervase Markham
> This sounds like a good idea, but please have a pref in user settings to
> make them use the normal bug filing form. Again and again we have people
> who're even working on bug and doing good stuff but they don't even have
> canconfirm yet. They should have an option to at least file bugs against
> the right components and with the right settings, even though it
> shouldn't be the default.
The barrier to getting canconfirm is pretty low - in practice, it's "a
couple of sane and helpful bugs or comments". Please encourage people to
request it.
> I would go as far as even thinking about 1) doing this for all of BMO
We'd need sign-off from some of the more independently-minded projects ;-)
> and 2) doing the same for very old NEW bugs (at least if they are from
> before the point where we made it the default that unprivileged users
> could only file UNCO bugs).
I suspect that previous attempts to clear out old bugs have dealt with
most of those. BICBW.
Gerv
Mike Shaver
> The barrier to getting canconfirm is pretty low - in practice, it's "a
> couple of sane and helpful bugs or comments". Please encourage people to
> request it.
Could bugzilla suggest it, after a certain number of comments have
been left or bugs filed? It could even tell them how to request it,
and prefab a link to their comments and bugs for email/entering like a
review request/etc..
Mike
Mike Connor
lot cooler/scalable.
-- Mike
L. David Baron
> On 07/04/09 19:38, Robert Kaiser wrote:
>> This sounds like a good idea, but please have a pref in user settings to
>> make them use the normal bug filing form. Again and again we have people
>> who're even working on bug and doing good stuff but they don't even have
>> canconfirm yet. They should have an option to at least file bugs against
>> the right components and with the right settings, even though it
>> shouldn't be the default.
>
> The barrier to getting canconfirm is pretty low - in practice, it's "a
> couple of sane and helpful bugs or comments". Please encourage people to
> request it.
The barrier of not knowing that one can request it is actually
probably quite high. Many bug reporters who file useful bugs
probably don't know that they could request it.
-David
--
L. David Baron http://dbaron.org/
Mozilla Corporation http://www.mozilla.com/
Robert Kaiser
Looking at the ancient SeaMonkey and MailNews Core bugs we have around,
I'm not so sure about that. If I suggest something like that, I have
concrete stuff in mind that triggers the suggestion ;-)
Robert Kaiser
avatr...@gmail.com
> > This year, I am mentoring a SoC project for automatic duplicate
> > detection, server-side. I anticipate doing something launchpad-like -
> > asking the user "here are five bugs - is yours one of these?".
>
> Is that the best UI? Doing some slick client-side stuff would seem a
> lot cooler/scalable.
Well, we don't have scaling problems, and it seems to work just fine
in Launchpad.
-Max
Michael Connor
That's not really arguing for it being the best UI. We shouldn't
settle for "just fine" for things we're building.
As for scaling, put this another way: any search that's fuzzy enough
to be useful probably isn't going to be returning in less than five
seconds, where something like shaver's precalculated JSON object of
keywords would be just a matter of loading the JSON bits with the bug
form. I'd rather go with the client-side solution that doesn't suck
up DB server cycles than the server-side solution which is slower and
puts more load on the DB.
Of course, when I say it _that_ way, it sounds ridiculous to do it the
other way...
-- Mike
John J. Barton
...
> As for scaling, put this another way: any search that's fuzzy enough to
> be useful probably isn't going to be returning in less than five
> seconds, where something like shaver's precalculated JSON object of
> keywords would be just a matter of loading the JSON bits with the bug
> form.
I sent Gerv a long-winded version of the following suggestion:
If the precalculated JSON object of keywords were extracted from the
product UI (rather than bugzilla), then users could approximately
simulate their actions, triage would be way ahead on localizing the
problem area, and developers could have at least some reproducible
steps. Imagine a schematic 'where does it hurt', users see text they
know ("file..open", not "core..xpcom"). Could be applied to API also.
jjb
Jean-Marc Desperrier
>> False dups are worse than missed dups, I would say.
>
> Fully agreed, especially when people dup to what are essentially metabugs.
>
> I've said this often in performance bugs: don't dup unless you have data
> (e.g. a profile) indicating that it's the same issue
But that problem lies more with some of the most active triagers than
with random bug reporters.
Michael Connor
That just means it's easier to talk to the people doing it!
-- Mike
Gervase Markham
> Could bugzilla suggest it, after a certain number of comments have
> been left or bugs filed? It could even tell them how to request it,
> and prefab a link to their comments and bugs for email/entering like a
> review request/etc..
Good idea.
https://bugzilla.mozilla.org/show_bug.cgi?id=487791
Gerv