New "INCOMPLETE" resolution in Bugzilla
Gervase Markham
In a few days, we plan to create a new "INCOMPLETE" resolution in
Bugzilla, taking advantage of the new "custom resolutions" feature.
This is basically for resolving bugs which have insufficient information
in them. Up to now, INVALID has been used, but semantically that's not
quite right, and it is a bit rude.
The name matches a resolution GNOME are already using; consistency is nice.
Gerv
Christopher Aillon
FWIW, Red Hat uses INSUFFICIENT_DATA, which is more accurate and IMO,
less ambiguous. What's incomplete? The bug report? The work on fixing
the bug?
Mike Shaver
> less ambiguous. What's incomplete? The bug report? The work on fixing
> the bug?
I agree, that would be clearer. "NEEDS INFORMATION" might be clearer still?
Mike
Christopher Aillon
We also have a NEEDINFO state for currently open bugs. It's a
flag-based state where someone can request info from a given user, e.g.
the reporter, or a module owner, et al. Typically a bug goes to
NEEDINFO first to request the information be provided, and after a
reasonable time has passed, it gets resolved INSUFFICIENT_DATA.
maj...@gmail.com
think the idea with any bug being resolved INCOMPLETE is that the
person marking it so would also give a brief explanation why, and
almost always directing the reporter to support avenues to help find
the complete information and refile. The other problem with NEEDS
INFORMATION is that it implies that more information is desired, which
we want to avoid.
A quick thesaurus search though, isn't providing any better examples
that are short, with the exception of "gremlins" everything short is
pretty insulting.
I personally think the context and the users would make INCOMPLETE
make sense (especially in combination with resolved), but respect the
argument that something more descriptive might be necessary. I'm not
sure how to get around that though without saying ____ REPORT.
Dan Mosedale
> The other problem with NEEDS INFORMATION is that it implies that more
> information is desired, which we want to avoid.
Why do we want to avoid that? Isn't that, in fact, exactly what we
want to express?
Dan
Mike Connor
This is unclearly stated, but my thinking is that we want to avoid
reopening that particular bug, instead encouraging the reporter to
file a clean and detailed report.
-- Mike
Gervase Markham
> We also have a NEEDINFO state for currently open bugs.
Sadly, the particular design and customisation capabilities of Bugzilla
mean that adding a new resolution is easy, whereas adding a new status
is hard. Ideally, technology wouldn't dictate policy, but that's just
the way it is.
Adding a new resolution is easy, and it's still taken us six months.
Gerv
Mike Shaver
>
> On 27-Mar-07, at 3:53 PM, Dan Mosedale wrote:
>
> reopening that particular bug, instead encouraging the reporter to
> file a clean and detailed report.
If the bug is a bad report, maybe, but telling them that they should
file a new bug because we want them to tell us more about which
version of something they're using, or if they have a given extension
installed, etc. seems like the wrong path. We lose all the cc: state
and such when the reporter does that, so I think we should generally
avoid it unless the bug is so messed up as to be a significant
impediment to actually fixing it.
Mike
Stephanie
A NEEDINFO state is needed much more than a missing info resolution
though. FWIW, a missing info resolution's more a technicality. Once a
bug's appropriately resolved, it tends to stay that way. Tracking what
bugs require action from the reporter (and being able to query or
filter by that) is of critical importance to efficient handling of
bugs, particularly unconfirmed bugs where we have to prod the reporter
for enough details to reproduce.
Given that this has been implemented by several downstream users of
Bugzilla, I still don't see what the difficulty is in having this, but
it's been held off since 2002 for a "perfect" solution, when at this
point, almost any solution would be an improvement, at least as far as
bugzilla.mozilla.org goes.
We have tremendous duplication of effort for the triagers dealing with
unconfirmed bugs for lack of a NEEDINFO state. The same unconfirmed
bugs have to be examined repeatedly only to see that they are waiting
for a reply. This takes time up that could be used confirming bugs for
which there's actually enough information to work with, and
contributes greatly to the frustration of anyone working with
unconfirmed bugs.
--Stephanie
maj...@gmail.com
that's the stats on how often a "my bookmarks are broken" bug actually
gets the necessary info added, and how useful bugs are that don't have
clear str until comment #8. This is mconnor's area, though so if there
are questions about that, ask him!
Shaver - you're right, and I don't think any of the examples you gave
were intended to be marked INCOMPLETE although definitely fit in the
description of the word. I think for the most part any bugs that would
really get the resolution we're trying to decide on would be support
requests, or people filing a bug that should be seeking support
instead, i.e. something broke and so they're filing, and they've done
nothing to try and reproduce or look to see if there's an answer
somewhere.
This in some ways presents a grey area for valid issues that don't
have steps to reproduce. Part of what I really liked about the idea
of marking these bugs separately is the idea that they're still there,
and once steps to reproduce are found, those incomplete bugs (if it's
clear they're the same issue) can be duped forward. If it's not clear
enough, it still gives people a place to track these sorts of issues,
to piece things together, rather than having to sort through what's
invalid because it didn't make sense and what's invalid because it
really isn't a bug or leaving them unconfirmed. I think we don't want
20 people all saying "I'm having the same issue" in one bug before
it's found that it's actually 3 different issues. That usually gets
messy (ala copy/paste or bookmarks issues).
The only issue I see this raising is how to get people who are cc'ed
to the INCOMPLETE bug aware of the new properly filed bugs to try out
the STR . I think this can be overcome either by using META bugs to
organize the incomplete bugs by symptom, if the priority is getting
all reporters onto a valid bug (how many care, how many leave after
filing?), or by searching incomplete bugs for similar symptoms and
asking the reporters to check out the new bug if we need confirmation
on the STR (if we're pointing these people to support avenues anyway,
they can stay in touch with those avenues to check up on new bugs.
Even with the current system there's no guarrantee their bug will be
the one that gets the work in it or that their bug will be duped to
the right one.)
-Majken "Lucy" Connor
maj...@gmail.com
>
> We have tremendous duplication of effort for the triagers dealing with
> unconfirmed bugs for lack of a NEEDINFO state. The same unconfirmed
> bugs have to be examined repeatedly only to see that they are waiting
> for a reply. This takes time up that could be used confirming bugs for
> which there's actually enough information to work with, and
> contributes greatly to the frustration of anyone working with
> unconfirmed bugs.
>
> --Stephanie
Stephanie - I might not be catching on to what you're meaning, but how
I'm understanding it, I don't see how it makes any difference to how
things are now. I'm not sure why the bugs have to be examined
repeatedly, and so maybe this is the heart of why I'm not getting it.
Even so, how would NEEDINFO change that? The way I'm grasping it,
you're watching the bug for more information either way.
The missing info state would mean you do one pass on the bug, see that
you can't simply ask a question like "do you have extensions
installed?" or "what version are you using?" Mark the bug as
INCOMPLETE (ideally this adds a canned response directing the reporter
to seek support who can either help solve their issue, or help them
troubleshoot the issue so they can file a complete bug) and then move
on.
Vladimir Vukicevic
> Given that this has been implemented by several downstream users of
> Bugzilla, I still don't see what the difficulty is in having this, but
> it's been held off since 2002 for a "perfect" solution, when at this
> point, almost any solution would be an improvement, at least as far as
> bugzilla.mozilla.org goes.
>
> We have tremendous duplication of effort for the triagers dealing with
> unconfirmed bugs for lack of a NEEDINFO state. The same unconfirmed
> bugs have to be examined repeatedly only to see that they are waiting
> for a reply. This takes time up that could be used confirming bugs for
> which there's actually enough information to work with, and
> contributes greatly to the frustration of anyone working with
> unconfirmed bugs.
I very much agree here -- I think that the way the Red Hat bugzilla
works is exactly the functionality that we want here: a bug in the
NEEDINFO state reverts to NEW as soon as a comment is added, without the
commenter explicitly needing to set anything. If the commenter wants it
to stay in NEEDINFO, they explicitly have to set the state to NEEDINFO
when they submit their comment.
- Vlad
maj...@gmail.com
>
> I very much agree here -- I think that the way the Red Hat bugzilla
> works is exactly the functionality that we want here: a bug in the
> NEEDINFO state reverts to NEW as soon as a comment is added, without the
> commenter explicitly needing to set anything. If the commenter wants it
> to stay in NEEDINFO, they explicitly have to set the state to NEEDINFO
> when they submit their comment.
>
> - Vlad
So is this a case where the bug is confirmed but you need to find out
things like a regression window? If so, then that's a bit of a
different beast and I would want to avoid letting discussion on it
delay action on this, which deals with unconfirmed bugs.
Gervase Markham
> I very much agree here -- I think that the way the Red Hat bugzilla
> works is exactly the functionality that we want here: a bug in the
> NEEDINFO state reverts to NEW as soon as a comment is added, without the
> commenter explicitly needing to set anything.
If people are interested in making it easier to have b.m.o. take
functional enhancements of this sort, they need to be aware of how
things work at the moment.
As I understand it, the current way that b.m.o. is maintained is via a
single large patch on top of a particular release of Bugzilla. This
large patch can be obtained on request. It contains all sorts of
different changes, all mixed together.
I'm sure there are several people who would be happy to write custom
improvements for b.m.o. (I certainly include myself in this), but the
above arrangements make it more difficult than it needs to be to obtain
a copy of the current production code which you know is absolutely
correct, write and test a patch against it, and share the results of
that with other people for further testing.
It also seems very difficult to get patches that have been written
applied. I've been waiting for 18 months to get a patch I wrote at
mconnor's request applied:
https://bugzilla.mozilla.org/show_bug.cgi?id=314056
If the code running on b.m.o. were a branch in an SCM instead of a big
tarball patch, people could check it out, get it working for them, write
patches and test them with much greater ease. It would also make it much
easier for b.m.o. to be upgraded (and downgraded again if necessary) to
take safe incremental improvements - because there would not be manual
patching and merging steps involved.
(This is not an attempt to attack anyone; I'm just outlining the facts
as I understand them, and suggesting why we might have the problem we have.)
Gerv
Gervase Markham
> As I understand it, the current way that b.m.o. is maintained is via a
> single large patch on top of a particular release of Bugzilla. This
> large patch can be obtained on request. It contains all sorts of
> different changes, all mixed together.
My apologies; this is incorrect. There are multiple, distinct patches,
and they can be obtained as a .tar.gz (or .bz2; I forget) - which is why
I originally remembered there being just one file.
Of course, while this is better patch management, it doesn't make it any
easier to create your own version of the b.m.o. code.
Gerv
Peter Weilbacher
> I think an important point in this discussion is in the bug, and
> that's the stats on how often a "my bookmarks are broken" bug actually
> gets the necessary info added, and how useful bugs are that don't have
> clear str until comment #8.
What's "str" (or "STR" as you use it further down in your post)? If it's
more frequently used, perhaps you can add a definition to the glossary on
MDC?
Peter.
Peter Weilbacher
Apart from that it would also make it more difficult to find the right bug
in a Bugzilla search. It's already difficult enough now, even for people
who search for bugs on a daily basis.
Peter.
Gijs Kruitbosch
Steps to reproduce. I added a Glossary entry.
~ Gijs
Gervase Markham
> My apologies; this is incorrect.
Double apologies; I was right the first time! The huge patch can be
obtained from here:
http://landfill.bugzilla.org/prodpatches/huge-bmo-patch8.diff
Gerv
Gervase Markham
> If the code running on b.m.o. were a branch in an SCM instead of a big
> tarball patch, people could check it out, get it working for them, write
> patches and test them with much greater ease. It would also make it much
> easier for b.m.o. to be upgraded (and downgraded again if necessary) to
> take safe incremental improvements - because there would not be manual
> patching and merging steps involved.
preed (who is responsible for maintaining the b.m.o. codebase) tells me
that there are no plans for this to happen until a better SCM than CVS
comes along in which to put the code. If there was a b.m.o. branch, he
would want to restrict checkins to it, and CVS doesn't allow that.
Gerv
Gervase Markham
> If the code running on b.m.o. were a branch in an SCM instead of a big
> tarball patch, people could check it out, get it working for them, write
> patches and test them with much greater ease. It would also make it much
> easier for b.m.o. to be upgraded (and downgraded again if necessary) to
> take safe incremental improvements - because there would not be manual
> patching and merging steps involved.
Reed Loden (who is responsible for maintaining the b.m.o. codebase)
Christopher Aillon
> Vladimir Vukicevic wrote:
>> I very much agree here -- I think that the way the Red Hat bugzilla
>> works is exactly the functionality that we want here: a bug in the
>> NEEDINFO state reverts to NEW as soon as a comment is added, without the
>> commenter explicitly needing to set anything.
>
> If people are interested in making it easier to have b.m.o. take
> functional enhancements of this sort, they need to be aware of how
> things work at the moment.
I just talked to dkl who maintains our bugzilla. Sadly he doesn't have
a patch that is upstreamable right now because of some dependencies on
RH-bugzilla-specific stuff. We're going to port to bugzilla 3.0 though
in the near future and this will need to happen. Expect a patch in the
next two months for NEEDINFO support which can be submitted to both
upstream bugzilla and it could probably be used in ours as well,
hopefully with minimal merge conflicts (I've not seen either patchsets).
Mike Connor
On 27-Mar-07, at 7:51 PM, Mike Shaver wrote:
> On 3/27/07, Mike Connor <mco...@mozilla.com> wrote:
>>
>> On 27-Mar-07, at 3:53 PM, Dan Mosedale wrote:
>>
>> This is unclearly stated, but my thinking is that we want to avoid
>> reopening that particular bug, instead encouraging the reporter to
>> file a clean and detailed report.
>
> If the bug is a bad report, maybe, but telling them that they should
> file a new bug because we want them to tell us more about which
> version of something they're using, or if they have a given extension
> installed, etc. seems like the wrong path. We lose all the cc: state
> and such when the reporter does that, so I think we should generally
> avoid it unless the bug is so messed up as to be a significant
> impediment to actually fixing it.
I think that there are many bugs that are filed in such a state that
they're unfixable. The amount of time and comments it takes to get
to a useful bug report is inversely proportional to the odds it will
get fixed, which is why I have argued that such a bug is typically
not worth significant effort. NEEDINFO is an added intermediate step
for the grey areas, I would argue that bugs needing more info that
don't get another comment after a given time should automatically
change to INCOMPLETE in time. If we need more info from the
reporter, and we don't get that info in a reasonable amount of time,
the odds are slim at best that we'll ever get the needed information.
In terms of what we've been doing with triage in the last 3-4 years,
the guideline has been to resolve as INVALID or WORKSFORME anything
that is lacking the info needed to fix it, rather than commenting and
leaving as UNCONFIRMED, based on the low (< 3-4%) rate of responses
to triager questions. This is just adding another potential state
that is less inflammatory and more correct (INVALID is overloaded, as
we use it for both "this is not a bug, this is expected behaviour"
and "this bug lacks the required information.")
So NEEDINFO -> INCOMPLETE for the grey areas, and INCOMPLETE for
obviously poor bugs, with links to the right ways to follow up on
their issue.
-- Mike
Gervase Markham
> I think that there are many bugs that are filed in such a state that
> they're unfixable. The amount of time and comments it takes to get to a
> useful bug report is inversely proportional to the odds it will get
> fixed, which is why I have argued that such a bug is typically not worth
> significant effort.
Mike makes an important point. We need to accept at this stage that we
do not have the resources to exhaustively investigate every bug report
or complaint we get. (That's one reason Hendrix exists.) Therefore, we
should focus on those which are well-reported - because they give the
most bang for the buck.
If there's a problem affecting multiple people, there will be good and
bad bug reports of the same issue. As long as one of the good ones gets
dealt with, then we are OK. Recognising and closing out the bad ones
quickly is just good use of scarce resources.
Gerv
Gervase Markham
> I just talked to dkl who maintains our bugzilla. Sadly he doesn't have
> a patch that is upstreamable right now because of some dependencies on
> RH-bugzilla-specific stuff. We're going to port to bugzilla 3.0 though
> in the near future and this will need to happen. Expect a patch in the
> next two months for NEEDINFO support which can be submitted to both
> upstream bugzilla and it could probably be used in ours as well,
> hopefully with minimal merge conflicts (I've not seen either patchsets).
I would point out that the last time we discussed the Bugzilla workflow,
the agreed "ideal workflow" didn't include a NEEDINFO state. Instead, it
had a READY state.
http://steelgryphon.com/testcases/bugzilla-workflow-9.png
Gerv
Stephanie Daugherty
Not really. Putting the bugs in NEEDINFO gets them out of UNCONFIRMED or
NEW, therefore making everything in the UNCONFIRMED state ready for some
form of triage - it will either go from UNCONFIRMED to NEW or from
UNCONFIRMED to NEEDINFO, or from UNCONFIRMED to RESOLVED.
There'd no longer be a wait in UNCONFIRMED for the reporter to reply to
comments - basically just making sure that the workflow clearly
indicates who needs to move the bug forward, instead of just a
UNCONFIRMED busy loop for the triagers of seeing the same bugs over and
over again until we realize they are old enough without further comment
to just RESOLVED INVALID.
Stephanie Daugherty
>
> I would point out that the last time we discussed the Bugzilla workflow,
> the agreed "ideal workflow" didn't include a NEEDINFO state. Instead, it
> had a READY state.
> http://steelgryphon.com/testcases/bugzilla-workflow-9.png
>
> Gerv
Then either theres a need to change the guide for triagers and the
bugzilla process to say that it's ok to ruthlessly close bugs without
enough info on sight (ie, such bugs go directly to RESOLVED INCOMPLETE
right away, without trying to prod the reporter), or the "ideal"
workflow needs to better reflect the realities of bugs being filed
without enough information, and how we go about getting enough information.
Regardless, if bugs are sitting in UNCONFIRMED, we need somewhere to
send them as they are triaged, rather than have them sit there waiting
for enough information to trickle in to confirm them.
There's a major problem to be solved though, and the unconfirmed bug
lists aren't getting any smaller :(
--Stephanie
Andrew Schultz
That workflow may be ideal, but it makes quite a few assumptions that
are unrealistic
1. A bug that was once considered NEW will not end up WORKSFORME (this
happens all the time).
2. A bug that was once considered READY will not end up a DUPLICATE,
INVALID, WONTFIX or WORKSFORME.
In reality bugs that seem valid have to be re-evaluated occasionally,
mostly to catch the ones that have become WORSFORME and DUPLICATE due to
being fixed elsewhere.
We have a clean-report keyword in b.m.o that basically mimics the READY
status. benc pushed for it a long time ago and nobody uses it (except
him while he was still active).
Anyway, I don't see a big difference between NEW/READY and NEEDINFO/NEW,
except that NEEDINFO more precisely describes the status of many bugs
that are waiting on feedback from someone specifically (usually the
reporter).
--
Andrew Schultz
ajsc...@verizon.net
http://www.sens.buffalo.edu/~ajs42/
Stephanie Daugherty
Hash: SHA1
Andrew Schultz wrote:
> Anyway, I don't see a big difference between NEW/READY and NEEDINFO/NEW,
> except that NEEDINFO more precisely describes the status of many bugs
> that are waiting on feedback from someone specifically (usually the
> reporter).
>
I see NEEDINFO more useful to UNCONFIRMED than it is to NEW, although, I
would expect that a transition from NEW to NEEDINFO and back wouldn't be
all that uncommon either.
Really, the only big difference is that we'd basically be able to ignore
bugs while we're waiting for feedback from the reporter, and
periodically clean those bugs out by querying for NEEDINFO bugs that
haven't been changed in a long time.
Most importantly, as I said in another branch off this thread, this
would eliminate the need to periodically poll UNCONFIRMED bugs to see if
they are ready to be sent elsewhere, and would provide triagers a place
to send every unconfirmed bug - they would all be able to be confirmed,
needinfo, or resolved from the unconfirmed state, so we only need to
look at them once for each time they enter UNCONFIRMED.
- --Stephanie
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iD8DBQFGC9k6sUIV80N+s8sRAu2bAKCe7zZkQmsthZrK2f36+s+G8dNhcwCfYpeW
W4kAL4Mg34j0bCKOZhT9eTI=
=51jc
-----END PGP SIGNATURE-----
Tony Mechelynck
[...]
> If there's a problem affecting multiple people, there will be good and
> bad bug reports of the same issue. As long as one of the good ones gets
> dealt with, then we are OK. Recognising and closing out the bad ones
> quickly is just good use of scarce resources.
>
> Gerv
If there are good and bad reports for a single issue, then one of the good
ones should stay open and get the patches etc. as attachments, and all others
should be RESOLVED DUPLICATE shouldn't they? Or is there something I missed?
Best regards,
Tony.
--
Anxiety, n.:
The first time you can't do it a second time.
Panic, n.:
The second time you can't do it the first time.
maj...@gmail.com
> If there are good and bad reports for a single issue, then one of the good
> ones should stay open and get the patches etc. as attachments, and all others
> should be RESOLVED DUPLICATE shouldn't they? Or is there something I missed?
>
> Best regards,
> Tony.
> --
> Anxiety, n.:
> The first time you can't do it a second time.
>
> Panic, n.:
> The second time you can't do it the first time.
If something is a bad report, you usually can't tell which bug it
falls under. "My bookmarks are missing" or "my clipboard is broken"
can each be duped to at least 3 different bugs depending on the root
cause. There will be the case where someone is trying to track down a
particular issue, so they can search the incomplete bugs for the
symptoms and contact the reporters (via email or the bug report, I
don't know which is ideal) for more info and dupe it and there will be
savvy users who will watch bugzilla for similar bugs and let us know
when they see a proper report that describes their issue. Also, if
users do seek support and do enough troubleshooting to figure out that
their issue is a duplicate, they can comment back. The idea with
INCOMPLETE is to give a better place for these bugs to live (instead
of UNCONFIRMED) until they can be duped, and a better place to live
(instead of INVALID) if they can't be.
Gervase Markham
> That workflow may be ideal, but it makes quite a few assumptions that
> are unrealistic
>
> 1. A bug that was once considered NEW will not end up WORKSFORME (this
> happens all the time).
> 2. A bug that was once considered READY will not end up a DUPLICATE,
> INVALID, WONTFIX or WORKSFORME.
I don't think it makes those assumptions; I just don't think it's
absolutely exhaustive in enumerating the possible transitions.
> Anyway, I don't see a big difference between NEW/READY and NEEDINFO/NEW,
> except that NEEDINFO more precisely describes the status of many bugs
> that are waiting on feedback from someone specifically (usually the
> reporter).
It's be interested in getting mconnor's feedback on whether he thinks
the two are equivalent. If they are, perhaps we should go in the
NEEDINFO direction, for process compatibility with other projects.
Gerv
maj...@gmail.com
don't lose what we're doing with INCOMPLETE while discussing the whole
process.
I want to check on two things,
a) are we all on board with INCOMPLETE at this point? I think
everyone is, and some would like to add NEEDSINFO as well (when it's
possible).
b) are we ok with calling it INCOMPLETE or do we want to go back to
discussing alternate names to make it more clear it's the report
that's incomplete and not work on the bug?
-Majken
Tony Mechelynck
The principle (with NEEDINFO in addition to UNCONFIRMED/NEW/RESOLVED/REOPENED
and some new resolution type) seems well-described enough. What about RESOLVED
LACKSINFO to make it clearer that the bug was resolved for want of sufficient
information?
Best regards,
Tony.
--
Whom the gods wish to destroy they first call promising.
Christopher Aillon
> Not trying to stifle discussion at all, just trying to make sure we
> don't lose what we're doing with INCOMPLETE while discussing the whole
> process.
>
> I want to check on two things,
> a) are we all on board with INCOMPLETE at this point? I think
> everyone is, and some would like to add NEEDSINFO as well (when it's
> possible).
I'm okay with the general principle of it,
> b) are we ok with calling it INCOMPLETE or do we want to go back to
> discussing alternate names to make it more clear it's the report
> that's incomplete and not work on the bug?
but not the name per my earlier posts.
Gervase Markham
> I want to check on two things,
> a) are we all on board with INCOMPLETE at this point? I think
> everyone is, and some would like to add NEEDSINFO as well (when it's
> possible).
Why are INCOMPLETE and NEEDSINFO not synonyms? Or is one a resolution
and another a status?
The current plan of record, should we ever change the statuses, is to
add a READY instead of a NEEDSINFO. A lot of discussion went into that;
I wouldn't want to change the plan until the original participants have
at least commented.
So I'm on board for a new INCOMPLETE resolution but not for status changes.
> b) are we ok with calling it INCOMPLETE or do we want to go back to
> discussing alternate names to make it more clear it's the report
> that's incomplete and not work on the bug?
I can't think of a better name.
Gerv