Re: New "INCOMPLETE" resolution in Bugzilla
Gervase Markham
> In a few days, we plan to create a new "INCOMPLETE" resolution in
> Bugzilla, taking advantage of the new "custom resolutions" feature.
To add more data to the discussion, I went through 26 other Bugzillas
from the biggest free software projects (as listed on
http://www.bugzilla.org/installation-list/) to see what they do. The
results are very interesting.
In general, the larger your project, the more likely you are to have
modified statuses and resolutions on your Bugzilla. And, even though
editing resolutions is an order of magnitude easier than editing
statuses, more have edited statuses than resolutions.
Sample size: 26
Unmodified stat/res: 13
Modified statuses: 11/13
Modified resolutions: 9/13
Looking at the mods, there's a single run-away winner. 9 of the 13
projects who made a modification added a NEEDINFO (eight as a status and
one as a resolution). No other status name got added more than once.
Of the resolutions, NOTABUG/NOTOURBUG was most popular with 4, then
UPSTREAM with 3, then INSUFFICIENT_DATA with 2 (+ 1 NEEDINFO).
The last interesting point is that several projects removed the REOPENED
state, presumably instead putting bugs straight back into NEW or equivalent.
Raw data follows.
Gerv
Kernel:
Status: +REJECTED, +DEFERRED, +NEEDINFO, -UNCONFIRMED, -REOPENED
Resolution: +CODE_FIX, +PATCH_ALREADY_AVAILABLE, +WILL_NOT_FIX,
+WILL_FIX_LATER, +UNREPRODUCIBLE, +DOCUMENTED,
+INSUFFICIENT_DATA, -FIXED, -WONTFIX, -WORKSFORME
GNOME:
Status: +NEEDINFO
Resolution: +NOTABUG, +NOTGNOME, +INCOMPLETE, +GNOME1.x, +OBSOLETE,
+NOTXIMIAN, -WORKSFORME
Apache:
Status: +NEEDINFO
Resolution: No changes
OpenOffice.org:
Status: +STARTED, -ASSIGNED
Resolution: No changes
Red Hat:
Status: +NEEDINFO, +MODIFIED, +ON_DEV, +PASSES_QA, +ON_QA,
+FAILS_QA, +RELEASE_PENDING, +POST, -UNCONFIRMED,
-RESOLVED, -REOPENED
Resolution: +CURRENTRELEASE, +RAWHIDE, +ERRATA, +UPSTREAM,
+NEXTRELEASE, +CANTFIX, +INSUFFICIENT_DATA
Novell:
Status: +NEEDINFO
Resolution: No changes
Turbolinux:
Status: -ASSIGNED, -REOPENED, -CLOSED (equivalents)
Resolution: +PATCHED, +UPDATE, +NOTABUG, +WORKAROUND, +PENDING
Gentoo:
Status: No changes
Resolution: +NEEDINFO, +TEST_REQUEST, +UPSTREAM, +CANTFIX
Mandriva:
Status: +NEEDINFO, -CLOSED
Resolution: +OLD, +REPORTEDUPSTREAM
freedesktop.org:
Status: +NEEDINFO
Resolution: +NOTABUG, +NOTOURBUG
freestandards.org:
Status: +NEEDINFO, +PLEASETEST
Resolution: +NOTABUG, +NOTOURBUG
gcc:
Status: +SUSPENDED, +WAITING
Resolution: No changes
WINE:
Status: No changes
Resolution: +ABANDONED
No changes: KDE, Eclipse, Abiword, distributed.net, mozdev.org, OSAF,
ssh, Samba, Squid, Fedora (the other one), W3C, Wikipedia, XFree86.
Statuses:
NEEDINFO 8
REJECTED
DEFERRED
STARTED
MODIFIED
ON_DEV
PASSES_QA
ON_QA
FAILS_QA
RELEASE_PENDING
POST
PLEASETEST
SUSPENDED
WAITING
Resolutions (with some equivalence mappings done and obviously
project-specific stuff removed):
NOTABUG 4
UPSTREAM 3
INSUFFICIENT_DATA 2
NOTOURBUG 2 (but the same ones as NOTABUG)
PATCH_ALREADY_AVAILABLE
NOTGNOME
INCOMPLETE
GNOME1.x
OBSOLETE
NOTXIMIAN
CANTFIX
PATCHED
UPDATE
WORKAROUND
PENDING
NEEDINFO
TEST_REQUEST
CANTFIX
OLD
ABANDONED
Neil
> Resolutions (with some equivalence mappings done and obviously
> project-specific stuff removed):
> NOTABUG 4
> UPSTREAM 3
> INSUFFICIENT_DATA 2
> NOTOURBUG 2 (but the same ones as NOTABUG)
> PATCH_ALREADY_AVAILABLE
> NOTGNOME
> INCOMPLETE
> GNOME1.x
> OBSOLETE
> NOTXIMIAN
> CANTFIX
> PATCHED
> UPDATE
> WORKAROUND
> PENDING
> NEEDINFO
> TEST_REQUEST
> CANTFIX
> OLD
> ABANDONED
So, 2x CANTFIX then?
--
Warning: May contain traces of nuts.
Gervase Markham
> So, 2x CANTFIX then?
Yeah; sorry I missed that one.
Gerv
Gervase Markham
> https://bugzilla.mozilla.org/show_bug.cgi?id=369544
>
> In a few days, we plan to create a new "INCOMPLETE" resolution in
> Bugzilla, taking advantage of the new "custom resolutions" feature.
I've just been back through the discussion; it seems that people are OK
with the principle but there is some disagreement on the name.
- majken and stephanie are happy with INCOMPLETE, which matches GNOME.
- caillon suggested INSUFFICIENT_DATA, which matches RedHat and the
kernel; shaver also liked this.
- shaver suggested NEEDS_INFORMATION (possibly shortenable to NEEDSINFO)
but either might be confused with a forthcoming NEEDINFO status.
** I suggest we choose between INCOMPLETE and INSUFFICIENT_DATA. I'm
happy with either. I don't really want to have a vote, though, because
it implies everyone's opinion carries equal weight. Thoughts? **
There's also some support for a NOTABUG/NOTMOZILLA/CANTFIX-style resolution.
I should also note that I've been working with Bugzilla developer
LpSolit to implement custom statuses for Bugzilla, which would allow us
to add the NEEDINFO and READY states if we wanted them. Let's not count
our chickens, but we hope to have this work finished in a few weeks.
Then, of course, b.m.o. would need to be upgraded.
Gerv
AC
(Even a memory dump leaves out the environment, and you can't have
complete information -- is the machine infected with a virus, or is the
motherboard failing, or was there a power spike, or did a cosmic ray
just flip a bit and cause the crash, or ... ?)
As others have complained, it also implies the bug can be completed
(like a grade of 'Incomplete' in a university class if you get an
extension on the final paper), but by the time it reaches this
resolution, it is too old (it was probably in the NEEDINFO state for a
while first). And there is confusion that it could mean that the
resolution is not complete.
RESOLVED: INSUFFICIENT_DATA, is partly better, because it uses the
concept of 'sufficient' (to reproduce the problem) rather than
'complete'. But since it contains the word 'sufficient', there is still
possible confusion that it is like NEEDINFO and can be continued with
more data.
The wiki diagram suggests RESOLVED: DEFICIENT, which is shorter (save
screen space) and slightly less likely to suggest it can be continued
with more data (to me it suggests 'defective', and you replace defective
part, you don't extend it, like a deficient sealed jar only half full.).
But, I'm not convinced that it needs a separate resolution --- wouldn't
RESOLVED: EXPIRED be good enough for bugs that NEEDINFO but waited
around too long and never got enough info added? What is the use case
for distinguishing them from other expired bugs?
(Great to hear bugzilla will be getting custom NEEDINFO status too!)
Andrew Schultz
> RESOLVED: INCOMPLETE erroneously implies it is possible to be COMPLETE.
...
> As others have complained, it also implies the bug can be completed
Why couldn't it? People file a bug and don't provide a testcase.
Currently the bug often sits for a very long time (with occasional
poking) before it gets resolved (often years). But it doesn't need to
be that way (and shouldn't). I hope the NEEDINFO status will help
identifying bugs that are ripe to be resolved and allow them to be
resolved sooner. Even a bug that's years old sometimes still exists and
with additional info (testcase, steps to reproduce) could be a valid bug.
> The wiki diagram suggests RESOLVED: DEFICIENT, which is shorter (save
> screen space) and slightly less likely to suggest it can be continued
> with more data (to me it suggests 'defective', and you replace defective
> part, you don't extend it, like a deficient sealed jar only half full.).
I don't know how DEFICIENT (or defective) is that much different than
INVALID.
--
Andrew Schultz
aj...@buffalo.edu
http://www.sens.buffalo.edu/~ajs42/
David E. Ross
As I understand, the new resolution is supposed to be used on a bug for
which insufficient description, test case, or other data prevent any
real work being done to correct the problem. If the bug is closed
because of this resolution but then sufficient data are later provided,
the bug might be reopened and actually fixed.
Why then not MORE_DATA_REQD, NOT_ENOUGH_DATA, or INCOMPLETE_REPT? After
all, the deficiency is in the bug report.
--
David E. Ross
<http://www.rossde.com/>.
Anyone who thinks government owns a monopoly on inefficient, obstructive
bureaucracy has obviously never worked for a large corporation. © 1997
Gervase Markham
> RESOLVED: INCOMPLETE erroneously implies it is possible to be COMPLETE.
Only as much as resolving WORKSFORME implies that I don't care if it
doesn't work for you - i.e. not very much. This seems to me to be
splitting semantic hairs somewhat.
> RESOLVED: INSUFFICIENT_DATA, is partly better, because it uses the
> concept of 'sufficient' (to reproduce the problem) rather than
> 'complete'. But since it contains the word 'sufficient', there is still
> possible confusion that it is like NEEDINFO and can be continued with
> more data.
Er... it can.
> The wiki diagram suggests RESOLVED: DEFICIENT, which is shorter (save
> screen space) and slightly less likely to suggest it can be continued
> with more data (to me it suggests 'defective', and you replace defective
> part, you don't extend it, like a deficient sealed jar only half full.).
That may or may not be more accurate from a technical point of view, but
it would not meet the social goals of the change - which is to provide a
resolution which is less offensive to bug reporters than INVALID.
If INCOMPLETE has the problem that it might be thought to apply to a fix
rather than the bug, DEFICIENT has the enormous problem that it might be
thought to apply to the reporter!
> But, I'm not convinced that it needs a separate resolution --- wouldn't
> RESOLVED: EXPIRED be good enough for bugs that NEEDINFO but waited
> around too long and never got enough info added? What is the use case
> for distinguishing them from other expired bugs?
If we ever have NEEDINFO, that might work semantically. But I think
people want to close some bad bugs as
INCOMPLETE/INSUFFICIENT_DATA/WHATEVER pretty quickly, so EXPIRED would
be semantically wrong.
Gerv
Michael Lefevre
> AC wrote:
[...]
>> RESOLVED: INSUFFICIENT_DATA, is partly better, because it uses the
>> concept of 'sufficient' (to reproduce the problem) rather than
>> 'complete'. But since it contains the word 'sufficient', there is still
>> possible confusion that it is like NEEDINFO and can be continued with
>> more data.
>
> Er... it can.
Can it? My understanding was that the current suggestion was that
incomplete/insufficient reports would be resolved quickly with this
resolution, and reporters encouraged to file new bug reports (in the hopes
that they'd make a better job of it next time).
(Based on discussion in and around:
http://groups.google.co.uk/group/mozilla.dev.planning/msg/2c6c1b361e17f93a?dmode=source
)
>> But, I'm not convinced that it needs a separate resolution --- wouldn't
>> RESOLVED: EXPIRED be good enough for bugs that NEEDINFO but waited
>> around too long and never got enough info added? What is the use case
>> for distinguishing them from other expired bugs?
>
> If we ever have NEEDINFO, that might work semantically. But I think
> people want to close some bad bugs as
> INCOMPLETE/INSUFFICIENT_DATA/WHATEVER pretty quickly, so EXPIRED would
> be semantically wrong.
Indeed.
Also, I'm not sure why you directed followups for this thread to
mozilla.dev.general, having just announced that this group will be going
away because it's not reaching the right audience.
--
Michael
David E. Ross
> On 2007-04-18, Gervase Markham <ge...@mozilla.org> wrote:
>> AC wrote:
> [...]
>>> RESOLVED: INSUFFICIENT_DATA, is partly better, because it uses the
>>> concept of 'sufficient' (to reproduce the problem) rather than
>>> 'complete'. But since it contains the word 'sufficient', there is still
>>> possible confusion that it is like NEEDINFO and can be continued with
>>> more data.
>> Er... it can.
>
> Can it? My understanding was that the current suggestion was that
> incomplete/insufficient reports would be resolved quickly with this
> resolution, and reporters encouraged to file new bug reports (in the hopes
> that they'd make a better job of it next time).
>
> (Based on discussion in and around:
> http://groups.google.co.uk/group/mozilla.dev.planning/msg/2c6c1b361e17f93a?dmode=source
> )
That would lead to a proliferation of bug reports, each reporting the
same problem but with increasing detail. In the process, the history of
when the problem was first discovered would be lost. I would prefer
reopening an old bug once more detail is available.
Gervase Markham
> On 2007-04-18, Gervase Markham <ge...@mozilla.org> wrote:
>> AC wrote:
> [...]
>>> RESOLVED: INSUFFICIENT_DATA, is partly better, because it uses the
>>> concept of 'sufficient' (to reproduce the problem) rather than
>>> 'complete'. But since it contains the word 'sufficient', there is still
>>> possible confusion that it is like NEEDINFO and can be continued with
>>> more data.
>> Er... it can.
>
> Can it? My understanding was that the current suggestion was that
> incomplete/insufficient reports would be resolved quickly with this
> resolution, and reporters encouraged to file new bug reports (in the hopes
> that they'd make a better job of it next time).
My point was merely that any bug can be reopened.
> Also, I'm not sure why you directed followups for this thread to
> mozilla.dev.general, having just announced that this group will be going
> away because it's not reaching the right audience.
Yes; see .planning for the proper discussion. Every time I notice I've
accidentally posted something here only, I copy it over there.
Gerv
Michael Lefevre
On 2007-04-20, Gervase Markham <ge...@mozilla.org> wrote:
> Michael Lefevre wrote:
>> On 2007-04-18, Gervase Markham <ge...@mozilla.org> wrote:
>>> AC wrote:
>> [...]
>>>> RESOLVED: INSUFFICIENT_DATA, is partly better, because it uses the
>>>> concept of 'sufficient' (to reproduce the problem) rather than
>>>> 'complete'. But since it contains the word 'sufficient', there is still
>>>> possible confusion that it is like NEEDINFO and can be continued with
>>>> more data.
>>> Er... it can.
>>
>> Can it? My understanding was that the current suggestion was that
>> incomplete/insufficient reports would be resolved quickly with this
>> resolution, and reporters encouraged to file new bug reports (in the hopes
>> that they'd make a better job of it next time).
>
> My point was merely that any bug can be reopened.
Well I know that's technically true, of course.
If the policy is going to be that bad bug reports are closed and reporters
are told to file new bugs, then the resolution should not suggest that a
bug report could be reopened (and people shouldn't be making the point
that bugs can be reopened!).
Actually I don't think that closing incomplete bugs and filing new ones is
the best idea, but if that's going to be the policy, then as far as
possible everyone should stick to it - we shouldn't have QA people doing
one thing and telling reporters that that is the way it should be done,
and then some developers doing something completely different.
--
Michael