Handling of Whiteboard entry for CLOSEME bugs
Henrik Skupin
There is one issue which came-up lately. How should we handle the
whiteboard entry for closeme bugs? The current way is to add a date
which is a month or so in the future. The reporter has time to answer
the outstanding question. After that period we close those bugs as
incomplete.
How shall we handle the remaining white board entry? Do we remove it? As
I remember we had a project (last year?) in which the whiteboard entry
was heavily used and not removed after the bug was getting closed. That
happened to have a reference for such bugs, which were closed during
this project. Some month ago I asked around and the answer was that I
can leave the entry as it is. But lately a lot of bug were closed and
the entry removed. So which way we wanna go?
Running a search against open bugs (http://tinyurl.com/7zhvfo) doesn't
bring up already resolved bugs. Further we know which bugs were closed,
because there was no answer for a longer time. Ok, and there is one
question... is each as incomplete resolved bug a closeme bug?
Would be nice to have a clean way here.
Henrik
Wayne Mery
On 1/6/2009 6:25 AM, Henrik Skupin wrote:
> Hey,
>
> There is one issue which came-up lately. How should we handle the
> whiteboard entry for closeme bugs? The current way is to add a date
> which is a month or so in the future. The reporter has time to answer
> the outstanding question. After that period we close those bugs as
> incomplete.
TB triagers are using ISO date now for a long time rather than month.
I think the benchmark most used is 3-4 weeks.
I sometimes make it as little as 2 weeks depending - from example super
old and very unlikely to get a response.
> How shall we handle the remaining white board entry? Do we remove it? As
> I remember we had a project (last year?) in which the whiteboard entry
> was heavily used and not removed after the bug was getting closed. That
> happened to have a reference for such bugs, which were closed during
> this project. Some month ago I asked around and the answer was that I
> can leave the entry as it is. But lately a lot of bug were closed and
> the entry removed. So which way we wanna go?
closeme originates here
http://archive.quality.mozilla.org/bugzilla/bugtriage but says nothing
about keeping it when closing the bug. Camino spells it out
http://wiki.caminobrowser.org/QA:Triage_Policies_and_Procedures#.E2.80.9CCLOSEME.E2.80.9D
- they keep the closeme in whiteboard.
I too prefer closeme not be removed by triagers when bug is closed
because the whiteboard is useful historical information, just like
keyword. (Very few people do in fact remove it) And keeping it does not
cause a problem afaik. Two examples of useful
* whiteboard seen in queries of closed bugs (I don't have to click bug
details, then history to find what I want)
* getting statistics of past "closeme" work - which is only possible by
having closeme in the whiteboard.
IMO closeme should only be removed if it becomes obsolete due to someone
concluding the bug is valid.
> Running a search against open bugs (http://tinyurl.com/7zhvfo) doesn't
> bring up already resolved bugs. Further we know which bugs were closed,
> because there was no answer for a longer time. Ok, and there is one
> question... is each as incomplete resolved bug a closeme bug?
IMO: closeme is not a prerequisite to closing incomplete. Nor does a
closeme with no response mean the proper resolution is (always)
incomplete - for example a bug could be resolved invalid if it meets the
requirements for invalid.
> Would be nice to have a clean way here.
>
> Henrik
Note also in some (most, all?) cases it is not wise to arbitrarily
resolve a bug incomplete just because there was no response. Sometimes
closeme is used to coax those watching the bug to make a response within
a reasonable period of time. Yeah, it's a misuse of the process, but it
is proven to work.
Justin Dolske
> I sometimes make it as little as 2 weeks depending - from example super
> old and very unlikely to get a response.
I think a couple of weeks after requesting required information is
generally sufficient. That keeps us from having bugs sitting in limbo
for a long period of time; bugs can always be reopened. Of course
circumstances vary. And I wouldn't be surprised if some module owners
want to keep bugs open longer.
> I too prefer closeme not be removed by triagers when bug is closed
> because the whiteboard is useful historical information
Personally, I prefer to see it removed. It's just stale clutter when
searching through closed bugs. Whiteboard changes are shown in the bug
history, so strictly speaking removing it doesn't lose any information.
The whiteboard is generally a fluid field for noting current state of a
bug, I don't think we should try to use it as some kind of permanent record.
Justin
Carsten Book
unconfirmed bugs. Keeping this CLOSEME Status allows us to track how
many bugs we have closed during this project.
- Tomcat
Tyler Downer
As I brought this up with Clint and Henrik, I do not think there is a
real reason to kep the CLOSEME. As Justin said, it just clutters the
bug. Sometimes, if the person who put the CLOSEME up did not write a
good comment along with it, you have to check history anyways. Also, to
use it to prompt users to close their own bugs, or re-read them is fine,
but once the bug is close, the CLOSEME has no impact on that. I usually
put the following comment in when I close a bug as INCO
>@Reporter, we have not heard back from you in a while, so I am closing
this bug as INCOMPLETE. You can >reopen this bug if more information
becomes available. Some helpful information you can provide us is found
at >http://quality.mozilla.org/bug-writing-guidelines. You should also
use a recent version of Firefox, from >http://www.getfirefox.com.
Of course it is different for the product and other circumstances, per bug.
The CLOSEME, as I see it, is an action to be taken. Once the action is
taken, we don't need to see closeme in the whiteboard, as the bug was
already closed, you can read the comment, and half of the bugs that are
closed as INCO, don't even go through the CLOSEME stage. They just sit
with no response, a triage mamber finds them, and closes them.
Just my two cents.
Tyler Downer
Wayne Mery
> Would be nice to have a clean way here.
Henrik, am in correct in interpreting this not as saying you want to see
whiteboard cleared, but rather it makes sense to have a mozilla-wide
standard or alternatively a per product standard?
Justin wrote:
> Whiteboard changes are shown in the bug history ...
True, but history is not queryable.
> The whiteboard is generally a fluid field for noting current state
> of a bug, I don't think we should try to use it as some kind
> of permanent record.
Whiteboard is in fact the adhoc version of keywords, and we don't remove
most keywords when closing a bug. (Granted, the future value of most
keywords exceeds what might be in whiteboard.) Also, neither bmo's
recent history [1] nor long term history [2] support the idea that
whiteboard must or should be cleared upon closing a bug. Is there doc or
informal standard indicating there is value to doing so?
Absolutely, from a workflow point of view CLOSEME is no longer needed
after a bug is closed. No disagreement there. And I understand the
desire to keep bugs "clean" of unnecessary or inaccurate information.
But, as has been stated, closeme does has QUERY value for some people.
That example of gathering statistics has been stated. And personally, I
have used it post bug closure to check whether bugs I flagged closeme
received comments, whether it was closed appropriately, etc.
And, respectfully, I don't see the use case where CLOSEME is clutter. If
we are talking queries, do people routinely run queries which include
status INCOMPLETE?
Henrik Skupin
> Henrik, am in correct in interpreting this not as saying you want to see
> whiteboard cleared, but rather it makes sense to have a mozilla-wide
> standard or alternatively a per product standard?
Yes, and this thread is for closeme only. I don't want to cover each
whiteboard entry here. Everyone has another opinion on that topic, means
as incomplete resolved bugs have this entry or not.
As Carsten has already pointed out this entry was used for a project to
catch-up with all the unconfirmed bugs last year. That gives me the
impression we should remove it now when resolving a bug. But meanwhile
we have hundreds of bugs, which were closed at a later time but still
have this whiteboard entry. And we aren't able to identify those
resolved bugs while the project was active. So should it be a factor for
out decision?
> > Whiteboard changes are shown in the bug history ...
>
> True, but history is not queryable.
Further it would be nice to see how long it takes until bugs were closed
as incomplete. Having the whiteboard entry you can easily compare its
date and the resolved date.
> And, respectfully, I don't see the use case where CLOSEME is clutter. If
> we are talking queries, do people routinely run queries which include
> status INCOMPLETE?
That also gives me that impression. You can simply exclude resolved bugs
from your query.
Henrik
Gordon P. Hemsley
> Wayne Mery wrote on 11.01.2009 2:05 Uhr:
>
> > Henrik, am in correct in interpreting this not as saying you want to see
> > whiteboard cleared, but rather it makes sense to have a mozilla-wide
> > standard or alternatively a per product standard?
>
> whiteboard entry here. Everyone has another opinion on that topic, means
> as incomplete resolved bugs have this entry or not.
>
> As Carsten has already pointed out this entry was used for a project to
> catch-up with all the unconfirmed bugs last year. That gives me the
> impression we should remove it now when resolving a bug. But meanwhile
> we have hundreds of bugs, which were closed at a later time but still
> have this whiteboard entry. And we aren't able to identify those
> resolved bugs while the project was active. So should it be a factor for
> out decision?
>
> > > Whiteboard changes are shown in the bug history ...
>
> > True, but history is not queryable.
>
> Further it would be nice to see how long it takes until bugs were closed
> as incomplete. Having the whiteboard entry you can easily compare its
> date and the resolved date.
>
> > we are talking queries, do people routinely run queries which include
> > status INCOMPLETE?
>
> That also gives me that impression. You can simply exclude resolved bugs
> from your query.
>
> Henrik
(Yes, I'm bringing this back up after a month, but there was no
consensus or even conclusion to the topic.)
Perhaps you could get a keyword implemented (I suggest either
"closeme" or "closedme") that can be added to the bug after it has
been closed. This would aid in your querying, but would clean up the
whiteboard for the current status of the bug. But on top of all that,
there needs to be a standardized form of adding the tag to the
whiteboard to begin with. IMO, the original standard of MM/DD was a
Bad Idea™. Even the inclusion of brackets around the tag is
inconsistent.
I recommend standardizing the tag to this:
[closeme YYYY-MM-DD (RESOLUTION(?) (######))]
In this standard, "closeme" is lowercase, ISO date format is used, the
resolution is uppercase but optional, the question mark is optional
(for cases when the correct resolution after the "closeme" date is
unclear), and the bug number can be listed if the DUPLICATE resolution
is used and the dupe bug is known.
Some examples:
[closeme 2009-03-15]
[closeme 2009-03-15 WORKSFORME]
[closeme 2009-03-15 DUPLICATE?]
[closeme 2009-03-15 DUPLICATE 123456]
The date can vary, with a minimum of 2 weeks into the future. A
maximum should probably be set, also. Also, since it is quite easy to
simply increment the month number, I would recommend a suggested
standard time period of one month from today's date.
Once a bug is closed as per the "closeme" tag, then the closer can
remove the tag from the whiteboard and add the "closedme" keyword to
the bug.
What do you think?
Kevin Brosnan
> dev-quality mailing list
> dev-q...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-quality
>
The simplicity of just closeme really appeals to me. Bugzilla can
easily create queries that show last active date.
The date at first seems like a good idea but looses a bit of its
functionally when you start having multiple people marking bugs
closeme. There is no way for me to know which dates you or any other
user have selected.
Picking a date seems wrong to me as well. The last of the month?
Fluctuates between months and years. A static number like the 14th
strikes me as the least troublesome of any of the date choices.
A floating date like the third Friday of a month, well now I need to
consult some calendar to check what the last few third Fridays have
been to "correctly" filter the bugs when Bugzilla has a bug changes
section for just this use case. When editing the whiteboard to avoid
the two week rule I need to think about what date it it is and how
close is it to the third Friday, and if it is too close I need to look
up the third Friday after next.
The Bugzilla search you need is very quick to create. Go to Bugzilla's
advanced query page; whiteboard: closeme, status: unconfirmed,
resolution: ---, replace now with 14d in the bug changes section,
search. Use additional search restrictions as you need. Use the tools
provided, don't create extra bookkeeping.
Gordon P. Hemsley
> >https://lists.mozilla.org/listinfo/dev-quality
>
> The simplicity of just closeme really appeals to me. Bugzilla can
> easily create queries that show last active date.
>
> The date at first seems like a good idea but looses a bit of its
> functionally when you start having multiple people marking bugs
> closeme. There is no way for me to know which dates you or any other
> user have selected.
>
> Picking a date seems wrong to me as well. The last of the month?
> Fluctuates between months and years. A static number like the 14th
> strikes me as the least troublesome of any of the date choices.
>
> A floating date like the third Friday of a month, well now I need to
> consult some calendar to check what the last few third Fridays have
> been to "correctly" filter the bugs when Bugzilla has a bug changes
> section for just this use case. When editing the whiteboard to avoid
> the two week rule I need to think about what date it it is and how
> close is it to the third Friday, and if it is too close I need to look
> up the third Friday after next.
>
> The Bugzilla search you need is very quick to create. Go to Bugzilla's
> advanced query page; whiteboard: closeme, status: unconfirmed,
> resolution: ---, replace now with 14d in the bug changes section,
> search. Use additional search restrictions as you need. Use the tools
> provided, don't create extra bookkeeping.
The date listed along with the "closeme" tag is not the date of last
activity, it is the date at which the bug should be closed. This can
vary based on the bug. The default date I suggested was simply one
month from "today". For example, since today is 2009-02-15, the tag
would be [closeme 2009-03-15]. This, however, can vary, as someone
mentioned that specific module handlers might have different needs.
The benefit of having the date in the tag is that it helps with
sorting bug lists (at least, IMO). The goal is not to have everyone
use the same date; it would vary on a per-bug basis. If someone makes
a change to the bug (even just CCing themselves), that would bump up
the "last changed" date, which would have it lose its usefulness in
this case.
Gary Kwong
closeme YYYY-MM-DD for almost a full year since Thunderbird bugdays
started early last year, here are my views:
- We should keep the closeme tags for bugs that eventually get resolved
incomplete. This allows for searching of bugs that have been resolved
incomplete by closemes in the future via advanced search. Having them
removed makes that real tough, because bug history isn't easily
searchable from advanced search.
- Standardize the date on the closeme every week. For us TB QA folks, we
have bugdays every Thursdays, so we tend to set closemes to the Thursday
of that week, when there will be others running regular searches on
closemes of that day of the week. Thus, the date has a purpose. The same
could be done for other apps, though I'm not advocating that they must
follow in the same footsteps. The duration tends to be at 3 weeks, 2 for
really old (and obviously out-of-date ones, such as those for
unsupported versions with menus that don't exist in new versions
anymore), and some get extended pending more information.
- closemes have occasionally (~5-10% is a guess) stirred up people's
old-time itches, and they (the ones that are cc'ed) will provide
up-to-date info on whether the bug is still valid, or start to re-elicit
discussions at the very least. Most of the time, though, no one mentions
anything. If the bug is eventually valid, we remove the closeme.
- We do put messages like "This bug has been resolved incomplete due to
lack of response to previous question for more information. Feel free to
reopen with reasons explaining why." when closing closeme bugs as
incomplete.
This policy has been (almost) largely responsible for the large decline
in unconfirmed bugs for Thunderbird from 3500+ to ~2200. See the graph
at
https://bugzilla.mozilla.org/reports.cgi?product=Thunderbird&datasets=UNCONFIRMED%3A
The TB QA folks have some existing documentation here:
https://wiki.mozilla.org/Thunderbird:Bug_Triage#Closemes and it has
served us well the past year or so. YMMV, though.
Wayne, Joshua, please fill me in if I've missed anything. Thanks.
Regards,
Gary
nth10sd
Tony Mechelynck
> [...] Ok, and there is one
> question... is each as incomplete resolved bug a closeme bug?
>
> Would be nice to have a clean way here.
>
> Henrik
In the sense of "having had a CLOSEME entry in its whiteboard for some
time before closing", not necessarily. I've seen quite a number of bugs
with a last comment requesting more info from the reporter (info without
which the bug report was too vague for anything to be done about it) and
the reporter never replied, sometimes even after several years. Such
bugs don't necessarily have an "explicit" CLOSEME entry. In such cases,
for what I've seen, the "usual" way to act for a passing-by triager is
to immediately resolve the bug (usually INCOMPLETE), sometimes
(preferably IMHO) with a comment like "No reply to comment #4 after 3½
years => RESO INCO. Feel free to REOPEN if the requested info is
forthcoming." In such cases, I've never seen a CLOSEME whiteboard entry
being added when RESOLVing the bug, and certainly not with a date in the
past (1 month after the question-asking comment for instance).
Best regards,
Tony.
--
"You can't survive by sucking the juice from a wet mitten."
-- Charles Schulz, "Things I've Had to Learn Over and
Over and Over"
Wayne Mery
I lean toward simplicity. At 2am or in a brief fit of triage I have zero
desire to look up what might be the next "standard" date. I pick what
rationally strikes the need and mood - 2 weeks out, first of the month,
3 weeks out, etc.
Further, IMO, the choice of date has no impact in terms of it's future
value or how many bugs eventually get triaged/closed. The IMPORTANT
point is closeme get used frequently, and effectively.
Joshua Cranmer
> - Standardize the date on the closeme every week. For us TB QA folks, we
> have bugdays every Thursdays, so we tend to set closemes to the Thursday
> of that week, when there will be others running regular searches on
> closemes of that day of the week. Thus, the date has a purpose. The same
> could be done for other apps, though I'm not advocating that they must
> follow in the same footsteps. The duration tends to be at 3 weeks, 2 for
> really old (and obviously out-of-date ones, such as those for
> unsupported versions with menus that don't exist in new versions
> anymore), and some get extended pending more information.
About the date: I always use N weeks after today (where N is an element
of {2, 3, 4}, depending on the bug), even though I pretty much only go
through the closemes on Thursday mornings (ET).
> - closemes have occasionally (~5-10% is a guess) stirred up people's
> old-time itches, and they (the ones that are cc'ed) will provide
> up-to-date info on whether the bug is still valid, or start to re-elicit
> discussions at the very least. Most of the time, though, no one mentions
> anything. If the bug is eventually valid, we remove the closeme.
I have noticed that a significant number of bugs have people responding
only about three hours after the bug is closed and not in the
intervening three week period. This may be due to the fact that most
closeme statements do not make mention of the fact that the bug will be
closed in N weeks if there is no response.