Finding a way to better manage component-less features
Anthony Hughes
do not have components in Bugzilla. My own personal examples include
Doorhanger notifications and Switch-to-Tab.
As the QA engineer attached to both features, it is my job to triage
bugs and vet Litmus results on a daily (or at least weekly) basis. I've
been somewhat coping by adding arbitrary tags to the whiteboard field of
any bugs I happen to come across. In the case of Doorhanger
notifications, I have to search for "doorhanger" and "popupnotification"
in the summary across all components, manually adding [doorhanger] to
the whiteboard. In the case of Switch-to-Tab, I have to search for
"switch" and "tab" across Locationbar, Toolkit:General, and
Firefox:General adding [switch-to-tab] to the whiteboard. This last
example is a huge minefield since there are thousands of bugs unrelated
to Switch-to-Tab which mention these words.
I find it hard to believe that I am the only person who has this problem.
I'm not sure what the solution is, in fact I have no clue. All I know
is that we need to somehow find a better way to manage these bugs. I'm
seriously afraid that there are tens (or more) bugs which are completely
off my radar.
At this point I'd like to open the discussion up to any ideas people
have to mitigate this issue and to add to the list of usecases.
Thanks
--
Anthony Hughes
Mozilla QA Engineer
Automation, Releases, and Community
Mike Beltzner
Could also do dependency chains, but that often breaks Bugzilla's little brain. I think the more general issue at hand though can be summarized as:
- we have too many components* in the Firefox product, meaning that bugs often get misplaced, and
- Firefox::General is a pretty big dumping ground and very rarely triaged
If we're to take Bugzilla seriously as a running database of existing issues, we need to invest more in triaging incoming bugs, duping them to existing ones and filing them in a system that's approachable and understandable by those who are trying to work on those issues.
(* many bugs end up touching multiple components, a key indicator that our taxonomy is too precise)
cheers,
mike
Anthony Hughes
and bugs resolved in the last week in my component areas. I make a
point of triaging these both daily and weekly; this has given me a
pretty good handle on the areas which have components. However, these
queries do not capture what is sitting in the "General" components, or
what is sitting in the wrong component (exactly what you point out).
Any suggestions on how this can be improved?
In the short term, I'd like any tips the bugzilla veterans can pass on
to us less experienced.
In the long term we need to continue this discussion and work toward a
better solution.
David Ascher
I had a talk at the Ubuntu conf with someone (whose name I unfortunately
can't recall) who'd done data mining of bug trackers, and with some
interesting clustering work was able to, IIRC, compute some of the
statistical links between words. If we could get something like that
working, and inline it in the bug creation process (or a new bug
triaging/recategorization process), we could have things like:
"From the words you use ('box', 'hanging', 'pointy'), we think this may
be a bug related to what we call doorhangers ([insert link & picture]).
Is that true? [y/n]
In general, bugzilla's a big enough corpus that some text analytics
could be very handy.
--da
"mitcho (Michael 芳貴 Erlewine)"
> compute some of the statistical links between words. If we could get something like that working, and inline it in the bug creation process (or a new bug triaging/recategorization process), we could have things like:
>
> "From the words you use ('box', 'hanging', 'pointy'), we think this may be a bug related to what we call doorhangers ([insert link & picture]). Is that true? [y/n]
>
> In general, bugzilla's a big enough corpus that some text analytics could be very handy.
I personally would be happy to help work on such a feature; I've worked on similar projects in some other contexts. It's possible that this could suggest existing bugs/dupe pairs as well.
mitcho
--
mitcho (Michael 芳貴 Erlewine)
mit...@mitcho.com
http://mitcho.com/
linguist, coder, teacher
Christian Legnitto
> On Feb 7, 2011, at 5:03 PM, David Ascher wrote:
>> compute some of the statistical links between words. If we could get something like that working, and inline it in the bug creation process (or a new bug triaging/recategorization process), we could have things like:
>>
>> "From the words you use ('box', 'hanging', 'pointy'), we think this may be a bug related to what we call doorhangers ([insert link & picture]). Is that true? [y/n]
>>
>> In general, bugzilla's a big enough corpus that some text analytics could be very handy.
>
> I personally would be happy to help work on such a feature; I've worked on similar projects in some other contexts. It's possible that this could suggest existing bugs/dupe pairs as well.
>
FYI, Bugzilla 4+ has dupe detection code (see https://landfill.bugzilla.org/bugzilla-tip/page.cgi?id=release-notes.html#v40_feat_dup). Not sure how good it is but when mozilla upgrades (soon!) we'll get that feature for free.
Thanks,
Christian
Chris Hofmann
If we have a new feature area, and new code, and new owner for that code
why don't you file a bug and get a new component added to bugzilla?
Failing to do this only makes the problem of an overload in General bugs
that get no trigage and attention even worse than it ready is.
Sounds like we need
Product: Firefox - Component: Doorhangers
but we already have Firefox: Tabbed Browser
so "Switch to Tab" probably belongs under that feature area.
On 2/7/11 1:59 PM, Anthony Hughes wrote:
> On 11-02-07 1:53 PM, Mike Beltzner wrote:
>> My suggestion for everything these days: whiteboard tags.
>>
freeform whiteboard tags to track features areas might be a short term
solution but the cost of this eventual gets out of control.
there is no discoverability of how to mark bugs, and no consistency
across people or time. That makes managing bugs and harvesting good
data out of bugzilla impossible.
keywords offer some level of discoverability and consistency in marking.
but if we have new features we ought to have new components.
>> Could also do dependency chains, but that often breaks Bugzilla's
>> little brain. I think the more general issue at hand though can be
>> summarized as:
>>
>> - we have too many components* in the Firefox product,
too many? or just the wrong set of components that just aren't good
matches for the set of features we have now?
>> meaning that bugs often get misplaced, and
>> - Firefox::General is a pretty big dumping ground and very rarely
>> triaged
>>
to few components also turns "General" into a dumping ground. The trick
is to have the right set of components with good names, and easy
visibility/find-ability of all the choices.
-chofmann
Aakash Desai
1. In an unconfirmed/new state
2. Don't have someone assigned to them
3. Are not nominated to blocking any release
4. Filed over the past week
I think that funnels down the number of bugs to a pretty manageable amount.
-- Aakash
----- Original Message -----
From: "\"mitcho (Michael 芳貴 Erlewine)\"" <mit...@mitcho.com>
To: dev-pl...@lists.mozilla.org
Sent: Monday, February 7, 2011 2:20:44 PM
Subject: Re: Finding a way to better manage component-less features
On Feb 7, 2011, at 5:03 PM, David Ascher wrote:
> compute some of the statistical links between words. If we could get something like that working, and inline it in the bug creation process (or a new bug triaging/recategorization process), we could have things like:
>
> "From the words you use ('box', 'hanging', 'pointy'), we think this may be a bug related to what we call doorhangers ([insert link & picture]). Is that true? [y/n]
>
> In general, bugzilla's a big enough corpus that some text analytics could be very handy.
I personally would be happy to help work on such a feature; I've worked on similar projects in some other contexts. It's possible that this could suggest existing bugs/dupe pairs as well.
mitcho
--
mitcho (Michael 芳貴 Erlewine)
mit...@mitcho.com
http://mitcho.com/
linguist, coder, teacher
_______________________________________________
dev-planning mailing list
dev-pl...@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-planning
Benjamin Smedberg
>
> If we have a new feature area, and new code, and new owner for that
> code why don't you file a bug and get a new component added to
> bugzilla? Failing to do this only makes the problem of an overload in
> General bugs that get no trigage and attention even worse than it
> ready is.
>
> Sounds like we need
>
> Product: Firefox - Component: Doorhangers
fragmented. Each doorhanger already belongs to some component. I don't
think it makes sense to also have a generic component for all doorhangers.
Anthony, how do you do triage? I think that most developer/owners do
this triage "constantly" by watching the QA emails for their components
in bugzilla. This doesn't work so well for "general", however. For that,
I suggest using keyword searches to find "general" bugs which might
belong to your areas of interest.
--BDS
Anthony Hughes
too noisy. For example, I have a query set up to watch for
"Firefox:Session Store bugs with STATUS set to RESOLVED between -1w and
NOW". I have many different queries like this. It give me very low
noise results, however it fails for features without components.
That's not to say I'm not open to the use of QA Contact. However, how
does one manage the noise?
Thanks
Benjamin Smedberg
> That's not to say I'm not open to the use of QA Contact. However, how
> does one manage the noise?
benj...@smedbergs.us for my "must-read" bugzilla mail. It watches no
(or few?) bugs. I use bsme...@gmail.com for all my watching (and I
watch a high number). I don't always read these bugs, and I scan them
very aggressively, marking them read en-masse. It's a system that works
for me, though everybody does something slightly different. What's
useful about it to me is that the entire comment is available in my
email, so I don't have to go through and load each bug separately as I
would with a bugzilla query.
You can use the various x-bugzilla headers to accomplish the same thing
with one bugzilla account and email filters, but that seems like a more
complicated setup to me.
--BDS
Joshua Cranmer
dose of the delete key (I also use TB's tagging to indicate, e.g.,
whether or not it's a bug that I need to specifically pay attention to).
For the stuff I want a passive eye on, I keep a RSS folder with a few
different queries of updates. That way I can just mark everything as
read, and only look at a bug if it really interests me.
Nicholas Nethercote
>
> You can use the various x-bugzilla headers to accomplish the same thing with
> one bugzilla account and email filters, but that seems like a more
> complicated setup to me.
I filter in my single gmail account. Gmail doesn't let you filter on
headers (AFAICT), here's how to do it with the in-message text:
Nick
Marco Bonardo
> My suggestion for everything these days: whiteboard tags.
Today I added whiteboard [about-home] tag to all bugs I found related to
about:home. Discussing through mail with Anthony about [switch-to-tab]
tag, appeared clear that, while the solution is clean and can work, we
found similar problems:
1. finding all bugs is time-consuming, because the words you look for
can be used in many contexts (about home, start page, firefox start,
...) and in bugs that existed before your feature.
2. you are not guaranteed to find all bugs. Indeed I missed one, pretty
important, but I did know it existed and I added it. This worked because
previously I had read all bugs through the QA contact.
3. New bugs are hardly going to be filed with the right tag, you have to
poll the QA contact daily.
Adding new components as an alternative is going to create
fragmentation, and none of the three above issues is really solved,
since bugs can still exist and being filed under the wrong component.
I guess the current status doesn't allow to make much better than this,
but it's worrysome that tracking bugs depends so heavily on a QA person
or the developer that worked on a feature, having to look for each bug
and tag it. This is not going to work reliably when, after some time,
attention on the feature decreases.
Plus there should also be a way to "teach" others about tags they can
use, I don't expect all persons helping triage to be aware we have
[about-home], [switch-to-tab], [doorhanger] whiteboard tags as of today.
Marco
Christian Legnitto
> Il 07/02/2011 22:53, Mike Beltzner ha scritto:
>> My suggestion for everything these days: whiteboard tags.
>
> Today I added whiteboard [about-home] tag to all bugs I found related to about:home. Discussing through mail with Anthony about [switch-to-tab] tag, appeared clear that, while the solution is clean and can work, we found similar problems:
>
> 1. finding all bugs is time-consuming, because the words you look for can be used in many contexts (about home, start page, firefox start, ...) and in bugs that existed before your feature.
Keywords are more appropriate but lack flexibility.
> 2. you are not guaranteed to find all bugs. Indeed I missed one, pretty important, but I did know it existed and I added it. This worked because previously I had read all bugs through the QA contact.
I don't think any method will provide that guarantee.
> 3. New bugs are hardly going to be filed with the right tag, you have to poll the QA contact daily.
A proper keyword system would have configurable triggers, such as "add keyword foo whenever the bug is moved into x component" or "if a new bug is filed in this component with these words in the summary, add keyword y".
> Adding new components as an alternative is going to create fragmentation, and none of the three above issues is really solved, since bugs can still exist and being filed under the wrong component.
>
> I guess the current status doesn't allow to make much better than this, but it's worrysome that tracking bugs depends so heavily on a QA person or the developer that worked on a feature, having to look for each bug and tag it. This is not going to work reliably when, after some time, attention on the feature decreases.
>
> Plus there should also be a way to "teach" others about tags they can use, I don't expect all persons helping triage to be aware we have [about-home], [switch-to-tab], [doorhanger] whiteboard tags as of today.
We're basically abusing the whiteboard as a poor man's tag/keyword system because Bugzilla's keyword system blows goats.
Christian
timeless
<cleg...@mozilla.com> wrote:
> A proper keyword system would have configurable triggers, such as "add keyword foo whenever the bug is moved into x component" or "if a new bug is filed in this component with these words in the summary, add keyword y".
yes, triggers would be nice.
> We're basically abusing the whiteboard as a poor man's tag/keyword system because Bugzilla's keyword system blows goats.
we could implement a policy of "if there are >x instances of SW y,
create a keyword". a more detailed version of this would involve
someone (anyone really, but ideally there'd be a trigger worst case)
filing a bug and cc'ing the people who've used the annotation asking
for a description. the keyword can be created immediately with a
placeholder description and changed when someone gets around to
providing one.
the people who manage bugzilla components and keywords are friendly
but not all knowing. i may know about all sw markings in existence (or
more accurately, my mailbox probably does), but that doesn't mean i
can figure out what they all mean in reasonable time.
to make things easier on people learning about sw markings, people
could actually file a bug "Use sw:[xxxx] for x" with an explanation of
what the marker means and when to use it they could even assign the
bug to themselves (indicating they "own" the marker). they could also
leave it in e.g. components and keywords, which would make it easier
for people to discover what the markers mean. Now, if that's really
too heavy for people, ok, but otherwise, i really wonder how people
figure these out.
from my perspective, running comments in a bug explaining changes in
how a marker should be used don't bother me at all. but i'm rather
desensitized wrt bugmail load.
someone asked about how people read bugmail....
I currently use:
[Filters]
Matches: ((crash OR crashes OR crashed OR abend OR abends OR coverity
OR klocwork OR "qacontact:timeless" OR bemail OR "webtools.bugs" OR
"insp...@extension.bugs" OR "j...@core.bugs"))
Do this: Apply label "t"
- it's slightly out of date, but oh well.
[Quick Links]
tag me: label:t -label:r
bugs: (label:t OR label:r) is:unread
i click "tag me", select all, and apply label "t" <- this is because
gmail is strange, a label applied by a filter is applied to an
individual message, not the conversation, and matches like 'is:unread'
actually apply to individual messages in a conversation, not the
conversation.
i click "bugs" to actually read bugmail.
I also have a query called "security month":
https://bugzilla.mozilla.org/query.cgi?chfield=%5BBug%20creation%5D&chfieldfrom=-1m&chfieldto=Now&field0-0-0=bug_group&type0-0-0=notequals&value0-0-0=0&known_name=security-month
this is similar to the triage query Anthony Hughes mentioned.
Philip Chee
I think mkanat mentioned that Bugzilla does support tagging. It's off by
default but can be enabled via your bugzilla preferences settings. Not
exactly a satisfactory solution perhaps, but better than nothing.
Phil
--
Philip Chee <phi...@aleytys.pc.my>, <phili...@gmail.com>
http://flashblock.mozdev.org/ http://xsidebar.mozdev.org
Guard us from the she-wolf and the wolf, and guard us from the thief,
oh Night, and so be good for us to pass.
Philip Chee
> Plus there should also be a way to "teach" others about tags they can
> use, I don't expect all persons helping triage to be aware we have
> [about-home], [switch-to-tab], [doorhanger] whiteboard tags as of today.
I think that this means that the extremely obscure Bugzilla tagging system:
1. Needs to have a significantly better UI, and.
2. Significantly greater visibility.
2a. Tag clouds? Perhaps Addons.mozilla which has a tagging feature might
like to chime in with their 2 cents.
timeless
> I think that this means that the extremely obscure Bugzilla tagging system:
> 1. Needs to have a significantly better UI, and.
> 2. Significantly greater visibility.
> 2a. Tag clouds? Perhaps Addons.mozilla which has a tagging feature might
> like to chime in with their 2 cents.
bugzilla's tagging feature is iirc done by maintaining per user
queries, which makes them useless for your average collaborative
effort.
Daniel Veditz
> bugzilla's tagging feature is iirc done by maintaining per user
> queries, which makes them useless for your average collaborative
> effort.
Yes, bugzilla "tags" are named queries that consist of a list of bug
numbers. As with other named queries they can be shared (read-only),
but can't be managed by a group. To enable tags or share named
queries see your bugzilla preference panes.
> On Tue, Feb 8, 2011 at 1:38 PM, Philip Chee <phili...@gmail.com> wrote:
>> 2a. Tag clouds? Perhaps Addons.mozilla which has a tagging feature might
>> like to chime in with their 2 cents.
AMO used to have a global tagging mechanism. Recent site updates
have restricted the ability to authors creating the tags for their
own add-ons. I imagine that change was prompted by experience that
would be useful to know about, although BMO is not exactly the same
and we could do things like restrict tagging to users trusted with
editbugs privileges.
-Dan Veditz