should initial bug state really default to NEW for filers with CANCONFIRM?
Dan Mosedale
My suspicion is that it actually results in a bunch of bugs being marked
as NEW that really should go through the confirmation process, and as a
result, our bug database is in a messier, noiser place. Ludo, Wayne,
others, do you guys agree?
Dan
Nathan Tuggy
I'm just ramping up my QA work, so I can't say I have a lot of
experience with this, but what I have seen of my own and others' bugs
definitely tends to support this.
I think it would be a good step forward to default to UNCO for all.
Note: I don't know Bugzilla's administration internals that well; are
you suggesting to change this only for the Thunderbird and MailNews Core
products, and is that possible?
Ron K.
I have a mixed view. The core Dev team I consider qualified to generate a
Bug Report and set a New status. These bugs are often entered in Bugzilla
as a result of issues discovered while working on patches. Some bugs are
entered to split off remaining work so the originating bug can be closed.
Where new reports should default to UNCONFIRMED would be reports from
people such as myself who have bug edit privileges. There is a risk that I
have a local bug case that would not happen on other systems. My report
would need QA to check it on other systems.
--
Ron K.
Who is General Failure, and why is he searching my HDD?
Kernel Restore reported Major Error used BSOD to msg the enemy!
Dan Mosedale
>> I'm just ramping up my QA work, so I can't say I have a lot of
>> experience with this, but what I have seen of my own and others' bugs
>> definitely tends to support this.
>>
>> I think it would be a good step forward to default to UNCO for all.
>>
>>
>> Note: I don't know Bugzilla's administration internals that well; are
>> you suggesting to change this only for the Thunderbird and MailNews
>> Core products, and is that possible?
easier to make happen. I suspect it's worth doing regardless of the
answer to that question.
> I have a mixed view. The core Dev team I consider qualified to
> generate a Bug Report and set a New status. These bugs are often
> entered in Bugzilla as a result of issues discovered while working on
> patches. Some bugs are entered to split off remaining work so the
> originating bug can be closed.
>
> Where new reports should default to UNCONFIRMED would be reports from
> people such as myself who have bug edit privileges. There is a risk
> that I have a local bug case that would not happen on other systems.
> My report would need QA to check it on other systems.
fields, so I don't tend to devote time to thinking about the exact right
value for each of them when I'm filing. In other words, I'm often
qualified to make the decision about whether a bug should be NEW or
UNCONFIRMED, but I usually have other stuff on my mind when I'm filing a
bug, and I suspect this is true of other devs as well. So I feel like
having the default be the thing that is right more of the time and
offers more clarity is likely to be a win.
Dan
Ron K.
A perspective I can understand and the reasoning, simplify getting the bug
filed and fix the details in QA.
Wayne Mery
Are you talking also for people with editbugs, or just canconfirm? (I
doubt there's much difference in numbers - because editbugs is
frequently given out with canconfirm)
I am unsure on several levels:
1. I've personally come across a few, typically repeats from the same
people, but certainly not in significant numbers.
2. Will reporters change the default to NEW when it's appropriate before
commit? In other words, if they don't ...
3. How much more work will it be for triagers to finish up or tidy up
reporter's UNCO bug report? (IOW is it an invitation for reporters to
be even *less* complete in their report than they have been?)
4. Changing the default doesn't change the fact that some people more
than others operate with a loose definition of NEW - an obvious example
is not checking for duplicates before filing, but there are bigger
transgressions
5. Might we be better/sufficiently served if instead, through workflow
(changing their bug to UNCO) and through admonishing words, we educate
those who misfile bugs as NEW?
A parallel to dmose's NEW issue is people not filing bugs as ENH when
it's clear that what's being requested is a behavior change, i.e. not a
"bug".
This failing, and #4 above, aren't limited to newbees! And these two IMO
are the bulk of bugs improperly filed NEW. Which is why when giving out
editbugs privileges my boiler plate blurb is to forewarn the person to
not file NEW if the bug doesn't meet the criteria for NEW.
I don't have a good feel for how much better off we would be, because I
don't pay much attention to NEW bugs in most components, but my sense is
it's not a significant %. Developers or those that look at significant
numbers of NEW bugs may have a better grasp (but then you'd have to also
know bug was filed as NEW, which you won't know unless you check
history). Definitely interested in insightful comments on the magnitude
of the issue.
In sum, I'm not sure having the default be UNCO will change people's way
of thinking about when a bug is properly NEW, that it's worth the hassle
of having to remember to change my bug reports to NEW (which are
probably 95%), and why I list #5 above as food for thought.
--
http://wiki.mozilla.org/Thunderbird:Testing
http://www.spreadthunderbird.com/aff/165/
Siddharth Agarwal
It'll be a bit of a hassle for me, because a lot of the bugs I file come
with patches to fix them, which implies that I believe that there is a
bug in the first place :)
Is a bugzilla setting possible? Or maybe another "privilege" which only
changes the default to new, and which anyone with canconfirm can opt
into by sending a message to an admin.
Siddharth Agarwal
>
> It'll be a bit of a hassle for me, because a lot of the bugs I file come
> with patches to fix them, which implies that I believe that there is a
> bug in the first place :)
Actually, I just realized that I need to change the status to assigned
anyway, so I wouldn't actually lose anything if the default status is
unconfirmed.
Kent James
> ... So I feel like
> having the default be the thing that is right more of the time and
> offers more clarity is likely to be a win.
>
I guess I have a different philosophy about that. Let's look at another
similar example, which is the bug severity. It currently defaults to
"Normal" which is "right more of the time", but is now overloaded to
mean both "Normal" severity and "Unclassified" severity. As a result,
"Normal" is pretty worthless as a measure of bug severity IMHO, because
I cannot tell the difference between Unclassified and Normal.
When I am setting up systems like this, I virtually always setup the
default to mean the equivalent of "Unclassified" so that I can tell the
difference. The act of someone thinking about the bug enough to set a
severity (or NEW versus UNCONFIRMED status in the current case) is a
worthwhile piece of data for looking at bugs.
So I think it is fine if the default is changed to UNCONFIRMED, because
that is very close to UNCLASSIFIED in meaning, while NEW is not. It is
important though that I can change the Status at initial definition
time, but I think that is the current expectation with the change anyway.
rkent
Gervase Markham
> This UI choice was made long ago, in a very different Mozilla world.
> My suspicion is that it actually results in a bunch of bugs being marked
> as NEW that really should go through the confirmation process, and as a
> result, our bug database is in a messier, noiser place.
The idea is that people with CANCONFIRM should be able to file good
bugs. If they aren't, let's take away their CANCONFIRM privs.
If you are saying "all bugs need a triage step", then I have sympathy
with that. One of the things on my list of governance issues:
https://wiki.mozilla.org/GovernanceIssues
is to reopen discussions about Bugzilla's workflow, perhaps including
the introduction of a READY state.
Gerv
Mark Banner
Like you the majority of the bugs I file are either bugs that I know are
bugs, either due to having patches for them or because I've investigated
enough to know it is definitely an issue.
The assigned status is a step that never really gets done with me - if
the bug is assigned to me (via the assignee field), I'm intending on
working on it at some stage and every now and again I review the list
I'm assigned to. I find the assigned status very unnecessary/unclear and
these days when we use nobody@ a lot, then I find the assignee field
much more authoritative.
For bugs I'm unsure about, I know that I'm unsure, and swap the bug
state to unconfirmed. These are generally in the minority.
Standard8
Siddharth Agarwal
> Like you the majority of the bugs I file are either bugs that I know are
> bugs, either due to having patches for them or because I've investigated
> enough to know it is definitely an issue.
>
> The assigned status is a step that never really gets done with me - if
> the bug is assigned to me (via the assignee field), I'm intending on
> working on it at some stage and every now and again I review the list
> I'm assigned to. I find the assigned status very unnecessary/unclear and
> these days when we use nobody@ a lot, then I find the assignee field
> much more authoritative.
I agree that getting rid of the distinction between new and assigned
would be great.
> For bugs I'm unsure about, I know that I'm unsure, and swap the bug state to unconfirmed. These are generally in the minority.
Same here.
Wayne Mery
> On 2/4/2010 1:46 PM, Dan Mosedale wrote:
>> ... So I feel like
>> having the default be the thing that is right more of the time and
>> offers more clarity is likely to be a win.
>>
>
> I guess I have a different philosophy about that. Let's look at another
> similar example, which is the bug severity. It currently defaults to
> "Normal" which is "right more of the time", but is now overloaded to
> mean both "Normal" severity and "Unclassified" severity. As a result,
> "Normal" is pretty worthless as a measure of bug severity IMHO, because
> I cannot tell the difference between Unclassified and Normal.
>
> When I am setting up systems like this, I virtually always setup the
> default to mean the equivalent of "Unclassified" so that I can tell the
> difference. The act of someone thinking about the bug enough to set a
> severity (or NEW versus UNCONFIRMED status in the current case) is a
> worthwhile piece of data for looking at bugs.
I disagree that SEVERITY has significant similarity of use/misuse:
a) it's even less uniformly applied than status / more routinely abused,
b) it's not a workflow problem strictly speaking,
c) it's only overloaded by those people who insist either in filing the
bug or in interpreting the bug that it means priority when it very
clearly does not.
That said, my experience is severity, like NEW, is not significantly
misused. I'm a stickler for proper severity setting and almost every bug
I touch gets a look at the field. In fact I occasionally sweep large
sets of bugs in triage work:
* last week I swept through filter bugs - changed very few severities
* yesterday morning I queried all NORMAL+NEW bugs filed in the last 60
days - I selected ~2 dozen of the ~180 bugs for a quick pass review, and
of I changed the severity of only 3-4 (2%), and changed status of one to
UNCO.
I do suggest they are similar in that, if the bug form asked 2-3
questions of the reporter instead of offering a field to set, then
severity might be more uniformly set programatically.
> So I think it is fine if the default is changed to UNCONFIRMED, because
> that is very close to UNCLASSIFIED in meaning, while NEW is not. It is
> important though that I can change the Status at initial definition
> time, but I think that is the current expectation with the change anyway.
>
> rkent
I'm in gerv's camp. Unless data indicates it's a significant issue, we
shouldn't create significantly more triage work. We should rather be
looking for ways to reduce triage work. I'd rather change a few bugs
per week from NEW back to UNCO or alter severity, than have all new bugs
be filed UNCO. In that line of thought, please note:
* we do zero follow up with people who get bugzilla privileges (a 60 day
checkup), and bugzilla (afaik) has no means of doing so in an automated way
* are there uniform standards for what we tell people when they get privs?
* the greatest abusers may be those who got privs 5-10 years ago (eg.
who insist on abusing severity to suit their personal needs, who
routinely don't check for dup before filing, etc)
* I'm not saying we shouldn't improve areas like dmose suggests, but
look at the DUP line of this chart http://bit.ly/9Rcn36 and tell me we
don't have greater room for improvement and reducing triage workload by
improving the bug creation process.
Having gone a bit off topic ... I close by saying thank goodness we
haven't made it easier for people to file bugs! (which I argued in
favor of a year or two ago - gulp)
Dan Mosedale
>> So I think it is fine if the default is changed to UNCONFIRMED, because
>> that is very close to UNCLASSIFIED in meaning, while NEW is not. It is
>> important though that I can change the Status at initial definition
>> time, but I think that is the current expectation with the change
>> anyway.
> shouldn't create significantly more triage work. We should rather be
> looking for ways to reduce triage work. I'd rather change a few bugs
> per week from NEW back to UNCO or alter severity, than have all new
> bugs be filed UNCO.
that I as a dev feel like there's too much bugmail, and so I was
proposing a change that would push it out of my purview into what I felt
was the more correct place. It makes sense to me that you, as one of
the, uh, "beneficiaries" of that push who also already has too much
bugmail, wouldn't see that as an improvement. :-)
To be fair, I don't feel like this would be a huge win, but suspected
when I proposed it that it would be something that would make sense to
most people. That's less clear to me know.
The core problem, that we have too many untriaged bugs, still remains.
From a QA perspective, does it feel like the course we're currently on
will catch up with that over time?
> In that line of thought, please note:
> * we do zero follow up with people who get bugzilla privileges (a 60
> day checkup), and bugzilla (afaik) has no means of doing so in an
> automated way
Very true.
> * are there uniform standards for what we tell people when they get
> privs?
Not that I'm aware of, and I'm one of the people granting those privs.
It's possible that some document exists that would be helpful here.
> * the greatest abusers may be those who got privs 5-10 years ago (eg.
> who insist on abusing severity to suit their personal needs, who
> routinely don't check for dup before filing, etc)
It would certainly be interesting to understand more about how much
effort people typically put into DUP-checking. Hard data to get, I suspect.
> * I'm not saying we shouldn't improve areas like dmose suggests, but
> look at the DUP line of this chart http://bit.ly/9Rcn36 and tell me we
> don't have greater room for improvement and reducing triage workload
> by improving the bug creation process.
This touches again on the fact that you're arguing from the triager
perspective, and I'm arguing from a developer perspective. Since these
just two sides of a many-sided coin (or something), I wonder if we don't
do a little better to try and look at the problem holistically. Before
I speculate too much further, I'd still be super interested in a couple
of things:
* A paragraph by you about what the current state of triage feels like,
both as a person in the triage community, and as someone who is deep
enough inside the picture to have a feel for what the trajectory looks
like, what the pain points are, and what's working well.
* Thoughts from Ludo.
Dan
Ron K.
> On 2/5/10 5:03 AM, Wayne Mery wrote:
>> * the greatest abusers may be those who got privs 5-10 years ago (eg.
>> who insist on abusing severity to suit their personal needs, who
>> routinely don't check for dup before filing, etc)
> It would certainly be interesting to understand more about how much
> effort people typically put into DUP-checking. Hard data to get, I
> suspect.
>
> Dan
>
A number of months ago I commented that DUP checking might be easier if
Bugzilla offered a wider search of recent reports. Currently if offers past
24 hours, which is not wide enough. I think a spinbox with a range of 1 to
7 days would be useful. In the past I have filed a few reports that were
DUPs because someone else spotted the issue 1-2 days before I did. Those
cases the bug was not in the Recent search.
David Bienvenu
> On 04/02/10 20:06, Dan Mosedale wrote:
>> This UI choice was made long ago, in a very different Mozilla world.
>> My suspicion is that it actually results in a bunch of bugs being marked
>> as NEW that really should go through the confirmation process, and as a
>> result, our bug database is in a messier, noiser place.
> The idea is that people with CANCONFIRM should be able to file good
> bugs. If they aren't, let's take away their CANCONFIRM privs.
that. But the default should be CONFIRMED, I think.
- David
Dan Mosedale
part because in the case of bugs that I encounter in day-to-day use, I
have no idea whether my bug is real or the result of the wacky set of
extensions I use with my regular profile that's been around forever. I
certainly could take the time to do that with a single bug, but the
reality is that I discover enough bugs that I can't spend the time to do
all of them. So my choice is either "file a few high-quality bugs" or
"file a bunch of lower quality bugs". I've been assuming that the
latter is the better choice to make.
It would be interesting to understand how other regular bug-filers deal
with this...
Dan
Wayne Mery
My brain keeps tempting me to say this .. for the Uber-Forgetful it may
be nice to have a choice/preference, regardless what the default is :)
But that does seem to be overkill.
Wayne Mery
Oh, it's not hard to pick out the people who file low quality bugs :)
But seriously, yes, better to have the bug than not. But if you are the
exception rather than the rule, as per my post a few seconds ago ... "it
may be nice to have a [per user] choice/preference, regardless what the
default is :)"
Gerv, might this be easily accomplished via extension or greasemonkey
script? To have some fields altered from the default? I use Jesse's
buzilla greasemonkey scripts, but I've never munged one up to do my bidding.
Nathan Tuggy
Gary Kwong
also filing in the extra lower quality ones. Gives the best of both
worlds, but the time factor would be an issue.
-Gary
Ludovic Hirlimann
> On 2/5/2010 1:05 AM, Kent James wrote:
>> On 2/4/2010 1:46 PM, Dan Mosedale wrote:
>>> ... So I feel like
>>> having the default be the thing that is right more of the time and
>>> offers more clarity is likely to be a win.
>>>
Sorry for the delay I've been busy with plenty of things. I'm in favor
of leaving the the default to NEW instead of UNCO - because most of the
time bugs that are filled NEW are properly filled. I did file a few last
year that were invalid - but now before filling I always make sure to
start Thunderbird in -safe-mode to verify that one of my add-on is not
the culprit.
And I totally agree with Gerv on :
>
> The idea is that people with CANCONFIRM should be able to file good
> bugs. If they aren't, let's take away their CANCONFIRM privs.
>
The only thing left here is spotting bad reporters and communicating
with them about it and ree-nforcing the nice email you get when you get
can confirm privs.
>> I guess I have a different philosophy about that. Let's look at another
>> similar example, which is the bug severity. It currently defaults to
>> "Normal" which is "right more of the time", but is now overloaded to
>> mean both "Normal" severity and "Unclassified" severity. As a result,
>> "Normal" is pretty worthless as a measure of bug severity IMHO, because
>> I cannot tell the difference between Unclassified and Normal.
>>
>> When I am setting up systems like this, I virtually always setup the
>> default to mean the equivalent of "Unclassified" so that I can tell the
>> difference. The act of someone thinking about the bug enough to set a
>> severity (or NEW versus UNCONFIRMED status in the current case) is a
>> worthwhile piece of data for looking at bugs.
>
> I disagree that SEVERITY has significant similarity of use/misuse:
> a) it's even less uniformly applied than status / more routinely abused,
It's not missused, it's just not set by reporters except for enh and
most of the time for crash reports. Which makes the other values a bit
less useful to the devs.
> b) it's not a workflow problem strictly speaking,
> c) it's only overloaded by those people who insist either in filing the
> bug or in interpreting the bug that it means priority when it very
> clearly does not.
One that never get's used is priority. And I think That one should be
set by devs to prioritize their bug queues.
> That said, my experience is severity, like NEW, is not significantly
> misused. I'm a stickler for proper severity setting and almost every bug
I agree. But people have a stronger feeling about SEV than they have
with UNCO/NEW.
> * we do zero follow up with people who get bugzilla privileges (a 60 day
> checkup), and bugzilla (afaik) has no means of doing so in an automated way
We could send them an email, and ask for feedback ....
> * are there uniform standards for what we tell people when they get privs?
Not that I know of. The email you send is good, might be worth
discussing that on mdq ?
> * the greatest abusers may be those who got privs 5-10 years ago (eg.
> who insist on abusing severity to suit their personal needs, who
> routinely don't check for dup before filing, etc)
Yes that's true - see my idea above about sending them your email as a
nice reminder.
So my state here would be to :
1) Not change the default.
2) Send a gentle reminder to the one who we've caught abusing.
3) If they continue - remove the privs.
Ludo
--
Ludovic Hirlimann MozillaMessaging QA lead
http://www.spreadthunderbird.com/aff/79/2
Ludovic Hirlimann
make sure this starts as a global thread so we don't miss it.
Ludovic Hirlimann
So you'd keep the bug new but assigned to you instead of nobody ?
Gervase Markham
> make sure this starts as a global thread so we don't miss it.
Oh, you won't miss it :-)
Gerv
Siddharth Agarwal
> So you'd keep the bug new but assigned to you instead of nobody ?
Well, I'd prefer that NEW and ASSIGNED are simply merged into one.
Ludovic Hirlimann
So bugs would only be changed by devs. In that world how you'ld I ping
you on a specific bug , cc you ?
Ludo