PSA: Archiving Old Unconfirmed Issues

[Matthew Yuan]

Team,

Currently there are 14,612 unconfirmed issues that do not have an owner and has not been modified within the past year.  At this point it should be safe to say that these issues will likely not receive any additional attention.

Starting Friday, we will begin to run automation which will change the status from unconfirmed into archived with the goal of keeping our issue tracking system efficient and palatable.

If you want to see a list of issues where auto archived you can access the full list through the following query.

Thanks,

MKY

[Peter Kasting]

Is there a query we can run now to get the list of what you're going to do this to (so I can search for things that shouldn't be archived)?

PK

[Matthew Yuan]

Yes, link is as follows:

https://code.google.com/p/chromium/issues/list?can=2&q=status%3AUnconfirmed+modified-before%3Atoday-365

--

MKY

Matthew Yuan |

Technical Program Manager |

matth...@google.com |

1-650-214-1277

[Rachel Blum]

Can we do the same thing to Untriaged, no owner?

On Thu, May 21, 2015 at 3:59 PM, Matthew Yuan <matth...@chromium.org> wrote:

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev

[Peter Kasting]

On Thu, May 21, 2015 at 6:09 PM, Rachel Blum <gr...@chromium.org> wrote:

Can we do the same thing to Untriaged, no owner?

Please no!  Untriaged are MUCH more likely to be useful than Unconfirmed, since a dev has touched those at least once (e.g. to file it).

PK 

[Peter Kasting]

On Thu, May 21, 2015 at 6:08 PM, Matthew Yuan <matth...@google.com> wrote:

Yes, link is as follows:

https://code.google.com/p/chromium/issues/list?can=2&q=status%3AUnconfirmed+modified-before%3Atoday-365

From this list, it seems apparent that you haven't posted anything on these bugs requesting an update.  And the very first such bug I looked at was actually a useful one filed by a Chromium dev.

It's much more reliable to close these sorts of bugs using a process like Mozilla does in similar cases: run a pass saying there's been no activity in a year and the bug will be auto-closed in another, say, two weeks; then run the actual archiving pass a couple weeks later.

Anything that actually gets an update during those two weeks has a much higher likelihood of having actual value.

Please use a process like this instead of just Archiving these bugs with no warning, since end users cannot un-archive the bugs in cases where there was still value.

PK

[Peter Kasting]

On Thu, May 21, 2015 at 6:27 PM, Rachel Blum <gr...@chromium.org> wrote:

I'd really like to see numbers on that usefulness.

Anecdotally, from just starting to go through some of the "unconfirmed" bugs here, more than half of the ones I've looked at have been valuable.  I would expect anything untriaged to be better than that.

 

Whenever I use crbug, I feel like we've become digital hoarders, keeping everything around just in case. There are untriaged bugs that haven't been touched in 5 years, and don't have an owner. I'd say it's a fairly safe bet to assume they're dead.

If it's easy enough to show that they're dead, then doing that manually doesn't take that long; I'm going to go through about 1% of the reports linked here right now and take action on all of them.

Honestly, I don't really see what harm it has to leave a bug open.  I frequently search open bugs to see if an issue has been previously reported or is longstanding.  If that issue has been archived due to no action and I don't find the bug, that misleads me about the longevity and/or severity of the issue.  I would vastly prefer we never auto-close bugs at all.  If there's a real slowdown issue happening here, that's a possible reason ("fix the bug tracker" would be even better), but "palatable" seems like a non-reason.

PK

[Rachel Blum]

I'd really like to see numbers on that usefulness. (Is there a way to query crbug programmatically?)  Whenever I use crbug, I feel like we've become digital hoarders, keeping everything around just in case. There are untriaged bugs that haven't been touched in 5 years, and don't have an owner. I'd say it's a fairly safe bet to assume they're dead. 

That said, fully supportive of the "ask first, archive later" approach. If the somebody is still replying, clearly there _is_ value in that bug. 

[Rachel Blum]

, more than half of the ones I've looked at have been valuable.

Valuable in theory? Well possible. But looking at our bug count growth (and the age of bugs), they are obviously not valuable enough to take action on. I'd rather use a "we're going to close this soon" sweep to surface the ones that actually still matter.

Honestly, I don't really see what harm it has to leave a bug open

Unless you've got very finely tuned searches, they clutter up results. They also pretend that we actually care about them, when in reality we do not. 

If that issue has been archived due to no action and I don't find the bug, that misleads me about the longevity and/or severity of the issue.  I would vastly prefer we never auto-close bugs at all. 

Archived bugs are still part of search, are they not?

 

I'd suspect we both have a different approach to the bug tracker - you see it as a giant notebook, I see it as the backlog of things we need to do. 

[Dan Sinclair]

For the Cr-Blink-Layout label I've been trying to go through some of the older bugs, a few each week. It seems like the majority of the really old ones end up being already fixed, but you do hit a few that are real bugs.

It's slow going to manually check each one, but having the old bugs has been useful. I'd be worried about the number of valid bugs we'll close that no-one can be bothered to comment on because we haven't touched them in so long.

dan

[Peter Kasting]

On Thu, May 21, 2015 at 6:51 PM, Rachel Blum <gr...@chromium.org> wrote:

, more than half of the ones I've looked at have been valuable.

Valuable in theory? Well possible. But looking at our bug count growth (and the age of bugs), they are obviously not valuable enough to take action on.

I wonder why I'm taking action on hundreds of them tonight then.

I don't really think they aren't valuable enough.  I think our triage process is broken.  I'm part of the problem here: I don't sweep the DB for bugs with "omnibox" in them regularly, for example.

Maybe we should treat this as a sign that we need to invest more in the triage process.

Honestly, I don't really see what harm it has to leave a bug open

Unless you've got very finely tuned searches, they clutter up results.

If we closed all these bugs, we'd reduce the number of open bugs by about 14%.  Presumably in your searches that means you'd see a 14% decrease in the number of results.  For most of my searches, that wouldn't make much difference to me finding what I was looking for.

Especially if, as I said, these bugs were precisely what I was looking for.

 

They also pretend that we actually care about them, when in reality we do not.

That's a reason I agree with, but that's primarily a problem for bugs that _are_ getting activity (hundreds of "WHY HAVEN'T YOU FIXED THIS" comments).  If a bug has really been abandoned, then no one is watching it feeling misled about our level of caring.

If that issue has been archived due to no action and I don't find the bug, that misleads me about the longevity and/or severity of the issue.  I would vastly prefer we never auto-close bugs at all. 

Archived bugs are still part of search, are they not?

They're treated as closed.  So you can only find them if you look for closed bugs, which also turns up all the fixed bugs, wontfixed, duplicate, etc.

You can change your search to "all bugs" and then look just for status=archived, but that's a separate search and is slower and inconvenient.  The entire point is that I want these to appear by default in searches for open bugs because they're legit problems and I want to know that.

I'd suspect we both have a different approach to the bug tracker - you see it as a giant notebook, I see it as the backlog of things we need to do.

Yes, I've observed this dichotomy in developers before.  However, the tools are already there to annotate the notebook sufficiently so you can easily tell what things you actually need to do.  Presumably, something you've agreed you need to do has a priority, assignee, or some other information done when triaging it.  For untriaged bugs, we don't _know_ what needs to be done, and indeed, what needs to be done is that we need to triage the bugs.

PK

[Ryan Sleevi]

On Thu, May 21, 2015 at 7:05 PM, Peter Kasting <pkas...@chromium.org> wrote:

<snip>

Maybe we should treat this as a sign that we need to invest more in the triage process.

<snip>

I'd suspect we both have a different approach to the bug tracker - you see it as a giant notebook, I see it as the backlog of things we need to do.

Yes, I've observed this dichotomy in developers before.  However, the tools are already there to annotate the notebook sufficiently so you can easily tell what things you actually need to do.  Presumably, something you've agreed you need to do has a priority, assignee, or some other information done when triaging it.  For untriaged bugs, we don't _know_ what needs to be done, and indeed, what needs to be done is that we need to triage the bugs.

At the risk of a me-too, I wholeheartedly agree with Peter here and many of his explanations here, but especially these two:

1) I think the number of bugs open and in this state is symptomatic is a break in the triage process.

My experience in the past several years is that as Chromium has grown, more and more subteams have developed and gone and created silos. Maybe it was always the case, and it just wasn't externally visible, but it does seem to be the case now. The number of mailing lists forking off chromium-dev (net-dev, security-dev, etc) I think are indicative of that. I'm not arguing it's a bad thing at all, but I think it ties in to the next point - that as these subteams have developed, there are vastly different approaches to triage, if at all.

Again, my experience several years ago was that Chromies were generally all over the codebase, and Chromiues were generally all over the bug tracker triaging. As more people specialize on teams, I think the "all over the bug tracker triaging" has gone down.

The Chrome network team has an explicit, documented triage rotation - https://chromium.googlesource.com/chromium/src/+/master/net/docs/bug-triage.md - that exists to try to make sure we're staying on top of //net bugs. It's not perfect, by any means, but it's a huge step up from the seeming blackhole of bugs that otherwise occur.

Ideally, the only Unconfirmed bugs we leave around are those with Needs-Feedback - and those we do aggressively close if we aren't getting feedback.

2) I'm with Peter in that I see the bug tracker as the notebook. It's a place for discussion, collaboration, tracking known issues and possible improvements. When reviewing code, if someone is adding a TODO, I almost universally request they file a bug as well. In the bug, they can expand upon and do the information dump that most certainly doesn't belong in the file - the discussion of the issues, the tradeoffs, the possible bugs that exist (which, in my experience, tends to be the leading cause of TODOs - "This thing is bugged for X but it shouldn't matter because Y" or "This is weird and needs more investigation but isn't appropriate for this CL"). It also serves as my way of nagging people who are sticking TODOs in to "do it fast" with a promise to "do it right later", to make sure that there is some visibility of "Hey, thing X needs to be revisited and reconsidered"

[Rachel Blum]

I absolutely appreciate you taking action via triaging. I mean action taken in form of actual code :)  I also completely agree with your conclusion about our triage process. 

If we closed all these bugs, we'd reduce the number of open bugs by about 14%

All of Unassigned and Untriaged is 40%. I'm not advocating closing all of them, but it _does_ mean that 40% of our bugs have not seen significant traction. And this isn't even counting the ones that have been triaged and are available, but don't have an owner - another 20% or so. 

I'm saying that at some point we should admit we're not going to get to all of them.

 

You can change your search to "all bugs" and then look just for status=archived

Maybe all we both _really_ want is just a customizable default set of tags :) (Well, OK, I also want "no owner", and I want boolean logic. And a pony)

 and indeed, what needs to be done is that we need to triage the bugs.

Whatever we do with regards to cleaning up or not cleaning up, completely agreed on this. 

 

[Peter Kasting]

On Thu, May 21, 2015 at 7:53 PM, Rachel Blum <gr...@chromium.org> wrote:

I'm saying that at some point we should admit we're not going to get to all of them.

I'm too idealistic to admit that :(

TRIAGE _ALL_ THE BUGS!

Anyway, maybe Ryan's reply is the useful one here: we should perhaps try and ensure that all areas of Chrome have some well-defined triage process.  The network stack team does a great job with this, and it seems like it'd be useful to replicate their model.  I don't know how to make this happen, though.

PK

[Mike Frysinger]

i can't emphasize enough how broken our triage process is.  when i run into bugs, half the time now i don't even bother filing because my expectation is no one is going to look at it.  i don't know the magic label combinations to make sure it gets routed to the right team/dev.  when i do file, the bug often languishes and half the time is resolved as a dupe of a newer bug -- purely because the other person filing picked a better label set.

-mike

--

[Peter Kasting]

On Thu, May 21, 2015 at 8:22 PM, Mike Frysinger <vap...@chromium.org> wrote:

i can't emphasize enough how broken our triage process is.  when i run into bugs, half the time now i don't even bother filing because my expectation is no one is going to look at it.  i don't know the magic label combinations to make sure it gets routed to the right team/dev.  when i do file, the bug often languishes and half the time is resolved as a dupe of a newer bug -- purely because the other person filing picked a better label set.

Hmm.  Our process is broken, but that shouldn't discourage you from filing bugs :(

I think one issue is that even if labels map to groups of recipients, those recipients all assume someone else is looking at the bug.  So whenever I file something, I try to explicitly assign it to someone and ask them to triage to a better owner if they're not right.  Seems to mostly work.

PK 

[Ryan Sleevi]

In the pile-on, one thing I miss (and I understand why it was changed) was the auto-cc vs the label-watching.

With the auto-cc, I could generally slap a label on a bug and see who worked on that, and do exactly what Peter suggested. If there wasn't an auto-cc, either I'd have to dig through the code myself and try to root cause, and THEN punt it to someone to fix (based on git log), or just hope and pray that someone would triage the label because maybe that team had a triage process.

It's one of those tragic things where if we could just get the bug in front of someone who knows the code, it'd be so much quicker to diagnose (triage) and, generally speaking, fix. Because the Chromium community is by and large filled with great people with a passion for building a great product, and part of that is fixing bugs.

But because we don't quite have the tools to do that automagically, and because bisecting requires a reliable repro, and many times that can be a challenge, that doesn't scale.

My hope is that everyone who hacks on Chromium is subscribing to labels relevant to their interests - https://code.google.com/p/chromium/issues/subscriptions - and if you see patterns with existing bugs that could warrant more nuanced labels, that you can add them (Cr-Internals-Network is an example of many little areas, not to mention the many areas "relevant to network's interest" at https://chromium.googlesource.com/chromium/src/+/master/net/docs/bug-triage-labels.md ). Even better would be for the subteams that have formed to actually add a regular rotation calendar to triage bugs. Chrome networking does it. Chrome security does. I imagine there are other teams that do it too.

[Mike Frysinger]

On Fri, May 22, 2015 at 11:25 AM, Peter Kasting <pkas...@chromium.org> wrote:

On Thu, May 21, 2015 at 8:22 PM, Mike Frysinger <vap...@chromium.org> wrote:

i can't emphasize enough how broken our triage process is.  when i run into bugs, half the time now i don't even bother filing because my expectation is no one is going to look at it.  i don't know the magic label combinations to make sure it gets routed to the right team/dev.  when i do file, the bug often languishes and half the time is resolved as a dupe of a newer bug -- purely because the other person filing picked a better label set.

Hmm.  Our process is broken, but that shouldn't discourage you from filing bugs :(

when i file a bug, i try to include reproduction steps (on multiple devices if possible) as well as screenshots.  that can take time.  if no one looks at the bug, then my time is wasted.

now i balance it based on:

- how bad is the bug for my day to day work ... maybe i can adjust my behavior and ride it out

- if it's really painful, maybe i'll just downgrade to beta from unstable/canary/dev/whatever-it's-called and try again in a week or two

- how long have i been hitting this ... maybe someone else will notice/fix it

I think one issue is that even if labels map to groups of recipients, those recipients all assume someone else is looking at the bug.  So whenever I file something, I try to explicitly assign it to someone and ask them to triage to a better owner if they're not right.  Seems to mostly work.

that presupposes you know people working in different areas :).  if we had a PM or someone whose job it was to triage, we could assign it to them by default.  Gentoo has a bug-wranglers@ alias that all bugs default to and is watched by various people who contribute (including users).  their job is 100% triage & routing.  maybe we could at least start off by creating an issue-w...@chromium.org list that anyone could join ?

-mike

[Peter Kasting]

On Thu, May 21, 2015 at 8:33 PM, Ryan Sleevi <rsl...@chromium.org> wrote:

On Thu, May 21, 2015 at 8:25 PM, Peter Kasting <pkas...@chromium.org> wrote:

On Thu, May 21, 2015 at 8:22 PM, Mike Frysinger <vap...@chromium.org> wrote:

i can't emphasize enough how broken our triage process is.  when i run into bugs, half the time now i don't even bother filing because my expectation is no one is going to look at it.  i don't know the magic label combinations to make sure it gets routed to the right team/dev.  when i do file, the bug often languishes and half the time is resolved as a dupe of a newer bug -- purely because the other person filing picked a better label set.

Hmm.  Our process is broken, but that shouldn't discourage you from filing bugs :(

I think one issue is that even if labels map to groups of recipients, those recipients all assume someone else is looking at the bug.  So whenever I file something, I try to explicitly assign it to someone and ask them to triage to a better owner if they're not right.  Seems to mostly work.

In the pile-on, one thing I miss (and I understand why it was changed) was the auto-cc vs the label-watching.

I agree with you so hard.  Plus gmail seemed to notice those mails and mark them as important more consistently.

PK

[Peter Kasting]

On Thu, May 21, 2015 at 8:34 PM, Mike Frysinger <vap...@chromium.org> wrote:

maybe we could at least start off by creating an issue-w...@chromium.org list that anyone could join ?

I'd be willing to join it.  I generally have at least some idea where to start looking if it's a UI bug.

PK

[Mike Frysinger]

i'm def subscribed to a bunch of labels in my area of interest and will generally hop on something that i know needs triaging (i.e. wasn't a tracking bug filed by a team member).  but users don't have the luxury of labels and often bugs that would be interesting to me (including things i broke) don't get routed until much later and largely by accident.

-mike

[Mike Frysinger]

more ideas along these lines, although i suspect we won't be able to do it with code.g.c going away ...

if the bug was assigned to issue-w...@chromium.org & in the Untriaged state, then anyone would be able to add/remove labels as well as assignment/cc fields.  that'd significantly lower the friction for users and would-be contributors.

-mike

[Paweł Hajdan, Jr.]

Are there plans to fix the bug triage process? Closing bugs is easy, but looks like the rate at which bugs are filed is probably increasing. Ignoring and the periodically closing old bugs may be dangerously close to what jwz described in http://www.jwz.org/doc/cadt.html :

This is, I think, the most common way for my bug reports to open source software projects to ever become closed. I report bugs; they go unread for a year, sometimes two; and then (surprise!) that module is rewritten from scratch -- and the new maintainer can't be bothered to check whether his new version has actually solved any of the known problems that existed in the previous version.

Paweł

[Elliott Sprehn]

There was no reply here about what's happening Friday, can we hold off on this?

I'm also not a fan of just closing bugs that are old. We should post a message to them and see if the reporter replies first.

[Daniel Bratell]

On Fri, 22 May 2015 05:25:27 +0200, Peter Kasting <pkas...@chromium.org> wrote:

I think one issue is that even if labels map to groups of recipients, those recipients all assume someone else is looking at the bug.  So whenever I file something, I try to explicitly assign it to someone and ask them to triage to a better owner if they're not right.  Seems to mostly work.

I have had some success with that as well, but only when I actually know who to ask to take a look. There are a number of high profile developers in the project that I know would be able to help, but I don't want to distract them with every issue. There are many others that silently (i.e. without raising their voice on mailing lists very often) just do lots of focused development work and that could help, but it's not trivial to know who they are.

If bugs will be archived, I really think there should be some kind of quarantine phase so that reporters or watchers can renew their case. Just closing them would be a bad blow to the people that have spent time on reporting issues to improve the products. "Since we've ignored you the last few years we are now going to ignore you even more".

Overall I'm on Peter's side in this and think that there being many open bugs isn't a problem in itself. That some of them are not triaged or addressed would be the problem. (Though there was the marketing disaster in Windows 2000 where "open reports" was equated with "serious defects" in some press: http://www.zdnet.com/article/bugfest-win2000-has-63000-defects/ )

/Daniel

[Jonathan Garbee]

I think it would be best to at least ping with a message asking for current verification that the issue exists. I agree with not auto-closing since a good number of old issues are actual issues that have fell through the triage cracks. While there are quite a few old ones that are no longer relevant (I've closed a few myself) others do provide useful information on issues and they shouldn't be archived and left unseen simply due to age. A good number of developers already believe no one in the Chromium project cares about bugs, mass-archiving old ones only adds more fuel to their belief (no matter how wrong it is.)

As far as improving the triage process, this is something I've been thinking about recently. I'll start a new thread on this topic since it is meaty enough to deserve it.

[Thiago Farina]

I think there is two kinds of bugs being filed in the issue tracker, so that means the issue tracker is been used by two ways. One is bugs filed by end users, reporting bugs or requesting features, and this where the most unconfirmed issues lie in I bet. The other is devs filing issues for things that should be done by their team mates (what I would call tasks), and in this camp most of the issues won't go unconfirmed, but instead will go available or assigned and probably with some relevant parties copied in.

The problem seems to be that users don't know the labels correspondent to the features we use and can't copy anyone from the larger Chromium team. And there does not seem to be a triage process that takes care of these issues that the users are entering.

At the time I'm writing this there are 77075 bugs opened.

--

Thiago Farina

[Scott Hess]

On Fri, May 22, 2015 at 4:47 AM, Daniel Bratell <bra...@opera.com> wrote:
> On Fri, 22 May 2015 05:25:27 +0200, Peter Kasting <pkas...@chromium.org> wrote:
> > I think one issue is that even if labels map to groups of recipients, those
> > recipients all assume someone else is looking at the bug. So whenever I
> > file something, I try to explicitly assign it to someone and ask them to
> > triage to a better owner if they're not right. Seems to mostly work.
>
> I have had some success with that as well, but only when I actually know who
> to ask to take a look. There are a number of high profile developers in the
> project that I know would be able to help, but I don't want to distract them
> with every issue. There are many others that silently (i.e. without raising
> their voice on mailing lists very often) just do lots of focused development
> work and that could help, but it's not trivial to know who they are.

In the long ago and faraway, the expectation was that Chromium
developers would look at Chromium bugs, and sometimes you just had to
figure it out. The problem with "high profile developers in the
project that I know would be able to help" is that by definition they
are overloaded with all this helping stuff. Unfortunately, Chromium
over time has become complicated enough that it really is challenging
for someone to perhaps spend hours quantifying repro cases and the
like, and maybe digging in to figure out where things are going wrong.
Even so, if there's someone out there who could figure out the same
problem in 1/4 or 1/10 of the time, there may not actually be anyone
so qualified who actually has free time to figure out the problem!

In the end, I think part of the solution is that Chromium developers
need to make sure that they are spending a certain amount of time
paying forward on maintenance by fixing bugs, even if they are not the
best qualified developer for that particular bug. Waiting for someone
else to develop a system to triage bugs is bootless, there isn't
someone else. If it takes you two days or a week to fix a bug that
someone else could have fixed in a couple hours, that's fine - because
the bug got fixed, and you probably learned something which will let
you fix other bugs, or you crossed paths with developers who you can
ping with future questions, etc.

Yes, I'm on a high horse, here. Yes, I consider myself equally at
fault for these things. But in the end, if a bug promises me a wasted
day trying to repro an ambiguous report with no results, but another
bug has tons of information about repro and a bisect and a stack
backtrace, I'm going to spend the time on the one which gives me the
greatest head start.

-scott

[Matthew Menke]

I disagree.  Assigning random bugs to someone seems like saying "Here you, interrupt what you're doing and spend a couple hours investigating this bug, or face the death of 1,000 5-year old bugs!"  Just went through and purged a huge number of one-off bugs that were assigned to me, many of them had followed this pattern.  Get the right people CCed, sure, but assigning bugs to random people is just really annoying.

[Peter Kasting]

Who said anything about random?  I assign bugs to people who are my best guess at the right person for the bug, and if I'm unsure, I say clearly that I'm not sure and could the person please reassign to a better owner if they're not right.

I absolutely don't assign 1000 randomly-chosen 5-year-old bugs to people to Just Go Triage.

PK 

[Matt Menke]

The bugs I've been assigned this way tend to be very low quality reports, unrelated to code I've touched in the last 6 months.

The crashers I've been assigned this way and invested have disproportionately been malware / badware crashes.  The "jank" bugs have generally been on the top of calls 11+ layers deep, going through multiple layers of ownership.

I've just found bugs assigned this way to be of very low quality, and generally not particularly actionable, or only actionable with a lot of time spent.

And I say this as someone who was so frustrated by bugs going completely ignored, that I independently started watching and triaging all Internals-Network bugs for over a year (Including investigating those with unclear ownership), before we got our new triage system in place.  I think we have a real problem here, but I don't think just assigning bugs to random (Fine, pseudo-random) people really helps much here.

[Ryan Sleevi]

On May 22, 2015 6:28 PM, "Peter Kasting" <pkas...@chromium.org> wrote:
>
>
> Who said anything about random?  I assign bugs to people who are my best guess at the right person for the bug, and if I'm unsure, I say clearly that I'm not sure and could the person please reassign to a better owner if they're not right.
>
> I absolutely don't assign 1000 randomly-chosen 5-year-old bugs to people to Just Go Triage.
>

Since I'm on a roll for me-too's on this thread, I do want to agree with Matt that just assigning bugs, even for investigation, tends to be very disruptive for my workflow. Even if not intended to be received this way, it feels like "This is your fault, go fix it and, if I was wrong, it's now your problem to prove it isn't you"

I absolutely hate being assigned a bug and being told "If it isn't you, find someone else" - because either way, it is equivalent to the assigner saying "You need to fix this" rather than a collaborative "Can you help me find an owner for this".

I realize there's lots of ways this can backfire, but just a $.02 when triaging. I tend to prefer to leave assignment for either someone taking the bug, or outside agreement on who should tackle it (e.g. triage meetings, email discussions, planning, etc) so that it is clear who, if there even is anyone, is actively working on it.

[Peter Kasting]

On Fri, May 22, 2015 at 6:36 PM, Matt Menke <mme...@google.com> wrote:

The bugs I've been assigned this way tend to be very low quality reports, unrelated to code I've touched in the last 6 months.

Certain QA people, unfortunately, tend to assign bugs to people seemingly at random with no real understanding of what's going on.

I don't think it's fair to paint all attempts to find a good owner for bugs with the same brush.  The fact that those reports are often terrible, and terribly assigned, does not mean that competent Chromium developers assign bugs poorly in general, or should avoid assigning such bugs.

I've just found bugs assigned this way to be of very low quality, and generally not particularly actionable, or only actionable with a lot of time spent.

That's orthogonal to whether you're the right owner.

Bugs that are bad and can't be worked on should just be closed, period.  Bugs that you're the right owner for, but which take a long time, can be left in your queue until later.  Bugs you're _not_ the right owner for can be reassigned.

If you're assigned a bug, it just means the onus is on you to take the next action.  That may be to close, reassign, or ignore the bug, whatever is appropriate.

PK

[Matt Menke]

Indeed...but if people don't have time to do anything about bugs, over 5 years, large numbers accumulate, assigned but basically blackholed.

 

PK 

[Anthony LaForge]

(This sounds like a horrible ad) One simple approach to finding a semi-non-random individual, is to look at the people in the last month who fixed bugs for a certain categorization label (i.e. Cr-*).

As an example, if I didn't know who to assign to for a Blink Audio bug (Cr-Blink-Audio), I'd query "label:Cr-Blink-Audio status=fixed,verified closed-after:today-30"... which would tell me three potential owners (even easier to look at in grid view).

Maybe we need to make a simple extension, to do this work and supplement the information in the tracker, for folks like TE (QA) and the rest of the team to help figure out who to reach out to for triage sake.

Kind Regards,

Anthony Laforge
Technical Program Manager
Mountain View, CA

--

--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev

To unsubscribe from this group and stop receiving emails from it, send an email to chromium-dev...@chromium.org.

[Peter Kasting]

On Fri, May 22, 2015 at 6:41 PM, Matt Menke <mme...@google.com> wrote:

if people don't have time to do anything about bugs, over 5 years, large numbers accumulate, assigned but basically blackholed.

I don't see what the problem with that is, other than, if you're a Chromium developer, you should be devoting some (reasonable) fraction of your work over time to taking action on your assigned bugs, so that not ALL of your bugs stagnate forever.  All of your bugs may stagnate somewhat, and some may stagnate forever, but not all of them should stagnate forever.

For bugs I know I will absolutely never get to, I unassign them.  It's OK for you to unassign a bug when you're not going to work on it and no one else should take your place in the queue for it.  (This is different than "I am the right owner, and this should get fixed someday, but I don't have time now", for which I generally lower the priority and leave the bug assigned to myself.)

PK

[Peter Kasting]

On Fri, May 22, 2015 at 6:38 PM, Ryan Sleevi <rsl...@chromium.org> wrote:

Even if not intended to be received this way, it feels like "This is your fault, go fix it and, if I was wrong, it's now your problem to prove it isn't you"

Being assigned a bug means the onus is on you to figure out what to do with it, yes.  That doesn't mean drop what you were doing and respond instantly.

I absolutely hate being assigned a bug and being told "If it isn't you, find someone else" - because either way, it is equivalent to the assigner saying "You need to fix this" rather than a collaborative "Can you help me find an owner for this".

I think that's all in how you choose to read the message.

But let's say you choose to read it as "you need to do something".  That's not wrong, or even rude.  I get lots of bugs where the "something" is "I don't know anything about this and I don't know who the right owner is, sorry" and I send it straight back to the person who assigned it to me.  The "something" you do can be just a few seconds of time.  And, again, it can happen later, not right in the middle of whatever you were doing.

I realize there's lots of ways this can backfire, but just a $.02 when triaging. I tend to prefer to leave assignment for either someone taking the bug, or outside agreement on who should tackle it (e.g. triage meetings, email discussions, planning, etc) so that it is clear who, if there even is anyone, is actively working on it.

The problem is that bugs with no assignee frequently live in a no-man's land where no one believes they have responsibility for the bug, so no one does anything with it.  A bug which is assigned makes it clear who the onus is on to someday do something, even if that something is just reassign the bug.

PK

[Ryan Sleevi]

On May 22, 2015 6:45 PM, "Peter Kasting" <pkas...@chromium.org> wrote:
>
>
>
> I don't see what the problem with that is, other than, if you're a Chromium developer, you should be devoting some (reasonable) fraction of your work over time to taking action on your assigned bugs, so that not ALL of your bugs stagnate forever.  All of your bugs may stagnate somewhat, and some may stagnate forever, but not all of them should stagnate forever.
>

While I agree with this, I do want to point out that combined with a policy of assigning bugs as a "please look into this", it creates a perverse sort of system where the people who are visible get more bugs assigned to them, until they start ignoring bugs.

I think the fallout of this approach is much like when an HR recruiters get a sense of person X being a good interviewer, they will keep scheduling more and more interviews until a person gets so overwhelmed they stop interviewing.

CC's help here, and in general, accomplish the same result (getting visibility to people who can help), without some of the downsides (such as the negative reactions you've heard).

> For bugs I know I will absolutely never get to, I unassign them.  It's OK for you to unassign a bug when you're not going to work on it and no one else should take your place in the queue for it.  (This is different than "I am the right owner, and this should get fixed someday, but I don't have time now", for which I generally lower the priority and leave the bug assigned to myself.)

In the broader context of this thread's discussion, I hope you can understand the discomfort and annoyance that some people may feel when you force them to unassign themselves, especially if you say "Find someone else to assign to if you disagree". It forces a degree of conflict internal to the person (feeling guilty for seemingly saying they don't care about the bug and aren't going to work) and external (by forcing them to implicitly say "No, I'm not going to help you find a better person")

I feel this is probably getting into the weeds of the core point of triaging being needed, but hopefully can provide light as to how some triage practices may be received. Just like the illuminating discussion on the approaches to how the bug tracker itself is viewed.

[Peter Kasting]

On Fri, May 22, 2015 at 6:55 PM, Ryan Sleevi <rsl...@chromium.org> wrote:

In the broader context of this thread's discussion, I hope you can understand the discomfort and annoyance that some people may feel when you force them to unassign themselves, especially if you say "Find someone else to assign to if you disagree". It forces a degree of conflict internal to the person (feeling guilty for seemingly saying they don't care about the bug and aren't going to work) and external (by forcing them to implicitly say "No, I'm not going to help you find a better person")

I can't know whether you consider a bug important and worth your time, and the way to respect your autonomy is to let you make that decision.  Assigning the bug to you means "hey! I think you'd be better at deciding what should be done here than I would!"  It means I respect you and I want you to make the call.

Feeling guilty about this requires a perverse reversal of this, where you assume that, if I assign you a bug, I think I know _better_ than you do about it.  Then, presumably, if you disagreed with my assignment, I might be annoyed.

I think the solution here is to view everyone through the eyes of respect and self-confidence: believe they're trying to do the right thing, and believe they also trust you to do the right thing.  At that point, when you make a call to unassign a bug or send it back to the owner, you're giving them helpful data, not putting your fists up to fight them.

So yes, I can understand why someone would feel discomfort, but I also think that that discomfort might be a worrying indicator of how they view the other members of the team and their relationships with them.  Even the word "force" in your description implies a presumption of combativeness.  If we assume our teammates don't trust us, or think they're better than us, or want to fight us, things are going to go downhill in a lot more ways than just bug triage.

PK

[Mike Frysinger]

users don't get access to labels or assignment or cc or really anything else.  just visit http://crbug.com/new from your personal gmail account.  all you get is a general drop down list of templates and each one includes a diff set of labels.

i'm not suggesting users should be responsible for setting the labels ... *i* have no idea what the majority of labels are for or how to even label my own reports.  i just randomly guess at topics and pick a few.

-mike

[PhistucK]

Note that there are (at least) three ways to file a new issue, that provide various levels of granularity.

1. crbug.com/new which is mainly for more technical people (those that join the various discussion groups).

2. crbug.com and there, clicking on "New issue" which brings up the issue creation wizard (less technical people who got there using a search engine and the like). This actually gives a bit more labelling options (not enough, in my opinion, but much more than 1. gives).

3. Chrome menu > Help > Report an issue, which is very user friendly, but includes nothing informative in terms of label. :(4. chrome:crashes > File bug, which is for more technical users, or following Chrome developer instructions, but also pretty not informative in terms of label other than "Crash".)

If all of those (well, specifically 1, 3 and 4) ways had a better way to let users label the issues, that would help triaging much (I believe). In case the user does not know, it will stay intact (no label, right where we are now).

☆PhistucK

To unsubscribe from this group and stop receiving emails from it, send an email to chromium-dev...@chromium.org.

[Rachel Blum]

On Fri, May 22, 2015 at 6:43 PM, Anthony LaForge <laf...@google.com> wrote:

As an example, if I didn't know who to assign to for a Blink Audio bug (Cr-Blink-Audio),

If you have a single correct label, the battle is almost won. The bigger issue is getting the correct label. (Quick, if my audio hiccups during a hangout on ChromeOS, is it Blink-Audio, Blink-WebRTC-Audio, Internal-Audio, or OS-Kernel-Audio?)

If you look just for untriaged without labels, you have a giant heap of bugs. Some are for components that aren't components. Some (a lot) are autofiled. Some are using a Hotlist-XYZ label instead of components. Some have been assigned, but not marked assigned, and it was obvious for everybody what it was about, so it was never labeled.

The issue with bugs (from my POV, at least) is that there's a giant intake pipe, then there's a box that says "magic happens here", and then there are developers working on bugs. The net triage guidelines are awesome - it'd help if all new bugs saw a somewhat well-defined triage, I think. At least to the point that there's one or more clear labels on the bug, and at least one knowledgable person has been CC'ed.

Maybe, as a very first step, a clear path from user-filed to at least Untriaged/labeled. (We have Via-Wizard/Unconfirmed bugs dating back to 2012).

[Matthew Yuan]

Hi folks,

After discussing things further with the other TPMs we've come up with the following on how we deal with older unconfirmed issues without compromising issues which might still be effecting the folks who reported them - per thread feedback.  The following flow is what we intend to implement in the coming weeks. 

We've also noticed the concerns around bug triage as whole and have begun discussions (ongoing) around how to tackle that in a way that better suites our environment.

Thank you for the previous feedback.

Best,

MKY

-----

Auto Archive Flow:

1. Identify issues which are unconfirmed and over a year old and archive those which have 0-stars since no one is subscribed to them.

2. Add the following note and label to the issue which fit the haven’t been modified for over a year and are unconfirmed criterion:

1. Note: “Due to inactivity, this issue will be closed after two weeks.  If you are still experiencing this issue then please add a comment to the issue with the following information: 1) Chrome version, 2) operating system and 3) steps to reproduce (if they’ve changed) to prevent the closure of this issue.”

2. Label: Auto-Archive-Candidate-<Today + 2 Weeks Date>

3. Check back if the date is beyond the the two weeks date and the issue has not been modified archive the issue along with the following label for reference Auto-Archived.  Otherwise, update the Auto-Archive-Candidate-<Today + 2 Weeks Date>.

If we were to repeat this process continuously we would be able to vet out issues which are probably stale while bubbling up issues which are still relevant.

--

MKY

Matthew Yuan |

Technical Program Manager |

matth...@google.com |

1-650-214-1277

[Paweł Hajdan, Jr.]

Are there plans to have a process in place to help us respond to more issues, and either get them confirmed or closed as not reproducible?

I'm glad some feedback is being taken into account. I just consider the above also a very important part.

FWIW I'm totally fine with auto-closing bugs with Needs-Feedback and no activity for a long time. It usually means the original reporter failed to provide some important info. However, other unconfirmed issues without activity usually indicate no one really looked at the bug. I'd still be concerned about putting burden on our users to keep these bugs open.

Paweł

[Ricardo Vargas]

On Mon, Jun 15, 2015 at 11:48 AM, 'Matthew Yuan' via Chromium-dev <chromi...@chromium.org> wrote:

Hi folks,

After discussing things further with the other TPMs we've come up with the following on how we deal with older unconfirmed issues without compromising issues which might still be effecting the folks who reported them - per thread feedback.  The following flow is what we intend to implement in the coming weeks. 

We've also noticed the concerns around bug triage as whole and have begun discussions (ongoing) around how to tackle that in a way that better suites our environment.

Thank you for the previous feedback.

Best,

MKY

-----

Auto Archive Flow:

1. Identify issues which are unconfirmed and over a year old and archive those which have 0-stars since no one is subscribed to them.

2. Add the following note and label to the issue which fit the haven’t been modified for over a year and are unconfirmed criterion:

1. Note: “Due to inactivity, this issue will be closed after two weeks.  If you are still experiencing this issue then please add a comment to the issue with the following information: 1) Chrome version, 2) operating system and 3) steps to reproduce (if they’ve changed) to prevent the closure of this issue.”

2. Label: Auto-Archive-Candidate-<Today + 2 Weeks Date>

3. Check back if the date is beyond the the two weeks date and the issue has not been modified archive the issue along with the following label for reference Auto-Archived.  Otherwise, update the Auto-Archive-Candidate-<Today + 2 Weeks Date>.

I'm reading this point like "old bugs will have to be touched every two weeks or we'll archive them". Is that correct?

 

If we were to repeat this process continuously we would be able to vet out issues which are probably stale while bubbling up issues which are still relevant.

On Wed, May 27, 2015 at 4:08 PM, Rachel Blum <gr...@chromium.org> wrote:

On Fri, May 22, 2015 at 6:43 PM, Anthony LaForge <laf...@google.com> wrote:

As an example, if I didn't know who to assign to for a Blink Audio bug (Cr-Blink-Audio),

If you have a single correct label, the battle is almost won. The bigger issue is getting the correct label. (Quick, if my audio hiccups during a hangout on ChromeOS, is it Blink-Audio, Blink-WebRTC-Audio, Internal-Audio, or OS-Kernel-Audio?)

If you look just for untriaged without labels, you have a giant heap of bugs. Some are for components that aren't components. Some (a lot) are autofiled. Some are using a Hotlist-XYZ label instead of components. Some have been assigned, but not marked assigned, and it was obvious for everybody what it was about, so it was never labeled.

The issue with bugs (from my POV, at least) is that there's a giant intake pipe, then there's a box that says "magic happens here", and then there are developers working on bugs. The net triage guidelines are awesome - it'd help if all new bugs saw a somewhat well-defined triage, I think. At least to the point that there's one or more clear labels on the bug, and at least one knowledgable person has been CC'ed.

Maybe, as a very first step, a clear path from user-filed to at least Untriaged/labeled. (We have Via-Wizard/Unconfirmed bugs dating back to 2012).

--

MKY

Matthew Yuan |

Technical Program Manager |

matth...@google.com |

1-650-214-1277

[Matthew Yuan]

Older unconfirmed bugs which have not been touched for over a year will become candidate for closure via comment, if no further comments are made after two weeks the issue will be closed.  Otherwise, they are left alone.

[Matt Mueller]

It seems like there is an important step missing, which is making sure that if the user does respond, that someone actually looks at the bug. If the bug hasn't been triaged in a year it probably doesn't have any labels or other way for any relevant developers to know it exists, and no one will ever see the comment the user adds. So it seems this modification of the process will just result in fully automated badgering of such users to update their bug every X weeks until the user gives up in frustration and lets the bug be auto-closed anyway.

[Matthew Yuan]

Good point, I'll work something out with the test folks to ensure that we follow up to hopefully repro and get some initial triage.

[Thiemo Nagel]

The way I read this thread, several people have raised valid concerns against auto-closing bugs (loss of useful information, bad user experience) whereas the benefit of auto-closing seems intangible (reduce number of open bugs). With that backdrop in mind I wonder why we are proceeding with the effort? Have I misread the sentiment of the thread?

Again, from my reading of the thread, we have a grave bug triage problem and (possibly) a minor too-many-open-bugs problem. In that situation to focus on auto-closing bugs feels like doing the second step before the first. Would it not be more urgent to solve the bug triage problem? And once we (as an organisation) have learned how to bug triage effectively, we can apply this capability to the backlog as well. And if then it turns out that the overwhelming majority of old bugs are of little value, we might decide to auto-close them based on that data. (Right now, we have no data that would justify closing old bugs. At least none that was mentioned on this thread.)

Older unconfirmed bugs which have not been touched for over a year will become candidate for closure via comment, if no further comments are made after two weeks the issue will be closed.  Otherwise, they are left alone.

If we proceed without fixing our triage process, the outcome can only be negative:

• The user doesn't reply
  We close the bug. We record in writing that we don't care about the issue, for anyone to see, including the reporter who might have missed the notification email (or been on vacation or busy or whatever).
  
• The user replies
  This case is even worse: We act as if we would care about the issue, the reporter sinks even more time into the issue, they draw renewed hope that their issue would be addressed only to find out over time that their issue still is being ignored.
  

Cheers,

Thiemo

[Paweł Hajdan, Jr.]

+1

Paweł

[pdknsk]

I notice mass-archiving is taking place now, with either of both messages.

> Slated to an old branch and no activity in the bug in over a year suggest that this can be archived.

> Archiving unconfirmed issues, which have not been modified (commented on, updated, etc...) in over 2 years.

The first message at least is not accurate and misleading. Usually a lack of activity is caused by Google, not the user. It also suggests users should ping the bug every new milestone to keep it active.

[Chris Nardi]

The first message should *never* be an archive reason. Just because a milestone is put on, doesn't mean that the bug magically goes away with the next few milestones.

I notice that 1,000 bugs were archived with that reason. A lot of those are probably still problems. I think those should be un-archived, given that some of them are even non-regressions and untriaged bugs (not unconfirmed bugs, untriaged).

[Paweł Hajdan, Jr.]

Do you have examples of incorrectly archived bugs?

I share the concerns about closing bugs where inactivity is caused by lack of reaction from the Chromium project.

Paweł

--

[pdknsk]

This is an example of an incorrectly archived bug.

https://code.google.com/p/chromium/issues/detail?id=365484

[Chris Nardi]

Two that I un-archived manually were https://crbug.com/385466 and https://crbug.com/389678.