Moving patches from posted-to-bug to in-the-tree: a process discussion
Jeff Walden
https://bugzilla.mozilla.org/show_bug.cgi?id=485217
A bug was filed, a patch was posted, but it didn't get in the tree until "too late" (for a not-worst-case-scenario definition of the phrase, but still way too close to it for comfort). Some questions to ponder:
When did we fail to keep things moving along?
How do we make the process clearer and more discoverable for new hands?
What documentation needs to be modified to help that clarification?
How and where should such documentation be found, and where should we position links to it on our sites to make it easy to find?
How should responsibility for making sure posted patches get in the tree be split up?
Are there any changes we should make to the super-review process (or the review process) to ensure patches don't fall through the cracks?
Where do we document any such changes? (Do we even *have* any documentation on doing non-super-review reviews and what the goal of a review is?)
Those are just for starters, and I'm sure there are others people can bring up as discussion happens.
All comments are appreciated.
Jeff
Robert O'Callahan
> When did we fail to keep things moving along?
> How do we make the process clearer and more discoverable for new hands?
> What documentation needs to be modified to help that clarification?
> How and where should such documentation be found, and where should we
> position links to it on our sites to make it easy to find?
> How should responsibility for making sure posted patches get in the tree
> be split up?
> Are there any changes we should make to the super-review process (or the
> review process) to ensure patches don't fall through the cracks?
> Where do we document any such changes? (Do we even *have* any
> documentation on doing non-super-review reviews and what the goal of a
> review is?)
I don't think documentation or processes are an issue here. Judging from
the bug, this was a very simple case of slow responses.
It's really important to stay on top of bugmail and handle review
requests as quickly as possible. This bug isn't even the worst case
(apart from the unfortunate publication of an exploit); the worst case
is that the contributor just gets fed up and disappears. Maybe for MoCo
employees we can monitor this and publish the stats? :-) Beyond my
Bugzilla skills though.
Rob
gzl...@googlemail.com
> --- Comment #23 from Jeff Walden <jwalden+...@mit.edu> 2009-03-26 22:47:37 PDT ---
> (In reply to comment #22)
>
> Passive-aggressive insinuations of bad faith generally don't win friends and
> influence people.
I am annoyed, but really at myself rather than any other individual.
As I have proved capable of not getting getting fixes committed in
multiple projects, I'd be interested in anything you could point out
that I should have done differently, so I can actually do better in
future, in Mozilla and elsewhere.
On Mar 27, 5:46 am, Jeff Walden <jwalden+...@mit.edu> wrote:
>
> Those are just for starters, and I'm sure there are others people can bring up as discussion happens.
>
> All comments are appreciated.
I don't think I have anything that hasn't been said already. Could
complain about bugzilla, or large software projects, or attitudes to
security bugs, but that's all clearly part of the territory if you
want to help out on mozilla.
Fundamentally, when the core contributors are such a bottleneck that
it's worth them spending the 5 minutes to post a bugzilla review that
consists mostly of nits about whitespace or whatever - rather than the
15 to pull it into their tree and just fix the things and push - then
random patchers are just going to have to suck up the slow back-and-
forth.
And as a counterpoint to where things go wrong, the reason I was
poking around looking for XSL bugs to fix in the first place was
because the first speculative patch I posted got a review in less than
a day - after I'd been waiting months for a patch review on a non-
mozilla project.
Martin
Mike Kristoffersen
Being relatively green in the Mozilla processes I don't know the details
of how it is handled today - but it seems to depend a lot on humans
doing things that could be happening automatically. Another problem is
documentation of the process - I have been told part of the process, but
never seen the document that describes it - now this might be because I
didn't search for it - I'll get back to that in a sec.
So the issues I see are:
1) Lack of automation
2) You need to pull to know the process - (especially bad when the
process changes)
----
What I would like to see is that this gets integrated into bugzilla, let
me explain:
- I'll assume that we have a well defined process, stating which states
a bug can enter from a given other state, and the reasons/triggers for
each state change. (I think it is out of scope to discuss the specific
states here, as the point being that there are states)
- Now, any active bug must at all times have someone or a group that is
responsible for it (by active I mean, a bug that is not fixed and in the
tree or one that has been closed for another reason - like "working as
intended")
- This means that if you are not responsible for a bug then you are not
expected to do anything with it. So when you call for a review on a
bug, you are no longer responsible for the bug, unless it comes back to
you with a failed review - this could be automated, if you set the
review flag "state", then if the reviewer fails the review (setting the
"failed review" state) it gets automatically assigned back to the one
who requested the review.
- When you get assigned to a bug (for validation, testing, reviewing,
fixing, committing, what ever reason) you should receive an e-mail.
(Where you at a glance, from the title of the e-mail, can see that it is
about something you (or a group you are a member of) are getting
assigned to)
- When a bug that you are assigned to is opened, you could get a
description that explains what you are supposed to do, this text being
defined by the state the bug is in, "Verify the bug", "Review attached
patch", etc.
You then only get the options for changing states that is defined by the
process - with a description of why you would want to move it to that
state (e.g. for a bug you are assigned to fixing - you could have:
1) A fix for the bug has been attached => state=waiting for review,
assignedTo=reviewer
2) You can't reproduce the bug => state=more info wanted,
assignedTo=reporter
3) You don't have time to work on it now => state=waiting for time,
assignedTo=you
4) You have accepted there is an issue and are working on fixing it =>
state=accepted working on it, assignedTo=you
etc. note that the above choice are only to illustrate the principle,
the actual states are again out of scope for this discussion.
----
The above should give visibility to the process - there is only one rule
you must know - if you get a bug assigned, it is your responsibility to
move it to the next state - you are told everything else you need to
know when you open the bug in bugzilla.
If the bug at any state gets assigned to a group instead of an
individual, everyone in that group can then change the assignment to an
individual person in the group (so we don't get multiple persons working
on the same issue unintentionally).
----
So how do we ensure that bugs don't gets forgotten by individuals / stay
in the system for ever (or past an important deadline)?
-Assuming there are someone responsible for any given product and/or
area then these people should be able to setup triggers with
time-limits, so if a high impact bug stays in the review state for more
than a specified time, then the one responsible for the product gets
notified, and can choose to take appropriate action.
-In my view, it should never be e.g. a developers responsibility to
prioritize for or push a reviewer - it must be the project that does
this kind of prioritization.
-It is the same if you report an issue, you are only expected to give a
useful description of the issue. In my view, the reporter should not
push the fixing, by any other means than perhaps setting the initial
severity of the bug - it is the responsibility of the project, or owner
of the area to do the prioritization with respect to the other tasks at
hand.
----
I have written a little about automation in the above, e.g. the triggers
the project owner can set if relevant bugs gets stalled in the system,
and the e-mails that are send with tags in the title if you get assigned
something new (at least we have the e-mails today)
But the automation could also continue into choosing who to assign bugs
to at any given state - so if I create a new bug, it could automatically
get assigned to the group (or individual) responsible for the area where
the bug is reported in (with the e-mail notification)
When a bug is moved to the review state, the reviewer could be chosen
from the files that are patched, etc.
----
I know the title of this thread talks about the post-fix process, but I
think we could deal with the whole process here :)
^__^
Mike
--
///////////////////////////////////////////////////////////
// Documentation is written for humans, not for machines //
///////////////////////////////////////////////////////////
Mike Beltzner
> On 27-03-2009 06:46, Jeff Walden wrote:
>> https://bugzilla.mozilla.org/show_bug.cgi?id=460090
>> https://bugzilla.mozilla.org/show_bug.cgi?id=485217
>>
>> A bug was filed, a patch was posted, but it didn't get in the tree
>> until
>> "too late" (for a not-worst-case-scenario definition of the phrase,
>> but
>> still way too close to it for comfort). Some questions to ponder:
>>
>> When did we fail to keep things moving along?
>> How do we make the process clearer and more discoverable for new
>> hands?
>> What documentation needs to be modified to help that clarification?
>> How and where should such documentation be found, and where should we
>> position links to it on our sites to make it easy to find?
>> How should responsibility for making sure posted patches get in the
>> tree
>> be split up?
I'm not sure that we need new or additional process here, to be
honest. It is the responsibility of a bug owner to ensure that it
transitions from state to state to state, and it's the responsibility
of a component owner to ensure that bugs are actively being triaged
and moved through the states and closed out if they're languishing or
not getting attention. My feeling is that the more process we have,
the harder things are to understand, and we have quite enough to
understand already.
To me it feels like what we need are more people interested in this
administration of our process, doing the work of walking through
buglists and making sure. Additionally, tools that make it easier to
see bug aging reports at a glance, and the ability to query for bugs
that have "reviewed, approved patches" would allow us to spot cases
where we have fixes in hand that are simply waiting for a landing
window. Further, our continued efforts to keep the trees as green as
possible as a mechanism for speeding landings will also help here.
More documentation is dandy, especially if annotated with screenshots
or even screencasts.
cheers,
mike
Boris Zbarsky
> Another problem is
> documentation of the process - I have been told part of the process, but
> never seen the document that describes it - now this might be because I
> didn't search for it - I'll get back to that in a sec.
https://developer.mozilla.org/en/Hacking_Mozilla
This probably needs to be adjusted to reflect the new sr policy, though.
> - Now, any active bug must at all times have someone or a group that is
> responsible for it (by active I mean, a bug that is not fixed and in the
> tree or one that has been closed for another reason - like "working as
> intended")
Yes, indeed. The problem with this is that our current ratio of bugs to
people is on the order of hundreds. That's far past what one can
actually keep track of usefully... We could use group "nobody"
placeholders like we do now to keep track of bugs, but those aren't
going to do anything about bugs.
> - When you get assigned to a bug (for validation, testing, reviewing,
> fixing, committing, what ever reason) you should receive an e-mail.
> (Where you at a glance, from the title of the e-mail, can see that it is
> about something you (or a group you are a member of) are getting
> assigned to)
For what it's worth, a review requestee already gets mail, and there is
a Bugzilla page where you can see all your pending review requests.
Some people look at this page frequently and do reviews. Some don't.
Tooling changes won't change this. :(
> - When a bug that you are assigned to is opened, you could get a
> description that explains what you are supposed to do, this text being
> defined by the state the bug is in, "Verify the bug", "Review attached
> patch", etc.
This could be good for novice contributors; it won't help reviewers
shirking their duties: they already know what they're supposed to do
with review requests!
> The above should give visibility to the process - there is only one rule
> you must know - if you get a bug assigned, it is your responsibility to
> move it to the next state - you are told everything else you need to
> know when you open the bug in bugzilla.
This does have some benefits, yes.
> So how do we ensure that bugs don't gets forgotten by individuals / stay
> in the system for ever (or past an important deadline)?
Right.
> -It is the same if you report an issue, you are only expected to give a
> useful description of the issue. In my view, the reporter should not
> push the fixing, by any other means than perhaps setting the initial
> severity of the bug - it is the responsibility of the project, or owner
> of the area to do the prioritization with respect to the other tasks at
> hand.
Though of course the reporter is welcome (but not required) to do more,
and some do.
> But the automation could also continue into choosing who to assign bugs
> to at any given state - so if I create a new bug, it could automatically
> get assigned to the group (or individual) responsible for the area where
> the bug is reported in (with the e-mail notification)
That already happens, to some extent. There are people watching the
component qa addresses, so they get mail when bugs are filed. For
example, I get mail on new form control, docshell, xbl bugs. I'd look
at more components, but there's no way to look for only new bugs; if I
watched all components I care about new bugs in I'd be getting over 2000
bugmails per day (yes, I tried this once). I thought we had a bug on
making it easier to watch only new bugs and bugs being moved into a
component, but I can't find it right now... Maybe I should re-file it
just in case. :(
-Boris
Mike Shaver
some tweaks to whitespace and tests, thanks!" and less "bounced until
you make this round of minor changes, please try again later". Cairo
does the former, and it makes it a lot more welcoming as well as, I
believe, putting a lower burden on reviewers in aggregate. Reloading
a patch's context, plus actually remembering to revisit at all, is
usually a lot more costly than just fixing up little whitespace and
naming nits, or even fixing some minor remaining bugs. (To say
nothing of then having to spend more time in the horrible bugzilla
patch-editing experience!). It may be that some part-time contributors
only have time for driveby reviews and not to actually check in, but I
don't think that holds for anyone on staff at Mozilla or elsewhere.
Mike
On 3/27/09, Mike Beltzner <belt...@mozilla.com> wrote:
> On 27-Mar-09, at 9:27 AM, Mike Kristoffersen wrote:
>
>> On 27-03-2009 06:46, Jeff Walden wrote:
>>> https://bugzilla.mozilla.org/show_bug.cgi?id=460090
>>> https://bugzilla.mozilla.org/show_bug.cgi?id=485217
>>>
>>> A bug was filed, a patch was posted, but it didn't get in the tree
>>> until
>>> "too late" (for a not-worst-case-scenario definition of the phrase,
>>> but
>>> still way too close to it for comfort). Some questions to ponder:
>>>
>>> When did we fail to keep things moving along?
>>> How do we make the process clearer and more discoverable for new
>>> hands?
>>> What documentation needs to be modified to help that clarification?
>>> How and where should such documentation be found, and where should we
>>> position links to it on our sites to make it easy to find?
>>> How should responsibility for making sure posted patches get in the
>>> tree
>>> be split up?
>
> I'm not sure that we need new or additional process here, to be
> honest. It is the responsibility of a bug owner to ensure that it
> transitions from state to state to state, and it's the responsibility
> of a component owner to ensure that bugs are actively being triaged
> and moved through the states and closed out if they're languishing or
> not getting attention. My feeling is that the more process we have,
> the harder things are to understand, and we have quite enough to
> understand already.
>
> To me it feels like what we need are more people interested in this
> administration of our process, doing the work of walking through
> buglists and making sure. Additionally, tools that make it easier to
> see bug aging reports at a glance, and the ability to query for bugs
> that have "reviewed, approved patches" would allow us to spot cases
> where we have fixes in hand that are simply waiting for a landing
> window. Further, our continued efforts to keep the trees as green as
> possible as a mechanism for speeding landings will also help here.
>
> More documentation is dandy, especially if annotated with screenshots
> or even screencasts.
>
> cheers,
> mike
> _______________________________________________
> governance mailing list
> gover...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/governance
>
--
Sent from Gmail for mobile | mobile.google.com
John J. Barton
> https://bugzilla.mozilla.org/show_bug.cgi?id=460090
> https://bugzilla.mozilla.org/show_bug.cgi?id=485217
>
> A bug was filed, a patch was posted, but it didn't get in the tree until
> "too late" (for a not-worst-case-scenario definition of the phrase, but
> still way too close to it for comfort). Some questions to ponder:
>
> When did we fail to keep things moving along?
> How do we make the process clearer and more discoverable for new hands?
The internal notation of the project confuses external contributors. For
example 'in-the-tree' is obviously very important to developers, as the
things like FIXED1.9.1. I watch a few dozen bugs and try to move them
forward but very often I cannot understand what state they are in. We
deal in values like "FF3.1b3" but these don't appear on the bug reports.
I still don't know how developers can tell if a fix is in the end
product or not. This seems like an area where automation would be feasible.
Separately, I notice that often a key time-consuming step near the end
is the introduction of second, verification test case. So a bug report
goes nowhere until a clear test case is reproduced. Then again it gets
stuck if the fix has no verification test. Usually it seems that two
tests are needed and both consume a lot of resources. If the tests were
easier to create, then maybe the process would smoother.
jjb
Phil Ringnalda
> I feel pretty strongly that we should be doing more "committed with
> some tweaks to whitespace and tests, thanks!" and less "bounced until
> you make this round of minor changes, please try again later".
I do some of that, and I've been told, I suspect correctly, that I'm not
improving things in the long run by doing so.
If our goal when someone we've never seen before attaches a patch is to
get that code in the tree, then there's no doubt that having "a Mozilla
developer," the reviewer (in the post-SR world, where that actually
works) or someone else, just de-nit and push is the way to go.
If our goal is to turn the person who attached the patch in the first
place into a Mozilla developer, I'm less sure it is. I know the feeling
that "I had to fight about whitespace, and make some changes that I
don't think made sense, and beg for a checkin, but I got *my code* in
Firefox" produces; what feeling does "I attached a patch that someone
else took, assigned the bug to themselves, changed around, and checked
in" produce? Would we really be better off treating patchers as
interchangeable disposable idea-factories, rather than treating them as
the person we're going to train to be the next peer for the module?
Mook
> I don't think documentation or processes are an issue here. Judging from
> the bug, this was a very simple case of slow responses.
As an external person, getting reviews is _hard_, especially since I
typically poke things that isn't needed by Firefox on a tier 1 platform.
My personal "best" was a single-token change that took 8 months
(request was to somebody who made a drive-by comment on my previous
iteration). I've heard of others who had to get to Mountain View,
invite people out to lunch, and ambush them with a review request once
they got there, just to get a review done.
Unfortunately, I don't see this changing anytime soon. While a few
people make valiant efforts (thanks!), there just isn't a project-wide
push to make sure people get on their review requests. At this point,
it seems like actually doing reviews is more of a voluntary thing that
can be pushed off - and that means that is exactly happens.
--
Mook
Robert O'Callahan
> Unfortunately, I don't see this changing anytime soon. While a few
> people make valiant efforts (thanks!), there just isn't a project-wide
> push to make sure people get on their review requests. At this point, it
> seems like actually doing reviews is more of a voluntary thing that can
> be pushed off - and that means that is exactly happens.
I think the situation is getting a little better: some individuals are
getting better, and in some areas we've got new peers or owners to
spread the load.
But I agree that responsiveness is still often unacceptable.
Rob
Robert O'Callahan
> On 27-Mar-09, at 9:27 AM, Mike Kristoffersen wrote:
>
>> On 27-03-2009 06:46, Jeff Walden wrote:
>>> https://bugzilla.mozilla.org/show_bug.cgi?id=460090
>>> https://bugzilla.mozilla.org/show_bug.cgi?id=485217
>>>
>>> A bug was filed, a patch was posted, but it didn't get in the tree until
>>> "too late" (for a not-worst-case-scenario definition of the phrase, but
>>> still way too close to it for comfort). Some questions to ponder:
>>>
>>> When did we fail to keep things moving along?
>>> How do we make the process clearer and more discoverable for new hands?
>>> What documentation needs to be modified to help that clarification?
>>> How and where should such documentation be found, and where should we
>>> position links to it on our sites to make it easy to find?
>>> How should responsibility for making sure posted patches get in the tree
>>> be split up?
>
> I'm not sure that we need new or additional process here, to be honest.
> It is the responsibility of a bug owner to ensure that it transitions
> from state to state to state, and it's the responsibility of a component
> owner to ensure that bugs are actively being triaged and moved through
> the states and closed out if they're languishing or not getting
> attention. My feeling is that the more process we have, the harder
> things are to understand, and we have quite enough to understand already.
I heartily, fully, enthusiastically agree. We don't need more process.
We just need to be more diligent about following the process we have,
and measuring it.
Rob
fried...@gmail.com
test suite when it was reported?
Would adding it as soon as it was seen as reproducible (even without
fix) caused enough attention to get the fix in sooner?
I'm not sure of the testing procedures in mozilla,
but I maybe getting such test cases in early on could have helped that
the bug got enough attention to get fixed sooner.
Daniel
L. David Baron
> That already happens, to some extent. There are people watching the
> component qa addresses, so they get mail when bugs are filed. For
> example, I get mail on new form control, docshell, xbl bugs. I'd look
> at more components, but there's no way to look for only new bugs; if I
> watched all components I care about new bugs in I'd be getting over 2000
> bugmails per day (yes, I tried this once). I thought we had a bug on
> making it easier to watch only new bugs and bugs being moved into a
> component, but I can't find it right now... Maybe I should re-file it
> just in case. :(
One thing that would probably help a lot is being able to configure
separate bugzilla email preferences for each address watched. I'd
probably watch a bunch of additional addresses if I could watch only
new bugs. (The "added to / removed from" checkbox is probably close
enough for catching incoming bugs to a component; the problem is
that I can't use that alone for all my bugmail.)
-David
--
L. David Baron http://dbaron.org/
Mozilla Corporation http://www.mozilla.com/
L. David Baron
> I'm not sure that we need new or additional process here, to be honest.
> It is the responsibility of a bug owner to ensure that it transitions
> from state to state to state, and it's the responsibility of a component
> owner to ensure that bugs are actively being triaged and moved through
> the states and closed out if they're languishing or not getting
> attention. My feeling is that the more process we have, the harder things
> are to understand, and we have quite enough to understand already.
One thing that Jesse suggested a few weeks ago (in conversation at
lunch, I think) is that Bugzilla have a (queryable) notion of who is
expected to take the next action in a bug (and what that action is).
This way we could see queues of things other than review requests.
Another thought I had is that perhaps we could find tracking this
information easier if Bugzilla's "Status" field reflected the actual
statuses, e.g., by replacing "ASSIGNED" with "fix under
development", "waiting for review", and "waiting for checkin" with
an appropriately-defaulted checkbox to update this status as one
would expect when editing attachments.
(Personally, I track my patches-in-progress by having them all in a
single mercurial queue. If it's not in that queue, I have no plans
to commit it.)
timeless
> One thing that would probably help a lot is being able to configure
> separate bugzilla email preferences for each address watched. I'd
> probably watch a bunch of additional addresses if I could watch only
> new bugs. (The "added to / removed from" checkbox is probably close
> enough for catching incoming bugs to a component; the problem is
> that I can't use that alone for all my bugmail.)
there's a bug (and plans i believe) for bugmail controls to be query
based, however i don't think anyone considered this additional
restriction, so it should probably be filed w/ a dependency....
timeless
that r-'ing repeatedly to get tests just right is an obvious part of
the failure for this specific bug.
it's one thing for someone who's paid to work on absolutely nothing
else to be paid to get tests just right, it's another to take a
crasher, have a fix from a contributor (who probably has hundreds of
other things to do) and make them do work over an extended period of
time before not taking their patch.
we have been leaning toward this latter pole.
we've need people to monitor review requests and help get patches
landed for 10 years now. it only gets worse as the number of bugs we
have grows.
that said, it turns out that this bug's story is an example of where
it's not a good idea to jump at the first suggestion.
fwiw, i'm currently on one of my "bug reporting sprees". but i'm not
filing ff bugs, i'm filing bugs against virtualbox, songbird,
mercurial, ....- I'm trying to work w/ products where the barrier to
filing bugs is lower. In many cases, I could write the fixes too, but
I probably won't,... too many bugs to file, too little time.
Smokey Ardisson
> I'm not sure that we need new or additional process here, to be
> honest. It is the responsibility of a bug owner to ensure that it
[...]
> To me it feels like what we need are more people interested in this
> administration of our process, doing the work of walking through
> buglists and making sure.
I think beltzner pretty well nailed it. If you know our process, and
if people follow our process, the process is sufficient. The
documentation on the process ( https://developer.mozilla.org/en/Getting_your_patch_in_the_tree
) feels adequate; it could probably be improved a little bit, but I
think it nails the high points pretty well, including the
responsibility of the patch author to periodically pester us for
review or checkin and to supply a checkin-ready patch and add checkin-
needed. Maybe we need to raise the visibility of that documentation,
particularly in Bugzilla? Perhaps linking it at the bottom of the
attachments table? There’s a link to it already on the patch
submission page, but since you don’t use that page while the patch is
waiting for r/sr, or “waiting in limbo” after sr+, that process
documentation ends up not being terribly visible.
That said, Mozilla's a large enough project where things can fall
through the cracks, and so we need some "Unloved Patches Ombudsmen" to
help ensure that patches do get moved along and patch authors don't
get discouraged. On a very small scale, we've done this sort of thing
to nearly eliminate the "lost patches" problem in Camino, whereas a
couple of years ago the problem was affecting 50% of the patches, even
from our own dev team. I took on the role of patch shepherd, and I
had two queries to help me: one for bugs with non-obsolete patches
where there had been no bug activity in over a month, and one for bugs
with non-obsolete patches that had r+ and sr+. I'd run the former
query occasionally and poke bugs, but I'd go through the latter query
on a weekly basis and set what eventually became checkin-needed and
then pester our folks with commit rights. Nowadays I do checkins, so
we've pretty much moved the patch bottleneck back into the review
process itself ;-) (q.v. various Mozilla discussions about old review
requests). The queries weren't perfect, so some better patch+flag
query tools in Bugzilla would help, but assuming a sufficient number
of potential ombudspersons, even this semi-manual method would be
feasible for major project areas.
Better tooling for locating unloved patches might help module and
component owners keep track of these themselves, but, if as Boris
says, people don’t use the existing tools to check on their review
queues, tools alone won’t solve this problem. Neither will one-time
events (like Gerv’s valiant “old review requests” quest from a couple
of years ago), which is why I really think we need some people whose
job description specifically includes spending time as "Unloved
Patches Ombudsmen." It would be great if Mozilla Corp could find some
not-already-overworked employees ;-) and include in their job
descriptions spending time in this ombudsman role for Gecko/Toolkit
and Firefox components. (Moz QA seems to do a good job tracking fixed
bugs, verifying them, and making sure things are in the testsuites;
since QA is also working through bugs on triage duty, perhaps they're
good candidates for this role, too?)
Once a week each ombudsman would run the queries in the components/
modules for which he’s responsible and poke the appropriate people
depending on the status of the bug (reviewer/superreviewer, patch
author, checkin-needed pool), and so forth. If there are too many
components and too few people, even cycling through each ombudsman’s
assigned components on a staggered basis so that all components are
checked for "unloved patches" just once a month would be a massive
improvement, I’d think.
Regular ombudsmen sweeps could ultimately help solve both the specific
problem here, where we had a good patch that never made it into the
tree, and the larger problem we have with getting patches through the
entire process, where getting reviews is often the blockage instead.
(I also wonder if it might not improve the upstreaming experience for
Mozilla-based projects; their bugs typically won’t be P1 Firefox
blockers, so having the ombudsmen interact with the bugs periodically
would keep those fixes from being completely forgotten.)
Smokey
Gervase Markham
> Neither will one-time
> events (like Gerv’s valiant “old review requests” quest from a couple
> of years ago),
That wasn't supposed to be a one-time event. The idea was that we would
clean out the backlog by either invalidating or reviewing them, and then
component/module owners could be whined at on a regular basis about new
bugs which seemed to be unloved.
This project has, sadly, like many other things I'd like to do, stalled
through lack of time (although another community member has done some
valuable work on it, it's sort of blocked on me, and a simple
bugzilla.mozilla.org change I can't seem to get any love for).
> Once a week each ombudsman would run the queries in the components/
> modules for which he’s responsible and poke the appropriate people
> depending on the status of the bug (reviewer/superreviewer, patch
> author, checkin-needed pool), and so forth.
You see, it's exactly that which could be automated.
Gerv
Gervase Markham
> naming nits, or even fixing some minor remaining bugs. (To say
> nothing of then having to spend more time in the horrible bugzilla
> patch-editing experience!).
I'm not quite sure what you mean by this. I didn't realise you could
edit patches directly on Bugzilla. Anyway, if there's something which
isn't pleasant, is there a bug filed on making it more so? :-)
Gerv
Gervase Markham
> Another thought I had is that perhaps we could find tracking this
> information easier if Bugzilla's "Status" field reflected the actual
> statuses, e.g., by replacing "ASSIGNED" with "fix under
> development", "waiting for review", and "waiting for checkin" with
> an appropriately-defaulted checkbox to update this status as one
> would expect when editing attachments.
We can certainly have more workflow states if we want them - Bugzilla is
now configurable in this way. There's been a long-standing request to
add a READY state:
https://wiki.mozilla.org/BugzillaWorkflowImprovements
http://steelgryphon.com/testcases/bugzilla-workflow-9.png
But are these "states" not trackable at the moment by querying for e.g.
set review flags? Having a separate state for "waiting for review" would
mean a lot more toggling of state, and we'd have a DRY problem as well.
Gerv
Brendan Eich
wrote:
> (quickhttps://bugzilla.mozilla.org/show_bug.cgi?id=485217carry-over)
>
> > --- Comment #23 from Jeff Walden <jwalden+...@mit.edu> 2009-03-26 22:47:37 PDT ---
> > (In reply to comment #22)
>
> > Passive-aggressive insinuations of bad faith generally don't win friends and
> > influence people.
>
> I am annoyed, but really at myself rather than any other individual.
> As I have proved capable of not getting getting fixes committed in
> multiple projects, I'd be interested in anything you could point out
> that I should have done differently, so I can actually do better in
> future, in Mozilla and elsewhere.
One rule from the old days, possibly not known now, is to mail
st...@mozilla.org if you are getting no response, or insuffucient or
(in your view) wrong response, from a module owner. Please feel free
to mail me in any case.
In the modern era, st...@mozilla.org has withered away, but now we
have a module ownership module:
You'll have to map the names listed there to emails by hand (I guess
we don't want to make it too easy for spammers). Again, please mail me
in any event.
/be
/be
Brendan Eich
> I don't think documentation or processes are an issue here. Judging from
> the bug, this was a very simple case of slow responses.
But why were responses slow, given the severity of the bug? I think
because the severity was not appreciated. Free Memory Read when
calling through a vtable pointer is exploitable. This may not be fully
appreciated by all committers and drivers.
/be
Mike Kristoffersen
> On 3/27/09 9:22 AM, Mike Shaver wrote:
>> I feel pretty strongly that we should be doing more "committed with
>> some tweaks to whitespace and tests, thanks!" and less "bounced until
>> you make this round of minor changes, please try again later".
>
> I do some of that, and I've been told, I suspect correctly, that I'm not
> improving things in the long run by doing so.
I understand why this can be seen as not improving things in the long
run. But, if we just look at the whitespace/unexpanded tabs stuff it's a
trivial thing and one that could be checked with a tool (it might even
be fixed automatically with a tool) - Seems a waste of time to do by
hand, could be something that is done automatically when you try to
attach the fix, or when you request a review on it. It must be annoying
for reviewers to have to check for this too.
> If our goal when someone we've never seen before attaches a patch is to
> get that code in the tree, then there's no doubt that having "a Mozilla
> developer," the reviewer (in the post-SR world, where that actually
> works) or someone else, just de-nit and push is the way to go.
> If our goal is to turn the person who attached the patch in the first
> place into a Mozilla developer, I'm less sure it is. I know the feeling
> that "I had to fight about whitespace, and make some changes that I
> don't think made sense, and beg for a checkin, but I got *my code* in
> Firefox" produces; what feeling does "I attached a patch that someone
> else took, assigned the bug to themselves, changed around, and checked
> in" produce? Would we really be better off treating patchers as
> interchangeable disposable idea-factories, rather than treating them as
> the person we're going to train to be the next peer for the module?
I know that I would prefer this to having "my code" modified and then
checked in (unless it was modified by a tool)
But could we work with "conditional approved", so if e.g. there is a
typo in a comment, then you would be told that, but would not need
another review, just you would promise that you fix that issue and not
change anything else?
Mike Kristoffersen
> Mike Kristoffersen wrote:
>> - Now, any active bug must at all times have someone or a group that
>> is responsible for it (by active I mean, a bug that is not fixed and
>> in the tree or one that has been closed for another reason - like
>> "working as intended")
>
> Yes, indeed. The problem with this is that our current ratio of bugs to
> people is on the order of hundreds. That's far past what one can
> actually keep track of usefully... We could use group "nobody"
> placeholders like we do now to keep track of bugs, but those aren't
> going to do anything about bugs.
If there are no-one responsible, then how can we ensure that we focus
our attention on the most important things?
We might have someone having implicit responsibility for all of the bugs
today, but lets make it explicit. It is true that there are much more
bugs than we have people - this is good, as we won't get bored :) -
no... to be more serious, if we were to use a "nobody" place holder we
risk that:
1) too many people spend time keeping an eye on the list
or
2) too few people spend time keeping an eye on the list
Both are bad for their own reason (first one is duplicate work, second
one that something is missed).
I don't think the average developer should be responsible for more than
a few bugs at any one time, but the backlog (the bugs with no active
fixing/review going on) should be assigned to someone (person/group).
The one assigned to this backlog could have the responsibility to make a
prioritized list of what is most important to be fixed at any one time -
when a developer want to take on another bug, he would then know what is
most important to look at, or what is most important of the things that
he has the ability / interest to look at. Depending on the relationship
between Mozilla and the developer it could also be a more direct thing,
where he is told what to look at.
The backlog owner should also be able to push something very important
into development, if it is better for the project to have a specific
developer look at something specific compared to what else he is doing
at the moment.
The backlog owner position could be a pure administrative one (e.g.
where the owner didn't do any development himself, that would depend on
the workload and what else the person is assigned to taking care of)
We could also think about putting reviews into this backlog - going with
my initial post to this thread, how do we level out the workload for
doing reviews, and how do we avoid developers being disturbed doing
reviews, if it is more important for them to work on new development?
No-one can benefit from the way it is now where you need to "bribe"
reviewers with lunches to have them look at a change, as someone else
wrote?
Currently it might be the ones with the best personal relationships or
the ones that are most pushy that gets their changes reviewed first, we
should be able to trust the system - if you as a developer request a
review you should have confidence that the review will happen, it should
be the exception that you would need to push it trough.
Another way to look at it: If I request a review, who is failing to do
their job if the review is not done? Is it me, because I didn't push,
or the reviewer who ignored the initial request?
I know we are humans, and I'm not talking about the case where it's a
human error that meant the review didn't happen. But again, why should
I remind a reviewer that might have "forgotten" the request, when this
can be automated, sending a reminder mail ones a week to people - e.g:
----
Hi this is bugzilla with your weekly mail to remind you that you have:
1 active bug(s)
-bug xxx
2 pending review(s):
-bug x111, 2 days old
-bug x222, 5 weeks old
---
Yes maybe we should all actively search bugzilla for stuff assigned to
us, but why, if we can get it pushed to our inbox - I bet someone could
even make a small desktop app that would provide this information "live".
I already made my point with trigger mails to project owners, so I won't
repeat it here :)
>
>> - When you get assigned to a bug (for validation, testing, reviewing,
>> fixing, committing, what ever reason) you should receive an e-mail.
>> (Where you at a glance, from the title of the e-mail, can see that it
>> is about something you (or a group you are a member of) are getting
>> assigned to)
>
> For what it's worth, a review requestee already gets mail, and there is
> a Bugzilla page where you can see all your pending review requests. Some
> people look at this page frequently and do reviews. Some don't. Tooling
> changes won't change this. :(
I didn't review anything yet, so I can't comment on the format of these,
but as far as remember, I can't see from the title line of the mails
from bugzilla, if it's about a bug I'm assigned to, or one I get because
I'm on the CC list of the bug, or because I'm watching someone who is
getting info about it....
The worst might be, that if you attach a new fix to a bug asking for a
review on it, canceling a previous review request in the process - the
title just says "review request canceled" or something like that (it
might be another mail the reviewer get), making it easy to miss that the
body of the mail actually contains a new request.
I believe that having a mail pushed to you with information will
actually increase the odds that the information is read, compared to
having to pull the information from bugzilla.
>> - When a bug that you are assigned to is opened, you could get a
>> description that explains what you are supposed to do, this text being
>> defined by the state the bug is in, "Verify the bug", "Review attached
>> patch", etc.
>
> This could be good for novice contributors; it won't help reviewers
> shirking their duties: they already know what they're supposed to do
> with review requests!
Yep, that is true, and no, it will be the exact same amount of work that
needs to be done - still think that it would be nice having the process
inline documented compared to having it in a seperate doc, especially in
mozilla where you never really know if a page that you are looking at is
maintained and up to date, or if it has been replaced by something else,
or what more often seems to be the case, that it is almost correct with
a few exceptions that you have to guess :)
<SNIP>
>> -It is the same if you report an issue, you are only expected to give
>> a useful description of the issue. In my view, the reporter should not
>> push the fixing, by any other means than perhaps setting the initial
>> severity of the bug - it is the responsibility of the project, or
>> owner of the area to do the prioritization with respect to the other
>> tasks at hand.
>
> Though of course the reporter is welcome (but not required) to do more,
> and some do.
Yes, of cause we all have a duty to act if we know that something is
being missed or not moving along in an ideal way. That goes if it is
because we have been actively involved in, or we happen to find out
about it by "accident".
>
>> But the automation could also continue into choosing who to assign
>> bugs to at any given state - so if I create a new bug, it could
>> automatically get assigned to the group (or individual) responsible
>> for the area where the bug is reported in (with the e-mail notification)
>
> That already happens, to some extent. There are people watching the
> component qa addresses, so they get mail when bugs are filed. For
> example, I get mail on new form control, docshell, xbl bugs. I'd look at
> more components, but there's no way to look for only new bugs; if I
> watched all components I care about new bugs in I'd be getting over 2000
> bugmails per day (yes, I tried this once). I thought we had a bug on
> making it easier to watch only new bugs and bugs being moved into a
> component, but I can't find it right now... Maybe I should re-file it
> just in case. :(
Sounds like we need some more clever filtering options yes :)
Mike Kristoffersen
Could we remove the "review flag" in the process? - it would be more
nice to have a single place to look for the state (the state field) than
multiple seeing if flags are set, and what they are set to... (e.g.
+,-,? which might not be ultimately user friendly, this is said with all
respect ;) )
Benjamin Smedberg
> We might have someone having implicit responsibility for all of the bugs
> today, but lets make it explicit. It is true that there are much more
We don't have implicit responsibility. Most bugs automatically are assigned
to nobody@, and only if somebody is watching the bug component and notices
the bug does it get action.
> The one assigned to this backlog could have the responsibility to make a
> prioritized list of what is most important to be fixed at any one time -
Most bugs aren't worth spending core developer time on them: some are
technically "valid", e.g. a bug about building Mozilla on Darwin+X11, but
only people who care about that particular case should work on them. They
*should* be assigned to nobody@ and not thought about any more.
> Another way to look at it: If I request a review, who is failing to do
> their job if the review is not done? Is it me, because I didn't push,
> or the reviewer who ignored the initial request?
It is the reviewers fault, certainly.
> I didn't review anything yet, so I can't comment on the format of these,
> but as far as remember, I can't see from the title line of the mails
> from bugzilla, if it's about a bug I'm assigned to, or one I get because
> I'm on the CC list of the bug, or because I'm watching someone who is
> getting info about it....
The bugmail says "Review requested..."
> I believe that having a mail pushed to you with information will
> actually increase the odds that the information is read, compared to
> having to pull the information from bugzilla.
I doubt it. Most mozilla contributors receive hundreds of emails per day. An
email about a review request is just another in the bucket. I personally use
the review queue page
As the primary tracker of review requests. Otherwise, my inbox would
overflow. But that's probably best left to the personal style and
organization system of the reviewer.
> If there are no-one responsible, then how can we ensure that we focus
> our attention on the most important things?
Most bugs are not important. The problem here seems to be that the small
number of bugs which are actually important are being overwhelmed by the
quantity of bugs which are less important.
Perhaps the answer is that we should go back to the system of bugs
automatically being assigned to a real person when they are filed. But that
person should be willing to reassign 90% of the bugs to nobody@ and find a
good owner for the remaining 10%.
--BDS
Gavin Sharp
<mkristo...@mozilla.com> wrote:
> But could we work with "conditional approved", so if e.g. there is a typo in
> a comment, then you would be told that, but would not need another review,
> just you would promise that you fix that issue and not change anything else?
We already have this. Reviewers grant r+ with "r=me with this typo
fixed and that trailing whitespace removed" comments all the time.
Gavin
Boris Zbarsky
> But could we work with "conditional approved", so if e.g. there is a
> typo in a comment, then you would be told that, but would not need
> another review, just you would promise that you fix that issue and not
> change anything else?
That does happen. It's more likely to happen with a known contributor,
because it's far too common for people to screw up changes like this
(making code changes in addition to the comment change, screwing up the
diffing, etc). You'd think this would be simple, but apparently for
some people it's not. :(
-Boris
Boris Zbarsky
> Could we remove the "review flag" in the process? - it would be more
> nice to have a single place to look for the state (the state field) than
> multiple seeing if flags are set, and what they are set to... (e.g.
> +,-,? which might not be ultimately user friendly, this is said with all
> respect ;) )
This doesn't work for bugs that require more than one review (e.g. any
patch that touches more than one module).
-Boris
Mike Kristoffersen
> On 3/30/09 9:11 AM, Mike Kristoffersen wrote:
>
>> We might have someone having implicit responsibility for all of the bugs
>> today, but lets make it explicit. It is true that there are much more
>
> We don't have implicit responsibility. Most bugs automatically are assigned
> to nobody@, and only if somebody is watching the bug component and notices
> the bug does it get action.
That was my fear, but didn't want to step on anyones toes :)
>
>> The one assigned to this backlog could have the responsibility to make a
>> prioritized list of what is most important to be fixed at any one time -
>
> Most bugs aren't worth spending core developer time on them: some are
> technically "valid", e.g. a bug about building Mozilla on Darwin+X11, but
> only people who care about that particular case should work on them. They
> *should* be assigned to nobody@ and not thought about any more.
If by "aren't worth spending core developer time on" you mean, we don't
have bandwith nor interest in fixing this, then we should make this
decision one time, and move them into an "archived" state or something
like that.
>
>> Another way to look at it: If I request a review, who is failing to do
>> their job if the review is not done? Is it me, because I didn't push,
>> or the reviewer who ignored the initial request?
>
> It is the reviewers fault, certainly.
>
>> I didn't review anything yet, so I can't comment on the format of these,
>> but as far as remember, I can't see from the title line of the mails
>> from bugzilla, if it's about a bug I'm assigned to, or one I get because
>> I'm on the CC list of the bug, or because I'm watching someone who is
>> getting info about it....
>
> The bugmail says "Review requested..."
And you don't get this mail with the same title if you are e.g. the one
requesting the review?
>
>> I believe that having a mail pushed to you with information will
>> actually increase the odds that the information is read, compared to
>> having to pull the information from bugzilla.
>
> I doubt it. Most mozilla contributors receive hundreds of emails per day. An
> email about a review request is just another in the bucket. I personally use
> the review queue page
>
> https://bugzilla.mozilla.org/request.cgi?action=queue&requester=&product=&type=all&requestee=benjamin%40smedbergs.us&component=&group=type
>
> As the primary tracker of review requests. Otherwise, my inbox would
> overflow. But that's probably best left to the personal style and
> organization system of the reviewer.
It could be an idea to have it configurable, but assuming you scan the
headers of your mails manually - and get summary reports from bugzilla,
you might not see these e-mails as another one in the bucket?
I'm sure most times the current system works and that there are no
issues with developers that regularly check it manually :)
>
>> If there are no-one responsible, then how can we ensure that we focus
>> our attention on the most important things?
>
> Most bugs are not important. The problem here seems to be that the small
> number of bugs which are actually important are being overwhelmed by the
> quantity of bugs which are less important.
>
> Perhaps the answer is that we should go back to the system of bugs
> automatically being assigned to a real person when they are filed. But that
> person should be willing to reassign 90% of the bugs to nobody@ and find a
> good owner for the remaining 10%.
Sounds like a good idea to have a person manually scan the bugs, and
yes, if we know that we aren't going to fix it, then I think we agree
that the bug should be marked as such, if that is by assigning it to a
"dummy" developer like nobody@ or "closing" it by some archived state is
secondary :)
Boris Zbarsky
>> Most bugs aren't worth spending core developer time on them: some are
>> technically "valid", e.g. a bug about building Mozilla on Darwin+X11, but
>> only people who care about that particular case should work on them. They
>> *should* be assigned to nobody@ and not thought about any more.
>
> If by "aren't worth spending core developer time on" you mean, we don't
> have bandwith nor interest in fixing this, then we should make this
> decision one time, and move them into an "archived" state or something
> like that.
Who is "we" in this case? It's not like there's an "us" or a "them"
here. The bug is valid. It affects some people. It doesn't affect
many people, so it's very low priority in general. It's pretty
high-priority for the people it affects, though.
That describes a lot of bugs, actually. Most of them.
>> The bugmail says "Review requested..."
>
> And you don't get this mail with the same title if you are e.g. the one
> requesting the review?
Correct. When a review request is made, the requestee gets an email
completely separate from the normal bugmail that gets sent. People who
care about that sort of thing filter these into a separate folder and go
through them, or use the bugzilla request query.
> Sounds like a good idea to have a person manually scan the bugs, and
> yes, if we know that we aren't going to fix it
Again, not sure who the "we" is. There's a significant difference
between "wontfix" (as in, will not accept a change to this behavior)
bugs and "low priority" (as in, no plans for the module owner to work on
this or hunt up resources to fix it, but if someone who cares about the
issue patches the patch will be accepted).
-Boris
Mike Kristoffersen
> Mike Kristoffersen wrote:
>>> Most bugs aren't worth spending core developer time on them: some are
>>> technically "valid", e.g. a bug about building Mozilla on Darwin+X11,
>>> but
>>> only people who care about that particular case should work on them.
>>> They
>>> *should* be assigned to nobody@ and not thought about any more.
>>
>> If by "aren't worth spending core developer time on" you mean, we
>> don't have bandwith nor interest in fixing this, then we should make
>> this decision one time, and move them into an "archived" state or
>> something like that.
>
> Who is "we" in this case? It's not like there's an "us" or a "them"
> here. The bug is valid. It affects some people. It doesn't affect many
> people, so it's very low priority in general. It's pretty high-priority
> for the people it affects, though.
>
> That describes a lot of bugs, actually. Most of them.
Ok, I missed the "only people who care about them" above - so the bugs
we talk about are some that are not parked in a state "will not be
done", it is just waiting for anyone to come along to fix it as any
other bug is - in that case I think there should still be someone
responsible for it (not responsible in the sense, that it is expected
that the responsible person fix it, but rather should know about it,
look at it at regular intervals (ones a year, every month, what ever is
suitable).
Now why not just leave it to nobody@ ? - Because then the things that
can be done might not be done - e.g. marking duplicates, close the ones
that over time has become "wont-fix" or boost the priority of the ones
that becomes more important due to changes in the environment (like
Darwin+X11 becomes the favorite for the wife *S*)
>
>>> The bugmail says "Review requested..."
>>
>> And you don't get this mail with the same title if you are e.g. the
>> one requesting the review?
>
> Correct. When a review request is made, the requestee gets an email
> completely separate from the normal bugmail that gets sent. People who
> care about that sort of thing filter these into a separate folder and go
> through them, or use the bugzilla request query.
Good, no issue there then - I know I received some "review canceled:
[Bug xxxxx] ...." mails, that I was not the reviewer of (an embedded in
these were the request for a new review), but happy to hear it isn't the
same for the person asked to do the review.
>
>> Sounds like a good idea to have a person manually scan the bugs, and
>> yes, if we know that we aren't going to fix it
>
> Again, not sure who the "we" is. There's a significant difference
> between "wontfix" (as in, will not accept a change to this behavior)
> bugs and "low priority" (as in, no plans for the module owner to work on
> this or hunt up resources to fix it, but if someone who cares about the
> issue patches the patch will be accepted).
We, like in the same people that can decide "wontfix" - which is, I
assume, the same in this context, for a moment I forgot that this is
open source ;)
Mike Kristoffersen
I see the problem with only having the state to reflect the review
state, when there can be multiple modules with different owners involved
- how is this handled today, do we make a changelist for each module
with one reviewer each, or make multiple reviewers for one changelist?
Boris Zbarsky
> I see the problem with only having the state to reflect the review
> state, when there can be multiple modules with different owners involved
> - how is this handled today, do we make a changelist for each module
> with one reviewer each, or make multiple reviewers for one changelist?
"yes". Both approaches are used. Further, sometimes two reviews are
requested for the same module (e.g. r= and sr=, or just two r= for
particularly complicated code).
-Boris
Michael Connor
On 30-Mar-09, at 7:31 AM, Brendan Eich wrote:
> In the modern era, st...@mozilla.org has withered away, but now we
> have a module ownership module:
>
> https://wiki.mozilla.org/index.php?title=Module_Owners_Activities_Modules
> #Governance_Submodule:__Module_Ownership_System
You can now reach this group at module-o...@mozilla.org
-- Mike
Benjamin Smedberg
> Ok, I missed the "only people who care about them" above - so the bugs
> we talk about are some that are not parked in a state "will not be
> done", it is just waiting for anyone to come along to fix it as any
> other bug is - in that case I think there should still be someone
> responsible for it (not responsible in the sense, that it is expected
> that the responsible person fix it, but rather should know about it,
> look at it at regular intervals (ones a year, every month, what ever is
> suitable).
The cost of having an owner than the cost of not having an owner. Besides
which, having an owner doesn't guarantee that the owner is going to actually
*do* all these thing you seem to think they're going to do.
There are currently 551 bugs assigned to nob...@mozilla.org in the XPCOM
component alone. Actively going through that list would take at least a
week. I don't think it's worth 1/26th of any developer's time for the
comparatively-small benefit of marking bugs WONTFIX or DUPLICATE. And XPCOM
is probably much better off than areas which touch web content, where the
inactive-bug list is probably measured in thousands or tens-of-thousands.
--BDS
Boris Zbarsky
> There are currently 551 bugs assigned to nob...@mozilla.org in the XPCOM
> component alone. Actively going through that list would take at least a
> week. I don't think it's worth 1/26th of any developer's time for the
> comparatively-small benefit of marking bugs WONTFIX or DUPLICATE. And XPCOM
> is probably much better off than areas which touch web content, where the
> inactive-bug list is probably measured in thousands or tens-of-thousands.
Just as a data point to support this, the number of UNCONFIRMED
Firefox:General bugs as of right now is 7349.
Of those, 334 have been filed since March 1, 2009.
In general, for the Firefox and Core products together we have had 2134
bugs filed since March 1, 2009. That's about 70 bugs per day; just
reading through those bug reports and trying to triage them would be a
full-time job for probably 3 fairly competent people. That's assuming
they don't try to create testcases for the layout bugs and such.
Of these 2134 bugs:
293 are fixed.
142 are resolved invalid.
29 are resolved wontfix.
270 are resolved duplicate.
50 are resolved worksforme (though a number of these should be
invalid, imo).
7 are resolve incomplete.
That leaves 1343 still-open bugs. Of these, 636 are unconfirmed. The
rest are definitely issues, of various sorts (617 NEW, 82 ASSIGNED, 8
REOPENED). So it looks like we're fixing bugs about 1/7 as fast as they
get filed, that about 25% of bugs filed are marked
invalid/worksforme/wontfix within a month or so, but that we still end
up with some fairly significant backlog. Looking through this on a
consistent basis would be a huge time-sink, as Benjamin says...
For another comparison, there are currently 2341 bugs in Layout:*
components assigned to nobody. Of these, 569 are unconfirmed.
-Boris
Blake Kaplan
> But why were responses slow, given the severity of the bug? I think
> because the severity was not appreciated. Free Memory Read when
> calling through a vtable pointer is exploitable. This may not be fully
> appreciated by all committers and drivers.
Becaues the bug was mis-triaged. The reporter didn't mention the magic words
"exploit" or "vulnerability". Furthermore, the bug was marked blocking1.9.1-,
so was deprioritized. Even small reviews take a noticeable amount of time to
do (more so if you haven't looked at the code in question in a while) so a
review on a crash that's blocking- could easily fall by the wayside.
--
Blake Kaplan
Robert O'Callahan
> I doubt it. Most mozilla contributors receive hundreds of emails per day. An
> email about a review request is just another in the bucket. I personally use
> the review queue page
>
> https://bugzilla.mozilla.org/request.cgi?action=queue&requester=&product=&type=all&requestee=benjamin%40smedbergs.us&component=&group=type
>
> As the primary tracker of review requests. Otherwise, my inbox would
> overflow. But that's probably best left to the personal style and
> organization system of the reviewer.
I personally get review requests in my inbox and leave them there until
they're done or obsoleted. I spend a lot effort keeping that list around
zero.
Rob
Dan Mosedale
>
> It's really important to stay on top of bugmail and handle review
> requests as quickly as possible. This bug isn't even the worst case
> (apart from the unfortunate publication of an exploit); the worst case
> is that the contributor just gets fed up and disappears. Maybe for MoCo
> employees we can monitor this and publish the stats? :-) Beyond my
> Bugzilla skills though.
I don't see how that relates to MoCo at all.
Mozilla has done a fantastic job of moving from a world of minimal
functionality and performance testing to a heavily automated world that
notices quickly when things go wrong. If we attack our bug throughput
issues with similar zeal, we should be able to make similarly large strides.
Dan
Dan Mosedale
> On 3/27/09 9:22 AM, Mike Shaver wrote:
>> I feel pretty strongly that we should be doing more "committed with
>> some tweaks to whitespace and tests, thanks!" and less "bounced until
>> you make this round of minor changes, please try again later".
Hear, hear!
> I do some of that, and I've been told, I suspect correctly, that I'm not
> improving things in the long run by doing so.
>
> If our goal when someone we've never seen before attaches a patch is to
> get that code in the tree, then there's no doubt that having "a Mozilla
> developer," the reviewer (in the post-SR world, where that actually
> works) or someone else, just de-nit and push is the way to go.
>
> If our goal is to turn the person who attached the patch in the first
> place into a Mozilla developer, I'm less sure it is. I know the feeling
> that "I had to fight about whitespace, and make some changes that I
> don't think made sense, and beg for a checkin, but I got *my code* in
> Firefox" produces; what feeling does "I attached a patch that someone
> else took, assigned the bug to themselves, changed around, and checked
> in" produce? Would we really be better off treating patchers as
> interchangeable disposable idea-factories, rather than treating them as
> the person we're going to train to be the next peer for the module?
It seems to me that there's a false dichotomy at work here, and that
there's a style of reviewing that balances the concerns you mention. To
wit: when reviewing patches from people that you not aware of having
significant Mozilla experience, point out the specific whitespace /
style differences, but if they're the only problem with the patch, then
just fix them directly and check them in, asking the contributor to
watch out for those issues next time. As you get to know contributors
better and get to the point where you're sure they should know better,
it's entirely reasonable to push back more.
Dan
John J. Barton
> Benjamin Smedberg wrote:
>> There are currently 551 bugs assigned to nob...@mozilla.org in the XPCOM
>> component alone. Actively going through that list would take at least a
>> week. I don't think it's worth 1/26th of any developer's time for the
>> comparatively-small benefit of marking bugs WONTFIX or DUPLICATE. And
>> XPCOM
>> is probably much better off than areas which touch web content, where the
>> inactive-bug list is probably measured in thousands or tens-of-thousands.
>
> Just as a data point to support this, the number of UNCONFIRMED
> Firefox:General bugs as of right now is 7349.
>
We're off the subject, but this problem seems like one that would
benefit from research and development at the leading edge of social
networking. In the community of 7000+ folks are clusters of people who
have similar problems and different perspectives. I've seen numerous
cases on the Firebug side where users effectively collaborate, starting
with 'me too', then sometimes a while later, a "oh the problem went away
when I uninstalled Firefoo". You see this a lot on bugzilla for 'high'
end problems like security and crashes, but better software could help
the large tail find each other. (I know there are other channels in
Firefox case, but evidence here is that they are not adequate for
solving the issues that get to bugzilla).
So mention this problem around, maybe something will fall out...
jjb
Mike Beltzner
confirmers. I'm talking about people like Ria, who goes through bugs
trying to reproduce, finding regression ranges, etc. This work is as
valuable as writing patches, yet I don't get the impression that's
felt throughout our community.
There are many people in our community who would like to help, but
"aren't really coders" or "don't have the skills to write a patch".
Those people do have the skills to open an unconfirmed bug and see if
they can reproduce it, though. A while back some Seneca students wrote
an add-on that loaded unconfirmed bugs in a sidebar and tried to make
it quick and easy for someone to spend 10-15 minutes confirming a few
bugs. This, more than many of the behemoth tasks we face, feels like
it can be divded and conquered.
Perhaps a job for the emerging MozQA community?
cheers,
mike
> _______________________________________________
> governance mailing list
> gover...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/governance
Gervase Markham
> IMO, we need to seriously elevate the cachet of bug triagers and
> confirmers. I'm talking about people like Ria, who goes through bugs
> trying to reproduce, finding regression ranges, etc. This work is as
> valuable as writing patches, yet I don't get the impression that's felt
> throughout our community.
Perhaps one reason might be because there's so much of it to be done
that no-one ever emerges as "owning" it or makes a significant
percentage contribution on their own? (It's not a good reason, of course.)
> There are many people in our community who would like to help, but
> "aren't really coders" or "don't have the skills to write a patch".
> Those people do have the skills to open an unconfirmed bug and see if
> they can reproduce it, though. A while back some Seneca students wrote
> an add-on that loaded unconfirmed bugs in a sidebar and tried to make it
> quick and easy for someone to spend 10-15 minutes confirming a few bugs.
> This, more than many of the behemoth tasks we face, feels like it can be
> divded and conquered.
Ubuntu has a "five a day" program, trying to encourage all community
members to work on at least five bugs a day:
https://wiki.ubuntu.com/5-A-Day
Is it worth us trying something like that?
Gerv
Gervase Markham
> Could we remove the "review flag" in the process? - it would be more
> nice to have a single place to look for the state (the state field) than
> multiple seeing if flags are set, and what they are set to... (e.g.
> +,-,? which might not be ultimately user friendly, this is said with all
> respect ;) )
No, because reviews are requested of particular people. We can't replace
it with just a TO_REVIEW state, because then we wouldn't know who was
responsible for the review, and they wouldn't get mailed.
Flags are learnable, not usable - like driving a car. They aren't
designed to be the part of the Bugzilla interface that new users
interact with.
Gerv
Boris Zbarsky
> IMO, we need to seriously elevate the cachet of bug triagers and
> confirmers.
Amen.
> I'm talking about people like Ria, who goes through bugs
> trying to reproduce, finding regression ranges, etc. This work is as
> valuable as writing patches, yet I don't get the impression that's felt
> throughout our community.
At least in layout, I'd sure hope it's recognized! For a layout bug,
creating a testcase and finding a regression range is often just as hard
as fixing the bug.
> Perhaps a job for the emerging MozQA community?
Back in the day, this was one of the things Asa used to do: organizing
bugdays to go through unconfirmed bugs, with a channel that people could
hang out in to do so, etc. It might make sense to try doing that again;
have an irc channel that people hang out in who're doing UNCO triage so
they can confer on bugs that aren't obvious; publicize this so that some
developers who have broad knowledge of the codebase can hang out too in
case things are really unclear for a particular bug. I'd be happy to
help out with this sort of thing...
I do think this would also be easier if component owners had a sane way
to track when a bug is moved into their component; that wouldn't help
the Firefox:General UNCO, but it would certainly help once a bug has
been initially triaged.
-Boris
Mike Kristoffersen
I agree that flags are learn-able (like a command like interface vs. a
gui interface), I'll try not to start a war by avoiding examples at this
point ;)
I have been thinking quite a bit about the issue with having multiple
reviewers looking at the code as mentioned by Boris, I think this is the
test case any new way of doing it must pass :) - in the case where
there is a single reviewer, as I have stated earlier, I think the
assignment of the bug should be with the reviewer, as he is the one who
are expected to do something with the bug.
But this way of doing it seems to have a hard time to handle multiple
reviewers, without making a system with the same problems as the flags.
I haven't abandoned the idea yet, need to think carefully :)
We might consider not assigning individuals to do reviews in the bugs,
for the reasons I have mentioned earlier - basically the one asking for
a review might not know the availability of the reviewer and we should
do conscious prioritization, not one based on personal relationships and
random e-mails to provoke a review.
Mike Kristoffersen
> Benjamin Smedberg wrote:
>> There are currently 551 bugs assigned to nob...@mozilla.org in the XPCOM
>> component alone. Actively going through that list would take at least a
> Just as a data point to support this, the number of UNCONFIRMED
> Firefox:General bugs as of right now is 7349.
>
> Of those, 334 have been filed since March 1, 2009.
> For another comparison, there are currently 2341 bugs in Layout:*
> components assigned to nobody. Of these, 569 are unconfirmed.
>
I just took some random bits of some very impressive numbers above :)
Yes, the numbers are big, but that is just more reason to setup a
process to deal with them - A couple of other people mentions different
ways of involving non-coders in the community - what an excellent idea!
Lets start to think how we can bring these bugs under control!
Boris Zbarsky
> Yes, the numbers are big, but that is just more reason to setup a
> process to deal with them - A couple of other people mentions different
> ways of involving non-coders in the community - what an excellent idea!
We've been trying to do that for years now. Heck, that's how I got
involved in the project to start with.... in 2001.
-Boris
Jonas Sicking
> On 27 Mrz., 09:28, Robert O'Callahan <rob...@ocallahan.org> wrote:
>> I don't think documentation or processes are an issue here. Judging from
>> the bug, this was a very simple case of slow responses.
>
> But why were responses slow, given the severity of the bug? I think
> because the severity was not appreciated. Free Memory Read when
> calling through a vtable pointer is exploitable. This may not be fully
> appreciated by all committers and drivers.
In this case it was just me not thinking through it enough. I'm
certainly aware that most crashes are exploitable. However I didn't
think about the fact when triaging the bug in my head and thus put off
massaging the patch to get it to the state I wanted before checking it in.
What ended causing me not to take immediate action was that a testcase
that I wanted was removed from the patch. So it wasn't as simple as
taking the patch and checking it in. Of course, it wasn't a whole lot
harder than that so I'm not going to argue that this was a valid excuse
for stalling.
/ Jonas
Jonas Sicking
>> Those are just for starters, and I'm sure there are others people can bring up as discussion happens.
>>
>> All comments are appreciated.
>
> I don't think I have anything that hasn't been said already. Could
> complain about bugzilla, or large software projects, or attitudes to
> security bugs, but that's all clearly part of the territory if you
> want to help out on mozilla.
>
> Fundamentally, when the core contributors are such a bottleneck that
> it's worth them spending the 5 minutes to post a bugzilla review that
> consists mostly of nits about whitespace or whatever - rather than the
> 15 to pull it into their tree and just fix the things and push - then
> random patchers are just going to have to suck up the slow back-and-
> forth.
Generally speaking the goal for me when pointing out nits when doing
reviews is to save me from future work. I.e. I'm generally hoping that
the person will contribute further patches in the future. When that is
the case it's good to get things corrected right away so you don't have
to mess with it in future patches.
However it's absolutely possible that I, and others, have too many nits
on patches from new contributers and we should just fix ourselves. I
totally agree it's important not to be discouraging new contributers, as
we want more of them :)
> And as a counterpoint to where things go wrong, the reason I was
> poking around looking for XSL bugs to fix in the first place was
> because the first speculative patch I posted got a review in less than
> a day - after I'd been waiting months for a patch review on a non-
> mozilla project.
Yay, I did something right in that bug! ;)
Seriously though. I am very sorry it took so long to get to the patch. I
hope you'll take my initial reaction as a better indicator of how glad I
was to see someone help out with, or even care about, the XSLT code.
/ Jonas
Jean-Marc Desperrier
> [...] I
> had two queries to help me: one for bugs with non-obsolete patches
> where there had been no bug activity in over a month, and one for bugs
> with non-obsolete patches that had r+ and sr+. [...]
I think doing that would be very useful, in fact so useful it could be
worth it to add two new steps in the workflow (HAS_PATCH,
READY_FOR_CHECKIN) to help spotting bugs in those two states.
But maybe keywords are probably more adequate.
> [...] The queries weren't perfect, so some better patch+flag
> query tools in Bugzilla would help,
Could you detail what kind of problems the queries had, and changes you
would consider useful to easier things ?
fantasai
>
> We might consider not assigning individuals to do reviews in the bugs,
> for the reasons I have mentioned earlier - basically the one asking for
> a review might not know the availability of the reviewer and we should
> do conscious prioritization, not one based on personal relationships and
> random e-mails to provoke a review.
Deciding who should review is less about personal relationships and more
about who knows that part of the code best. If the patch author flags the
wrong person, he will usually (in my experience) take a look and then flag
the appropriate reviewer in his place.
Also, there's the question of ownership: if nobody owns the review, it's
much less likely to get done. I think we had this problem a long time
ago and switched to asking specific people... because I seem to remember
asking a mailing list for review at some point. Or something.
It's the point where the reviewer tackles (or doesn't) his review queue
that we're having trouble with here. Maybe having a query that shows
everyone's pending reviews (before some given date) would help here.
The review would still be owned by a specific person, but drivers could
then take on making sure the reviewers are making progress on their
review queues, like they do with blockers today.
~fantasai