A bug was filed, a patch was posted, but it didn't get in the tree until "too late" (for a not-worst-case-scenario definition of the phrase, but still way too close to it for comfort). Some questions to ponder:
When did we fail to keep things moving along? How do we make the process clearer and more discoverable for new hands? What documentation needs to be modified to help that clarification? How and where should such documentation be found, and where should we position links to it on our sites to make it easy to find? How should responsibility for making sure posted patches get in the tree be split up? Are there any changes we should make to the super-review process (or the review process) to ensure patches don't fall through the cracks? Where do we document any such changes? (Do we even *have* any documentation on doing non-super-review reviews and what the goal of a review is?)
Those are just for starters, and I'm sure there are others people can bring up as discussion happens.
> When did we fail to keep things moving along? > How do we make the process clearer and more discoverable for new hands? > What documentation needs to be modified to help that clarification? > How and where should such documentation be found, and where should we > position links to it on our sites to make it easy to find? > How should responsibility for making sure posted patches get in the tree > be split up? > Are there any changes we should make to the super-review process (or the > review process) to ensure patches don't fall through the cracks? > Where do we document any such changes? (Do we even *have* any > documentation on doing non-super-review reviews and what the goal of a > review is?)
I don't think documentation or processes are an issue here. Judging from the bug, this was a very simple case of slow responses.
It's really important to stay on top of bugmail and handle review requests as quickly as possible. This bug isn't even the worst case (apart from the unfortunate publication of an exploit); the worst case is that the contributor just gets fed up and disappears. Maybe for MoCo employees we can monitor this and publish the stats? :-) Beyond my Bugzilla skills though.
> --- Comment #23 from Jeff Walden <jwalden+...@mit.edu> 2009-03-26 22:47:37 PDT --- > (In reply to comment #22)
> Passive-aggressive insinuations of bad faith generally don't win friends and > influence people.
I am annoyed, but really at myself rather than any other individual. As I have proved capable of not getting getting fixes committed in multiple projects, I'd be interested in anything you could point out that I should have done differently, so I can actually do better in future, in Mozilla and elsewhere.
On Mar 27, 5:46 am, Jeff Walden <jwalden+...@mit.edu> wrote:
> Those are just for starters, and I'm sure there are others people can bring up as discussion happens.
> All comments are appreciated.
I don't think I have anything that hasn't been said already. Could complain about bugzilla, or large software projects, or attitudes to security bugs, but that's all clearly part of the territory if you want to help out on mozilla.
Fundamentally, when the core contributors are such a bottleneck that it's worth them spending the 5 minutes to post a bugzilla review that consists mostly of nits about whitespace or whatever - rather than the 15 to pull it into their tree and just fix the things and push - then random patchers are just going to have to suck up the slow back-and- forth.
And as a counterpoint to where things go wrong, the reason I was poking around looking for XSL bugs to fix in the first place was because the first speculative patch I posted got a review in less than a day - after I'd been waiting months for a patch review on a non- mozilla project.
> A bug was filed, a patch was posted, but it didn't get in the tree until > "too late" (for a not-worst-case-scenario definition of the phrase, but > still way too close to it for comfort). Some questions to ponder:
> When did we fail to keep things moving along? > How do we make the process clearer and more discoverable for new hands? > What documentation needs to be modified to help that clarification? > How and where should such documentation be found, and where should we > position links to it on our sites to make it easy to find? > How should responsibility for making sure posted patches get in the tree > be split up? > Are there any changes we should make to the super-review process (or the > review process) to ensure patches don't fall through the cracks? > Where do we document any such changes? (Do we even *have* any > documentation on doing non-super-review reviews and what the goal of a > review is?)
> Those are just for starters, and I'm sure there are others people can > bring up as discussion happens.
> All comments are appreciated.
> Jeff
Being relatively green in the Mozilla processes I don't know the details of how it is handled today - but it seems to depend a lot on humans doing things that could be happening automatically. Another problem is documentation of the process - I have been told part of the process, but never seen the document that describes it - now this might be because I didn't search for it - I'll get back to that in a sec.
So the issues I see are:
1) Lack of automation 2) You need to pull to know the process - (especially bad when the process changes)
----
What I would like to see is that this gets integrated into bugzilla, let me explain:
- I'll assume that we have a well defined process, stating which states a bug can enter from a given other state, and the reasons/triggers for each state change. (I think it is out of scope to discuss the specific states here, as the point being that there are states)
- Now, any active bug must at all times have someone or a group that is responsible for it (by active I mean, a bug that is not fixed and in the tree or one that has been closed for another reason - like "working as intended")
- This means that if you are not responsible for a bug then you are not expected to do anything with it. So when you call for a review on a bug, you are no longer responsible for the bug, unless it comes back to you with a failed review - this could be automated, if you set the review flag "state", then if the reviewer fails the review (setting the "failed review" state) it gets automatically assigned back to the one who requested the review.
- When you get assigned to a bug (for validation, testing, reviewing, fixing, committing, what ever reason) you should receive an e-mail. (Where you at a glance, from the title of the e-mail, can see that it is about something you (or a group you are a member of) are getting assigned to)
- When a bug that you are assigned to is opened, you could get a description that explains what you are supposed to do, this text being defined by the state the bug is in, "Verify the bug", "Review attached patch", etc.
You then only get the options for changing states that is defined by the process - with a description of why you would want to move it to that state (e.g. for a bug you are assigned to fixing - you could have:
1) A fix for the bug has been attached => state=waiting for review, assignedTo=reviewer
2) You can't reproduce the bug => state=more info wanted, assignedTo=reporter
3) You don't have time to work on it now => state=waiting for time, assignedTo=you
4) You have accepted there is an issue and are working on fixing it => state=accepted working on it, assignedTo=you
etc. note that the above choice are only to illustrate the principle, the actual states are again out of scope for this discussion.
----
The above should give visibility to the process - there is only one rule you must know - if you get a bug assigned, it is your responsibility to move it to the next state - you are told everything else you need to know when you open the bug in bugzilla.
If the bug at any state gets assigned to a group instead of an individual, everyone in that group can then change the assignment to an individual person in the group (so we don't get multiple persons working on the same issue unintentionally).
----
So how do we ensure that bugs don't gets forgotten by individuals / stay in the system for ever (or past an important deadline)?
-Assuming there are someone responsible for any given product and/or area then these people should be able to setup triggers with time-limits, so if a high impact bug stays in the review state for more than a specified time, then the one responsible for the product gets notified, and can choose to take appropriate action.
-In my view, it should never be e.g. a developers responsibility to prioritize for or push a reviewer - it must be the project that does this kind of prioritization.
-It is the same if you report an issue, you are only expected to give a useful description of the issue. In my view, the reporter should not push the fixing, by any other means than perhaps setting the initial severity of the bug - it is the responsibility of the project, or owner of the area to do the prioritization with respect to the other tasks at hand.
----
I have written a little about automation in the above, e.g. the triggers the project owner can set if relevant bugs gets stalled in the system, and the e-mails that are send with tags in the title if you get assigned something new (at least we have the e-mails today)
But the automation could also continue into choosing who to assign bugs to at any given state - so if I create a new bug, it could automatically get assigned to the group (or individual) responsible for the area where the bug is reported in (with the e-mail notification)
When a bug is moved to the review state, the reviewer could be chosen from the files that are patched, etc.
----
I know the title of this thread talks about the post-fix process, but I think we could deal with the whole process here :)
^__^ Mike
-- /////////////////////////////////////////////////////////// // Documentation is written for humans, not for machines // ///////////////////////////////////////////////////////////
>> A bug was filed, a patch was posted, but it didn't get in the tree >> until >> "too late" (for a not-worst-case-scenario definition of the phrase, >> but >> still way too close to it for comfort). Some questions to ponder:
>> When did we fail to keep things moving along? >> How do we make the process clearer and more discoverable for new >> hands? >> What documentation needs to be modified to help that clarification? >> How and where should such documentation be found, and where should we >> position links to it on our sites to make it easy to find? >> How should responsibility for making sure posted patches get in the >> tree >> be split up?
I'm not sure that we need new or additional process here, to be honest. It is the responsibility of a bug owner to ensure that it transitions from state to state to state, and it's the responsibility of a component owner to ensure that bugs are actively being triaged and moved through the states and closed out if they're languishing or not getting attention. My feeling is that the more process we have, the harder things are to understand, and we have quite enough to understand already.
To me it feels like what we need are more people interested in this administration of our process, doing the work of walking through buglists and making sure. Additionally, tools that make it easier to see bug aging reports at a glance, and the ability to query for bugs that have "reviewed, approved patches" would allow us to spot cases where we have fixes in hand that are simply waiting for a landing window. Further, our continued efforts to keep the trees as green as possible as a mechanism for speeding landings will also help here.
More documentation is dandy, especially if annotated with screenshots or even screencasts.
Mike Kristoffersen wrote: > Another problem is > documentation of the process - I have been told part of the process, but > never seen the document that describes it - now this might be because I > didn't search for it - I'll get back to that in a sec.
This probably needs to be adjusted to reflect the new sr policy, though.
> - Now, any active bug must at all times have someone or a group that is > responsible for it (by active I mean, a bug that is not fixed and in the > tree or one that has been closed for another reason - like "working as > intended")
Yes, indeed. The problem with this is that our current ratio of bugs to people is on the order of hundreds. That's far past what one can actually keep track of usefully... We could use group "nobody" placeholders like we do now to keep track of bugs, but those aren't going to do anything about bugs.
> - When you get assigned to a bug (for validation, testing, reviewing, > fixing, committing, what ever reason) you should receive an e-mail. > (Where you at a glance, from the title of the e-mail, can see that it is > about something you (or a group you are a member of) are getting > assigned to)
For what it's worth, a review requestee already gets mail, and there is a Bugzilla page where you can see all your pending review requests. Some people look at this page frequently and do reviews. Some don't. Tooling changes won't change this. :(
> - When a bug that you are assigned to is opened, you could get a > description that explains what you are supposed to do, this text being > defined by the state the bug is in, "Verify the bug", "Review attached > patch", etc.
This could be good for novice contributors; it won't help reviewers shirking their duties: they already know what they're supposed to do with review requests!
> The above should give visibility to the process - there is only one rule > you must know - if you get a bug assigned, it is your responsibility to > move it to the next state - you are told everything else you need to > know when you open the bug in bugzilla.
This does have some benefits, yes.
> So how do we ensure that bugs don't gets forgotten by individuals / stay > in the system for ever (or past an important deadline)?
Right.
> -It is the same if you report an issue, you are only expected to give a > useful description of the issue. In my view, the reporter should not > push the fixing, by any other means than perhaps setting the initial > severity of the bug - it is the responsibility of the project, or owner > of the area to do the prioritization with respect to the other tasks at > hand.
Though of course the reporter is welcome (but not required) to do more, and some do.
> But the automation could also continue into choosing who to assign bugs > to at any given state - so if I create a new bug, it could automatically > get assigned to the group (or individual) responsible for the area where > the bug is reported in (with the e-mail notification)
That already happens, to some extent. There are people watching the component qa addresses, so they get mail when bugs are filed. For example, I get mail on new form control, docshell, xbl bugs. I'd look at more components, but there's no way to look for only new bugs; if I watched all components I care about new bugs in I'd be getting over 2000 bugmails per day (yes, I tried this once). I thought we had a bug on making it easier to watch only new bugs and bugs being moved into a component, but I can't find it right now... Maybe I should re-file it just in case. :(
I feel pretty strongly that we should be doing more "committed with some tweaks to whitespace and tests, thanks!" and less "bounced until you make this round of minor changes, please try again later". Cairo does the former, and it makes it a lot more welcoming as well as, I believe, putting a lower burden on reviewers in aggregate. Reloading a patch's context, plus actually remembering to revisit at all, is usually a lot more costly than just fixing up little whitespace and naming nits, or even fixing some minor remaining bugs. (To say nothing of then having to spend more time in the horrible bugzilla patch-editing experience!). It may be that some part-time contributors only have time for driveby reviews and not to actually check in, but I don't think that holds for anyone on staff at Mozilla or elsewhere.
Mike
On 3/27/09, Mike Beltzner <beltz...@mozilla.com> wrote:
>>> A bug was filed, a patch was posted, but it didn't get in the tree >>> until >>> "too late" (for a not-worst-case-scenario definition of the phrase, >>> but >>> still way too close to it for comfort). Some questions to ponder:
>>> When did we fail to keep things moving along? >>> How do we make the process clearer and more discoverable for new >>> hands? >>> What documentation needs to be modified to help that clarification? >>> How and where should such documentation be found, and where should we >>> position links to it on our sites to make it easy to find? >>> How should responsibility for making sure posted patches get in the >>> tree >>> be split up?
> I'm not sure that we need new or additional process here, to be > honest. It is the responsibility of a bug owner to ensure that it > transitions from state to state to state, and it's the responsibility > of a component owner to ensure that bugs are actively being triaged > and moved through the states and closed out if they're languishing or > not getting attention. My feeling is that the more process we have, > the harder things are to understand, and we have quite enough to > understand already.
> To me it feels like what we need are more people interested in this > administration of our process, doing the work of walking through > buglists and making sure. Additionally, tools that make it easier to > see bug aging reports at a glance, and the ability to query for bugs > that have "reviewed, approved patches" would allow us to spot cases > where we have fixes in hand that are simply waiting for a landing > window. Further, our continued efforts to keep the trees as green as > possible as a mechanism for speeding landings will also help here.
> More documentation is dandy, especially if annotated with screenshots > or even screencasts.
> A bug was filed, a patch was posted, but it didn't get in the tree until > "too late" (for a not-worst-case-scenario definition of the phrase, but > still way too close to it for comfort). Some questions to ponder:
> When did we fail to keep things moving along? > How do we make the process clearer and more discoverable for new hands?
The internal notation of the project confuses external contributors. For example 'in-the-tree' is obviously very important to developers, as the things like FIXED1.9.1. I watch a few dozen bugs and try to move them forward but very often I cannot understand what state they are in. We deal in values like "FF3.1b3" but these don't appear on the bug reports. I still don't know how developers can tell if a fix is in the end product or not. This seems like an area where automation would be feasible.
Separately, I notice that often a key time-consuming step near the end is the introduction of second, verification test case. So a bug report goes nowhere until a clear test case is reproduced. Then again it gets stuck if the fix has no verification test. Usually it seems that two tests are needed and both consume a lot of resources. If the tests were easier to create, then maybe the process would smoother.
> I feel pretty strongly that we should be doing more "committed with > some tweaks to whitespace and tests, thanks!" and less "bounced until > you make this round of minor changes, please try again later".
I do some of that, and I've been told, I suspect correctly, that I'm not improving things in the long run by doing so.
If our goal when someone we've never seen before attaches a patch is to get that code in the tree, then there's no doubt that having "a Mozilla developer," the reviewer (in the post-SR world, where that actually works) or someone else, just de-nit and push is the way to go.
If our goal is to turn the person who attached the patch in the first place into a Mozilla developer, I'm less sure it is. I know the feeling that "I had to fight about whitespace, and make some changes that I don't think made sense, and beg for a checkin, but I got *my code* in Firefox" produces; what feeling does "I attached a patch that someone else took, assigned the bug to themselves, changed around, and checked in" produce? Would we really be better off treating patchers as interchangeable disposable idea-factories, rather than treating them as the person we're going to train to be the next peer for the module?
Robert O'Callahan wrote: > I don't think documentation or processes are an issue here. Judging from > the bug, this was a very simple case of slow responses.
As an external person, getting reviews is _hard_, especially since I typically poke things that isn't needed by Firefox on a tier 1 platform. My personal "best" was a single-token change that took 8 months (request was to somebody who made a drive-by comment on my previous iteration). I've heard of others who had to get to Mountain View, invite people out to lunch, and ambush them with a review request once they got there, just to get a review done.
Unfortunately, I don't see this changing anytime soon. While a few people make valiant efforts (thanks!), there just isn't a project-wide push to make sure people get on their review requests. At this point, it seems like actually doing reviews is more of a voluntary thing that can be pushed off - and that means that is exactly happens.
> Unfortunately, I don't see this changing anytime soon. While a few > people make valiant efforts (thanks!), there just isn't a project-wide > push to make sure people get on their review requests. At this point, it > seems like actually doing reviews is more of a voluntary thing that can > be pushed off - and that means that is exactly happens.
I think the situation is getting a little better: some individuals are getting better, and in some areas we've got new peers or owners to spread the load.
But I agree that responsiveness is still often unacceptable.
>>> A bug was filed, a patch was posted, but it didn't get in the tree until >>> "too late" (for a not-worst-case-scenario definition of the phrase, but >>> still way too close to it for comfort). Some questions to ponder:
>>> When did we fail to keep things moving along? >>> How do we make the process clearer and more discoverable for new hands? >>> What documentation needs to be modified to help that clarification? >>> How and where should such documentation be found, and where should we >>> position links to it on our sites to make it easy to find? >>> How should responsibility for making sure posted patches get in the tree >>> be split up?
> I'm not sure that we need new or additional process here, to be honest. > It is the responsibility of a bug owner to ensure that it transitions > from state to state to state, and it's the responsibility of a component > owner to ensure that bugs are actively being triaged and moved through > the states and closed out if they're languishing or not getting > attention. My feeling is that the more process we have, the harder > things are to understand, and we have quite enough to understand already.
I heartily, fully, enthusiastically agree. We don't need more process. We just need to be more diligent about following the process we have, and measuring it.
Was the crasher which was reported in 460090 added to an automated test suite when it was reported? Would adding it as soon as it was seen as reproducible (even without fix) caused enough attention to get the fix in sooner? I'm not sure of the testing procedures in mozilla, but I maybe getting such test cases in early on could have helped that the bug got enough attention to get fixed sooner.
On Friday 2009-03-27 12:21 -0400, Boris Zbarsky wrote:
> That already happens, to some extent. There are people watching the > component qa addresses, so they get mail when bugs are filed. For > example, I get mail on new form control, docshell, xbl bugs. I'd look > at more components, but there's no way to look for only new bugs; if I > watched all components I care about new bugs in I'd be getting over 2000 > bugmails per day (yes, I tried this once). I thought we had a bug on > making it easier to watch only new bugs and bugs being moved into a > component, but I can't find it right now... Maybe I should re-file it > just in case. :(
One thing that would probably help a lot is being able to configure separate bugzilla email preferences for each address watched. I'd probably watch a bunch of additional addresses if I could watch only new bugs. (The "added to / removed from" checkbox is probably close enough for catching incoming bugs to a component; the problem is that I can't use that alone for all my bugmail.)
On Friday 2009-03-27 10:59 -0400, Mike Beltzner wrote:
> I'm not sure that we need new or additional process here, to be honest. > It is the responsibility of a bug owner to ensure that it transitions > from state to state to state, and it's the responsibility of a component > owner to ensure that bugs are actively being triaged and moved through > the states and closed out if they're languishing or not getting > attention. My feeling is that the more process we have, the harder things > are to understand, and we have quite enough to understand already.
One thing that Jesse suggested a few weeks ago (in conversation at lunch, I think) is that Bugzilla have a (queryable) notion of who is expected to take the next action in a bug (and what that action is). This way we could see queues of things other than review requests.
Another thought I had is that perhaps we could find tracking this information easier if Bugzilla's "Status" field reflected the actual statuses, e.g., by replacing "ASSIGNED" with "fix under development", "waiting for review", and "waiting for checkin" with an appropriately-defaulted checkbox to update this status as one would expect when editing attachments.
(Personally, I track my patches-in-progress by having them all in a single mercurial queue. If it's not in that queue, I have no plans to commit it.)
On Sat, Mar 28, 2009 at 10:54 PM, L. David Baron <dba...@dbaron.org> wrote:
> One thing that would probably help a lot is being able to configure > separate bugzilla email preferences for each address watched. I'd > probably watch a bunch of additional addresses if I could watch only > new bugs. (The "added to / removed from" checkbox is probably close > enough for catching incoming bugs to a component; the problem is > that I can't use that alone for all my bugmail.)
there's a bug (and plans i believe) for bugmail controls to be query based, however i don't think anyone considered this additional restriction, so it should probably be filed w/ a dependency....
we've had this problem for years. personally, i'd like to point out that r-'ing repeatedly to get tests just right is an obvious part of the failure for this specific bug.
it's one thing for someone who's paid to work on absolutely nothing else to be paid to get tests just right, it's another to take a crasher, have a fix from a contributor (who probably has hundreds of other things to do) and make them do work over an extended period of time before not taking their patch.
we have been leaning toward this latter pole.
we've need people to monitor review requests and help get patches landed for 10 years now. it only gets worse as the number of bugs we have grows.
that said, it turns out that this bug's story is an example of where it's not a good idea to jump at the first suggestion.
fwiw, i'm currently on one of my "bug reporting sprees". but i'm not filing ff bugs, i'm filing bugs against virtualbox, songbird, mercurial, ....- I'm trying to work w/ products where the barrier to filing bugs is lower. In many cases, I could write the fixes too, but I probably won't,... too many bugs to file, too little time.
On Mar 27, 10:59 am, Mike Beltzner <beltz...@mozilla.com> wrote:
> I'm not sure that we need new or additional process here, to be > honest. It is the responsibility of a bug owner to ensure that it [...] > To me it feels like what we need are more people interested in this > administration of our process, doing the work of walking through > buglists and making sure.
I think beltzner pretty well nailed it. If you know our process, and if people follow our process, the process is sufficient. The documentation on the process ( https://developer.mozilla.org/en/Getting_your_patch_in_the_tree ) feels adequate; it could probably be improved a little bit, but I think it nails the high points pretty well, including the responsibility of the patch author to periodically pester us for review or checkin and to supply a checkin-ready patch and add checkin- needed. Maybe we need to raise the visibility of that documentation, particularly in Bugzilla? Perhaps linking it at the bottom of the attachments table? There’s a link to it already on the patch submission page, but since you don’t use that page while the patch is waiting for r/sr, or “waiting in limbo” after sr+, that process documentation ends up not being terribly visible.
That said, Mozilla's a large enough project where things can fall through the cracks, and so we need some "Unloved Patches Ombudsmen" to help ensure that patches do get moved along and patch authors don't get discouraged. On a very small scale, we've done this sort of thing to nearly eliminate the "lost patches" problem in Camino, whereas a couple of years ago the problem was affecting 50% of the patches, even from our own dev team. I took on the role of patch shepherd, and I had two queries to help me: one for bugs with non-obsolete patches where there had been no bug activity in over a month, and one for bugs with non-obsolete patches that had r+ and sr+. I'd run the former query occasionally and poke bugs, but I'd go through the latter query on a weekly basis and set what eventually became checkin-needed and then pester our folks with commit rights. Nowadays I do checkins, so we've pretty much moved the patch bottleneck back into the review process itself ;-) (q.v. various Mozilla discussions about old review requests). The queries weren't perfect, so some better patch+flag query tools in Bugzilla would help, but assuming a sufficient number of potential ombudspersons, even this semi-manual method would be feasible for major project areas.
Better tooling for locating unloved patches might help module and component owners keep track of these themselves, but, if as Boris says, people don’t use the existing tools to check on their review queues, tools alone won’t solve this problem. Neither will one-time events (like Gerv’s valiant “old review requests” quest from a couple of years ago), which is why I really think we need some people whose job description specifically includes spending time as "Unloved Patches Ombudsmen." It would be great if Mozilla Corp could find some not-already-overworked employees ;-) and include in their job descriptions spending time in this ombudsman role for Gecko/Toolkit and Firefox components. (Moz QA seems to do a good job tracking fixed bugs, verifying them, and making sure things are in the testsuites; since QA is also working through bugs on triage duty, perhaps they're good candidates for this role, too?)
Once a week each ombudsman would run the queries in the components/ modules for which he’s responsible and poke the appropriate people depending on the status of the bug (reviewer/superreviewer, patch author, checkin-needed pool), and so forth. If there are too many components and too few people, even cycling through each ombudsman’s assigned components on a staggered basis so that all components are checked for "unloved patches" just once a month would be a massive improvement, I’d think.
Regular ombudsmen sweeps could ultimately help solve both the specific problem here, where we had a good patch that never made it into the tree, and the larger problem we have with getting patches through the entire process, where getting reviews is often the blockage instead. (I also wonder if it might not improve the upstreaming experience for Mozilla-based projects; their bugs typically won’t be P1 Firefox blockers, so having the ombudsmen interact with the bugs periodically would keep those fixes from being completely forgotten.)
> Neither will one-time > events (like Gerv’s valiant “old review requests” quest from a couple > of years ago),
That wasn't supposed to be a one-time event. The idea was that we would clean out the backlog by either invalidating or reviewing them, and then component/module owners could be whined at on a regular basis about new bugs which seemed to be unloved.
This project has, sadly, like many other things I'd like to do, stalled through lack of time (although another community member has done some valuable work on it, it's sort of blocked on me, and a simple bugzilla.mozilla.org change I can't seem to get any love for).
> Once a week each ombudsman would run the queries in the components/ > modules for which he’s responsible and poke the appropriate people > depending on the status of the bug (reviewer/superreviewer, patch > author, checkin-needed pool), and so forth.
You see, it's exactly that which could be automated.
> naming nits, or even fixing some minor remaining bugs. (To say > nothing of then having to spend more time in the horrible bugzilla > patch-editing experience!).
I'm not quite sure what you mean by this. I didn't realise you could edit patches directly on Bugzilla. Anyway, if there's something which isn't pleasant, is there a bug filed on making it more so? :-)
> Another thought I had is that perhaps we could find tracking this > information easier if Bugzilla's "Status" field reflected the actual > statuses, e.g., by replacing "ASSIGNED" with "fix under > development", "waiting for review", and "waiting for checkin" with > an appropriately-defaulted checkbox to update this status as one > would expect when editing attachments.
But are these "states" not trackable at the moment by querying for e.g. set review flags? Having a separate state for "waiting for review" would mean a lot more toggling of state, and we'd have a DRY problem as well.
> > --- Comment #23 from Jeff Walden <jwalden+...@mit.edu> 2009-03-26 22:47:37 PDT --- > > (In reply to comment #22)
> > Passive-aggressive insinuations of bad faith generally don't win friends and > > influence people.
> I am annoyed, but really at myself rather than any other individual. > As I have proved capable of not getting getting fixes committed in > multiple projects, I'd be interested in anything you could point out > that I should have done differently, so I can actually do better in > future, in Mozilla and elsewhere.
One rule from the old days, possibly not known now, is to mail st...@mozilla.org if you are getting no response, or insuffucient or (in your view) wrong response, from a module owner. Please feel free to mail me in any case.
In the modern era, st...@mozilla.org has withered away, but now we have a module ownership module:
You'll have to map the names listed there to emails by hand (I guess we don't want to make it too easy for spammers). Again, please mail me in any event.
On 27 Mrz., 09:28, Robert O'Callahan <rob...@ocallahan.org> wrote:
> I don't think documentation or processes are an issue here. Judging from > the bug, this was a very simple case of slow responses.
But why were responses slow, given the severity of the bug? I think because the severity was not appreciated. Free Memory Read when calling through a vtable pointer is exploitable. This may not be fully appreciated by all committers and drivers.
> On 3/27/09 9:22 AM, Mike Shaver wrote: >> I feel pretty strongly that we should be doing more "committed with >> some tweaks to whitespace and tests, thanks!" and less "bounced until >> you make this round of minor changes, please try again later".
> I do some of that, and I've been told, I suspect correctly, that I'm not > improving things in the long run by doing so.
I understand why this can be seen as not improving things in the long run. But, if we just look at the whitespace/unexpanded tabs stuff it's a trivial thing and one that could be checked with a tool (it might even be fixed automatically with a tool) - Seems a waste of time to do by hand, could be something that is done automatically when you try to attach the fix, or when you request a review on it. It must be annoying for reviewers to have to check for this too.
> If our goal when someone we've never seen before attaches a patch is to > get that code in the tree, then there's no doubt that having "a Mozilla > developer," the reviewer (in the post-SR world, where that actually > works) or someone else, just de-nit and push is the way to go. > If our goal is to turn the person who attached the patch in the first > place into a Mozilla developer, I'm less sure it is. I know the feeling > that "I had to fight about whitespace, and make some changes that I > don't think made sense, and beg for a checkin, but I got *my code* in > Firefox" produces; what feeling does "I attached a patch that someone > else took, assigned the bug to themselves, changed around, and checked > in" produce? Would we really be better off treating patchers as > interchangeable disposable idea-factories, rather than treating them as > the person we're going to train to be the next peer for the module?
I know that I would prefer this to having "my code" modified and then checked in (unless it was modified by a tool)
But could we work with "conditional approved", so if e.g. there is a typo in a comment, then you would be told that, but would not need another review, just you would promise that you fix that issue and not change anything else?
-- /////////////////////////////////////////////////////////// // Documentation is written for humans, not for machines // ///////////////////////////////////////////////////////////
> Mike Kristoffersen wrote: >> - Now, any active bug must at all times have someone or a group that >> is responsible for it (by active I mean, a bug that is not fixed and >> in the tree or one that has been closed for another reason - like >> "working as intended")
> Yes, indeed. The problem with this is that our current ratio of bugs to > people is on the order of hundreds. That's far past what one can > actually keep track of usefully... We could use group "nobody" > placeholders like we do now to keep track of bugs, but those aren't > going to do anything about bugs.
If there are no-one responsible, then how can we ensure that we focus our attention on the most important things?
We might have someone having implicit responsibility for all of the bugs today, but lets make it explicit. It is true that there are much more bugs than we have people - this is good, as we won't get bored :) - no... to be more serious, if we were to use a "nobody" place holder we risk that:
1) too many people spend time keeping an eye on the list
or
2) too few people spend time keeping an eye on the list
Both are bad for their own reason (first one is duplicate work, second one that something is missed).
I don't think the average developer should be responsible for more than a few bugs at any one time, but the backlog (the bugs with no active fixing/review going on) should be assigned to someone (person/group).
The one assigned to this backlog could have the responsibility to make a prioritized list of what is most important to be fixed at any one time - when a developer want to take on another bug, he would then know what is most important to look at, or what is most important of the things that he has the ability / interest to look at. Depending on the relationship between Mozilla and the developer it could also be a more direct thing, where he is told what to look at.
The backlog owner should also be able to push something very important into development, if it is better for the project to have a specific developer look at something specific compared to what else he is doing at the moment.
The backlog owner position could be a pure administrative one (e.g. where the owner didn't do any development himself, that would depend on the workload and what else the person is assigned to taking care of)
We could also think about putting reviews into this backlog - going with my initial post to this thread, how do we level out the workload for doing reviews, and how do we avoid developers being disturbed doing reviews, if it is more important for them to work on new development? No-one can benefit from the way it is now where you need to "bribe" reviewers with lunches to have them look at a change, as someone else wrote?
Currently it might be the ones with the best personal relationships or the ones that are most pushy that gets their changes reviewed first, we should be able to trust the system - if you as a developer request a review you should have confidence that the review will happen, it should be the exception that you would need to push it trough.
Another way to look at it: If I request a review, who is failing to do their job if the review is not done? Is it me, because I didn't push, or the reviewer who ignored the initial request?
I know we are humans, and I'm not talking about the case where it's a human error that meant the review didn't happen. But again, why should I remind a reviewer that might have "forgotten" the request, when this can be automated, sending a reminder mail ones a week to people - e.g:
----
Hi this is bugzilla with your weekly mail to remind you that you have:
1 active bug(s) -bug xxx 2 pending review(s): -bug x111, 2 days old -bug x222, 5 weeks old
---
Yes maybe we should all actively search bugzilla for stuff assigned to us, but why, if we can get it pushed to our inbox - I bet someone could even make a small desktop app that would provide this information "live".
I already made my point with trigger mails to project owners, so I won't repeat it here :)
>> - When you get assigned to a bug (for validation, testing, reviewing, >> fixing, committing, what ever reason) you should receive an e-mail. >> (Where you at a glance, from the title of the e-mail, can see that it >> is about something you (or a group you are a member of) are getting >> assigned to)
> For what it's worth, a review requestee already gets mail, and there is > a Bugzilla page where you can see all your pending review requests. Some > people look at this page frequently and do reviews. Some don't. Tooling > changes won't change this. :(
I didn't review anything yet, so I can't comment on the format of these, but as far as remember, I can't see from the title line of the mails from bugzilla, if it's about a bug I'm assigned to, or one I get because I'm on the CC list of the bug, or because I'm watching someone who is getting info about it....
The worst might be, that if you attach a new fix to a bug asking for a review on it, canceling a previous review request in the process - the title just says "review request canceled" or something like that (it might be another mail the reviewer get), making it easy to miss that the body of the mail actually contains a new request.
I believe that having a mail pushed to you with information will actually increase the odds that the information is read, compared to having to pull the information from bugzilla.
>> - When a bug that you are assigned to is opened, you could get a >> description that explains what you are supposed to do, this text being >> defined by the state the bug is in, "Verify the bug", "Review attached >> patch", etc.
> This could be good for novice contributors; it won't help reviewers > shirking their duties: they already know what they're supposed to do > with review requests!
Yep, that is true, and no, it will be the exact same amount of work that needs to be done - still think that it would be nice having the process inline documented compared to having it in a seperate doc, especially in mozilla where you never really know if a page that you are looking at is maintained and up to date, or if it has been replaced by something else, or what more often seems to be the case, that it is almost correct with a few exceptions that you have to guess :)
<SNIP>
>> -It is the same if you report an issue, you are only expected to give >> a useful description of the issue. In my view, the reporter should not >> push the fixing, by any other means than perhaps setting the initial >> severity of the bug - it is the responsibility of the project, or >> owner of the area to do the prioritization with respect to the other >> tasks at hand.
> Though of course the reporter is welcome (but not required) to do more, > and some do.
Yes, of cause we all have a duty to act if we know that something is being missed or not moving along in an ideal way. That goes if it is because we have been actively involved in, or we happen to find out about it by "accident".
>> But the automation could also continue into choosing who to assign >> bugs to at any given state - so if I create a new bug, it could >> automatically get assigned to the group (or individual) responsible >> for the area where the bug is reported in (with the e-mail notification)
> That already happens, to some extent. There are people watching the > component qa addresses, so they get mail when bugs are filed. For > example, I get mail on new form control, docshell, xbl bugs. I'd look at > more components, but there's no way to look for only new bugs; if I > watched all components I care about new bugs in I'd be getting over 2000 > bugmails per day (yes, I tried this once). I thought we had a bug on > making it easier to watch only new bugs and bugs being moved into a > component, but I can't find it right now... Maybe I should re-file it > just in case. :(
Sounds like we need some more clever filtering options yes :)
^__^ Mike
-- /////////////////////////////////////////////////////////// // Documentation is written for humans, not for machines // ///////////////////////////////////////////////////////////
