We discussed this at the last Gecko and Firefox meetings - but I wanted to get some notes on the plan for scheduling of future milestones down in print.
Here's the context we are using to evaluate scheduling of milestones:
1) We are driven by quality, not time. We want to Firefox 3 to be something that we are all proud of. This means features that delight users and the same or higher quality than previous releases. "Quality" includes performance (Tp/Ts/TDHTML/etc), footprint, web compatibility, regressions, and general fit and finish. Having said that, we want to move the web forward and are in a competitive market. So we should converge on a release as fast as possible.
2) There has been almost 2 years of development on the 1.9 platform incorporating major changes: Reflow, Cairo, Cycle Collector, Native Mac Widgets, contenteditable, many parts of the Web Apps 1.0 Spec, etc. We need to have enough "bake time", public milestones, and regression fix time to ensure we meet our quality goal. We should also endeavor to get this functionality into the hands of users and web developers as soon as possible. The sooner we ship this the sooner web authors can count on >15% of their users supporting the latest capabilities and standards.
3) The Firefox front-end has had significantly less development time than the platform and has yet to have the opportunity to innovate on top of infrastructure built for places, password manager, and others. So we'd like to give them until M8 to continue to develop user-visible features on top of the core infrastructure.
4) A milestone schedule with a release every 6 weeks (4 weeks till code freeze from last milestone, 2 weeks of stabilization/build work) seems to work the best. Note that actual tree closures will in practice likely be shorter than 2 weeks if there are not multiple re-spins.
Based on this context the proposed schedule is:
* M7: Freeze on July 25 * Platform feature freeze * This is the "web developer preview release" since it is platform complete. This will be marketed at a higher volume than other alphas to help get wider-scale testing. * M8: Freeze on Sept 5 * Firefox feature freeze * M9: Freeze on Oct 16 * M*: Ongoing as needed
Feature Freeze = all planned features are implemented and exposed (through APIs and user interface elements) in ways that are usable, but not necessarily polished. After freeze, landings will be restricted to regressions (from 1.8), performance and footprint fixes, as well as additional functional or unit test coverage and changes to APIs and user interface elements based on feedback from the beta cycle.
In order to hit our goals above we are going to do the following:
1) Only explicitly named platform features are available for landing before M7 (with exceptions heard by the release drivers). At the time of this writing the only platform features remaining to land before M7 that I'm aware of are Anti-malware, Secure wrappers, and some Offline work. This means if you are working on a platform feature for 1.9 that's not on this list you should help close out the long blocker list.
2) The trunk will go under release driver control after M7. This means all check-ins will require release driver approval after July 25. Release drivers currently include MConnor, CBeard, Betlzner, Basil, Schrep, Damon, Vlad. Additional volunteers welcomed :-). As always these folks will do frequent triage and will rely heavily on the judgment and assistance of module owners and experts in each major functional area.
3) We'll switch from Alphas to Betas as soon as we believe Firefox is stable and usable enough for daily browsing for a large number of people. Until we hit this criteria we'll continue to release Alphas on the 6 week cadence above. Criteria: a) Footprint at or below that of 1.8. This is being measured regularly through Talos working set size (http://tinyurl.com/252ka3) and through informal dogfooding. b) Most sites should display properly and regression free (from previous major release) c) No known common dataloss bugs d) No common hangs or crashes e) No problems with major features in common use cases
"Common" is defined as usage of the browser with any popular websites or frequent occurrence in daily browsing for our dogfood or beta population. We'll measure this through frequency of bug reports and direct feedback from users.
Based on this criteria it does not appear that M7 will be ready to be called a beta. Talos is showing a ~18% increase in Footprint and informal dogfooding confirms things are currently worse on the trunk. Search for keyword mlk in bugzilla to find plenty of known bugs here.
4) We'll release betas until we complete our regression work and incorporate feedback from wider-scale testing. Before we release the final beta Performance (specifically Ts, Tp, Tdhtml, Txul, and any other benchmarks we add to the main tinderboxes) will be as good or better than 1.8. We should strive for improved Tp and Tdhtml scores performance v.s. 1.8.
5) After the last beta we'll release a Release Candidate. The Release Candidate is meant to be bit-for-bit the final release. Only new problems found after the RC is released will cause additional RC's to be published. Once we are confident there are no new issues we'll release the final release.
So in summary:
* Can I land platform feature or old bug fix X? * In general no, but read above carefully * When will Beta 1 Ship? * As soon as it is ready (see #3 above) * When is the next Milestone? * 6 weeks from the last one. * When will the last Beta ship? * As soon as it is ready (see #4 above) * What can I do to help? * Platform folks let's sprint to the finish. Footprint, performance, regressions, unit tests! Everyone involved wants to get a beta into people's hands asap. We could also use your help getting the blocker lists managed. If it doesn’t fit that criteria please minus it. * Firefox - you've got a little bit of time left to crank. Delight us! * Everyone else - plenty of help needed reproducing, filing, and confirming bugs. Dogfood. Run the nightly tester tools + leak gauge, help us hammer this thing into shape.
Simon Paquet wrote: > And on the seventh day Mike Schroepfer spoke:
>> Questions or Thoughts?
> Is a M-release the same as a alpha release?
> If yes, could you please call it an alpha as before, because the > M-releases are still widely associated by people with releases before > Mozilla 0.6.
They're not the same. It is not clear at this time whether those releases will be alphas or betas beyond M7, so calling them alphas (i.e. alpha 8/9) is possibly inaccurate/misleading. I think there's very little likelihood of confusion here. I suggested, for the purpose of scheduling, to resurrect the M* numbering convention, though we will publicly use alpha or beta versioning as appropriate.
1) Where do wanted-1.9 bugs fit into this setup? Especially regressions since 1.8?
2) Where do long-standing patches that have been waiting on reviews for months that are neither blocking1.9 nor marked wanted-1.9 fit in at this point?
3) Does "platform features" (the things that should no longer be worked on) include platform bug and regression fixes that are not blockers, or only new functionality?
4) How does one request approval for patches?
I assume the answers to the above are:
1) OK to land before M7, need approval after M7 like everything else. The notation is meaningless in terms of release scheduling, and only there to indicate to people who want something to do what to work on (outside the blockers). 2) As #1, unless they're feature additions. 3) New functionality. 4) We'll add a flag before M7 ships.
> 1) Where do wanted-1.9 bugs fit into this setup? Especially regressions
> since 1.8? > OK to land before M7, need approval after M7 like everything else. The > notation is meaningless in terms of release scheduling, and only there > to indicate to people who want something to do what to work on (outside > the blockers).
That's correct. But more generally regressions fixes since 1.8 are encouraged and welcomed throughout the schedule.
> 2) Where do long-standing patches that have been waiting on reviews > for months that are neither blocking1.9 nor marked wanted-1.9 fit in > at this point? > 2) As #1, unless they're feature additions.
Correct
> 3) Does "platform features" (the things that should no longer be worked > on) include platform bug and regression fixes that are not blockers, > or only new functionality? > 3) New functionality.
Correct. We are trying to start reducing the total amount of code churn by closing the gate to new stuff and focusing on regression fixes.
To recap what we talked about in the Firefox meeting yesterday and to broadcast this to .l10n:
We'll push back the string freeze along with additional milestones. We know that that's unfortunate as it's making it harder to plan for the hot localization phase, but we don't have any other realistic choice. We don't intend to change the amount of time we plan for localization, but we may have to shift that time window.
We will start to require l10n-swags (*) by the time we require per-patch approvals in general, that should be after the Firefox feature freeze, IIRC.
Questions?
Axel
(*) l10n-swag is the number of lines added to the localization files, excluding comments, of course. That's not supposed to be a precise number, but 1, 10, or 100 is important to know.
Mike Schroepfer wrote: > We discussed this at the last Gecko and Firefox meetings - but I wanted > to get some notes on the plan for scheduling of future milestones down > in print.
> Here's the context we are using to evaluate scheduling of milestones:
> 1) We are driven by quality, not time. We want to Firefox 3 to be > something that we are all proud of. This means features that delight > users and the same or higher quality than previous releases. "Quality" > includes performance (Tp/Ts/TDHTML/etc), footprint, web compatibility, > regressions, and general fit and finish. Having said that, we want to > move the web forward and are in a competitive market. So we should > converge on a release as fast as possible.
> 2) There has been almost 2 years of development on the 1.9 platform > incorporating major changes: Reflow, Cairo, Cycle Collector, Native Mac > Widgets, contenteditable, many parts of the Web Apps 1.0 Spec, etc. We > need to have enough "bake time", public milestones, and regression fix > time to ensure we meet our quality goal. We should also endeavor to get > this functionality into the hands of users and web developers as soon as > possible. The sooner we ship this the sooner web authors can count on > >15% of their users supporting the latest capabilities and standards.
> 3) The Firefox front-end has had significantly less development time > than the platform and has yet to have the opportunity to innovate on top > of infrastructure built for places, password manager, and others. So > we'd like to give them until M8 to continue to develop user-visible > features on top of the core infrastructure.
> 4) A milestone schedule with a release every 6 weeks (4 weeks till code > freeze from last milestone, 2 weeks of stabilization/build work) seems > to work the best. Note that actual tree closures will in practice > likely be shorter than 2 weeks if there are not multiple re-spins.
> Based on this context the proposed schedule is:
> * M7: Freeze on July 25 > * Platform feature freeze > * This is the "web developer preview release" since it is > platform complete. This will be marketed at a higher volume > than other alphas to help get wider-scale testing. > * M8: Freeze on Sept 5 > * Firefox feature freeze > * M9: Freeze on Oct 16 > * M*: Ongoing as needed
> Feature Freeze = all planned features are implemented and exposed > (through APIs and user interface elements) in ways that are usable, but > not necessarily polished. After freeze, landings will be restricted to > regressions (from 1.8), performance and footprint fixes, as well as > additional functional or unit test coverage and changes to APIs and user > interface elements based on feedback from the beta cycle.
> In order to hit our goals above we are going to do the following:
> 1) Only explicitly named platform features are available for landing > before M7 (with exceptions heard by the release drivers). At the time > of this writing the only platform features remaining to land before M7 > that I'm aware of are Anti-malware, Secure wrappers, and some Offline > work. This means if you are working on a platform feature for 1.9 > that's not on this list you should help close out the long blocker list.
> 2) The trunk will go under release driver control after M7. This means > all check-ins will require release driver approval after July 25. > Release drivers currently include MConnor, CBeard, Betlzner, Basil, > Schrep, Damon, Vlad. Additional volunteers welcomed :-). As always > these folks will do frequent triage and will rely heavily on the > judgment and assistance of module owners and experts in each major > functional area.
> 3) We'll switch from Alphas to Betas as soon as we believe Firefox is > stable and usable enough for daily browsing for a large number of > people. Until we hit this criteria we'll continue to release Alphas on > the 6 week cadence above. Criteria: > a) Footprint at or below that of 1.8. This is being measured > regularly through Talos working set size (http://tinyurl.com/252ka3) and > through informal dogfooding. > b) Most sites should display properly and regression free (from > previous major release) > c) No known common dataloss bugs > d) No common hangs or crashes > e) No problems with major features in common use cases
> "Common" is defined as usage of the browser with any popular > websites or frequent occurrence in daily browsing for our dogfood or > beta population. We'll measure this through frequency of bug reports > and direct feedback from users.
> Based on this criteria it does not appear that M7 will be ready to > be called a beta. Talos is showing a ~18% increase in Footprint and > informal dogfooding confirms things are currently worse on the trunk. > Search for keyword mlk in bugzilla to find plenty of known bugs here.
> 4) We'll release betas until we complete our regression work and > incorporate feedback from wider-scale testing. Before we release the > final beta Performance (specifically Ts, Tp, Tdhtml, Txul, and any other > benchmarks we add to the main tinderboxes) will be as good or better > than 1.8. We should strive for improved Tp and Tdhtml scores > performance v.s. 1.8.
> 5) After the last beta we'll release a Release Candidate. The Release > Candidate is meant to be bit-for-bit the final release. Only new > problems found after the RC is released will cause additional RC's to be > published. Once we are confident there are no new issues we'll release > the final release.
> So in summary:
> * Can I land platform feature or old bug fix X? > * In general no, but read above carefully > * When will Beta 1 Ship? > * As soon as it is ready (see #3 above) > * When is the next Milestone? > * 6 weeks from the last one. > * When will the last Beta ship? > * As soon as it is ready (see #4 above) > * What can I do to help? > * Platform folks let's sprint to the finish. Footprint, > performance, regressions, unit tests! Everyone involved wants to get a > beta into people's hands asap. We could also use your help getting the > blocker lists managed. If it doesn’t fit that criteria please minus it. > * Firefox - you've got a little bit of time left to crank. Delight us! > * Everyone else - plenty of help needed reproducing, filing, and > confirming bugs. Dogfood. Run the nightly tester tools + leak gauge, > help us hammer this thing into shape.
Mike Connor wrote: > Simon Paquet wrote: >> And on the seventh day Mike Schroepfer spoke:
>>> Questions or Thoughts?
>> Is a M-release the same as a alpha release? >> If yes, could you please call it an alpha as before, because the >> M-releases are still widely associated by people with releases before >> Mozilla 0.6.
> They're not the same. It is not clear at this time whether those > releases will be alphas or betas beyond M7, so calling them alphas (i.e. > alpha 8/9) is possibly inaccurate/misleading. I think there's very > little likelihood of confusion here. I suggested, for the purpose of > scheduling, to resurrect the M* numbering convention, though we will > publicly use alpha or beta versioning as appropriate.
Can we use lower-case 'm's, then? The early M* scheme used capital Ms. This is also consistent with how we use lower-case for alphas and betas. An even clearer abbreviation would be e.g. 1.9m7.
On Jul 8, 9:37 am, Mike Schroepfer <sch...@mozilla.com> wrote:
> 2) The trunk will go under release driver control after M7. This means > all check-ins will require release driver approval after July 25. > Release drivers currently include MConnor, CBeard, Betlzner, Basil, > Schrep, Damon, Vlad. Additional volunteers welcomed :-). As always > these folks will do frequent triage and will rely heavily on the > judgment and assistance of module owners and experts in each major > functional area.
Going under release driver control for non-blockers at M7 seems like a good step, but I don't think that should require approval for bugs already marked blocking+. Requiring approval for blockers will slow down their rate of fix and they've already gotten one level of approval to be blocking+. I wouldn't start throttling blocker bugs until much closer to shipping -- M9?
stua...@gmail.com wrote: > On Jul 8, 9:37 am, Mike Schroepfer <sch...@mozilla.com> wrote:
>> 2) The trunk will go under release driver control after M7. This means >> all check-ins will require release driver approval after July 25. >> Release drivers currently include MConnor, CBeard, Betlzner, Basil, >> Schrep, Damon, Vlad. Additional volunteers welcomed :-). As always >> these folks will do frequent triage and will rely heavily on the >> judgment and assistance of module owners and experts in each major >> functional area.
> Going under release driver control for non-blockers at M7 seems like a > good step, but I don't think that should require approval for bugs > already marked blocking+. Requiring approval for blockers will slow > down their rate of fix and they've already gotten one level of > approval to be blocking+. I wouldn't start throttling blocker bugs > until much closer to shipping -- M9?
Agreed. I thought the plan was to do just that, though it wasn't stated here.
We'll need to post a plan for approvals going forward, since we're not going to require approvals for the front end until after M8, given how much work is going to be going on with rapid iteration.
fantasai wrote: > Mike Connor wrote: >> Simon Paquet wrote: >>> And on the seventh day Mike Schroepfer spoke:
>>>> Questions or Thoughts?
>>> Is a M-release the same as a alpha release? >>> If yes, could you please call it an alpha as before, because the >>> M-releases are still widely associated by people with releases before >>> Mozilla 0.6.
>> They're not the same. It is not clear at this time whether those >> releases will be alphas or betas beyond M7, so calling them alphas >> (i.e. alpha 8/9) is possibly inaccurate/misleading. I think there's >> very little likelihood of confusion here. I suggested, for the >> purpose of scheduling, to resurrect the M* numbering convention, >> though we will publicly use alpha or beta versioning as appropriate.
> Can we use lower-case 'm's, then? The early M* scheme used capital Ms. > This is also consistent with how we use lower-case for alphas and betas. > An even clearer abbreviation would be e.g. 1.9m7.
