Firefox 4 Beta 7: Road to completion, b7relbranch vs. trunk
Mike Beltzner
Progress continues on the beta 7 blocker list:
https://bugzilla.mozilla.org/buglist.cgi?quicksearch=blocking2:7
Of those 11 bugs, 2 are fixed on tracemonkey, 2 are awaiting checkin, 3 are tracking a single issue (JSD & Firebug), and 2 are tracking items. That leaves only two items (a common crash in @ JSString::flatten() and partial incompatibility with the JetPack SDK) to be resolved, and progress is being made. Any help on those remaining two issues would be greatly appreciated.
Once we're done with these blockers, we will be in a position to ship Firefox 4 Beta 7. At that point, we will need to decide whether or not to create builds from the Beta 7 "relbranch" we created on October 6th, as opposed to simply shipping a release based off of mozilla-central.
// The Beta 7 Relbranch
On October 6th we created the GECKO20b7pre_20101006_RELBRANCH and reduced the checkin restrictions on mozilla-central in order to allow progress to continue on blocker work. At that time we were waiting for the JS "compartments" changes to land, and believed that:
- the only changes to be landed on the beta 7 relbranch would be related to these JS changes,
- nightly testing on mozilla-central could act as a proxy for those JS changes
In fact, the JS "compartments" changes have yet to land on this relbranch (though some of the regression fixes required by that work have landed there!). Some additional items have landed, mostly string changes in preparation for string freeze, as well as some small UI tweaks (the new progress indicators) and API changes.
Prior to branch, while the tree was closed, the QA team tested the changes from the 400+ blockers fixed to that point, and declared the nightly builds stable enough for release. If we wanted to ship off the Beta 7 relbranch, we would need to:
- merge the JS "compartment" changes and regression fixes,
- verify that all b7 blockers had landed in that codebase,
- generate builds, run some basic functional tests (or full smoketests if QA decides them to be necessary) and issue a release.
// Trunk (mozilla-central)
Since the October 6th branch point, fixes for over 300 beta8+, betaN+ and final+ blocker bugs have landed on mozilla-central:
These changes have not gone through the same rigorous QA testing, and would need additional time to stabilize prior to release. Additionally, the stability metrics for mozilla-central are still slightly worse than they were before the branch point. The majority of this instability is believed, however, to be due to the JS compartments changes, and so we'd expect to see it in builds generated from either mozilla-central or the Beta 7 relbranch.
If we wanted to ship off mozilla-central, we would need to:
- check recent changes for regressions and newly introduced instability,
- back out or resolve outstanding work to stabilize the trunk,
- generate builds (numbered 4.0b7, trunk would remain 4.0b8pre) run full QA smoketests and issue a release
// Decision time!
There are pros and cons to building Beta 7 from either source, but the key issue is to resolve the outstanding Beta 7 blockers. JagerMonkey is incomplete without the JS compartments work, and so our next milestone must include that set of changes. We can discuss the merits of the approaches at tomorrow's development meeting, taking place at 11am PT. At this point my recommendation is still to ship from the relbranch in order to limit the amount of code change that goes out in a milestone, but I can see rationale on either side.
Looking forward to a discussion tomorrow.
cheers,
mike
John O'Duinn
Added one missing step, but otherwise, that looks like an accurate summary.
tc
John.
=====
On 10/25/10 4:04 PM, Mike Beltzner wrote:
> Hi,
>
> Progress continues on the beta 7 blocker list:
>
> https://bugzilla.mozilla.org/buglist.cgi?quicksearch=blocking2:7
>
> Of those 11 bugs, 2 are fixed on tracemonkey, 2 are awaiting checkin, 3 are tracking a single issue (JSD & Firebug), and 2 are tracking items. That leaves only two items (a common crash in @ JSString::flatten() and partial incompatibility with the JetPack SDK) to be resolved, and progress is being made. Any help on those remaining two issues would be greatly appreciated.
>
> Once we're done with these blockers, we will be in a position to ship Firefox 4 Beta 7. At that point, we will need to decide whether or not to create builds from the Beta 7 "relbranch" we created on October 6th, as opposed to simply shipping a release based off of mozilla-central.
>
> // The Beta 7 Relbranch
>
> On October 6th we created the GECKO20b7pre_20101006_RELBRANCH and reduced the checkin restrictions on mozilla-central in order to allow progress to continue on blocker work. At that time we were waiting for the JS "compartments" changes to land, and believed that:
>
> - the only changes to be landed on the beta 7 relbranch would be related to these JS changes,
> - nightly testing on mozilla-central could act as a proxy for those JS changes
>
> In fact, the JS "compartments" changes have yet to land on this relbranch (though some of the regression fixes required by that work have landed there!). Some additional items have landed, mostly string changes in preparation for string freeze, as well as some small UI tweaks (the new progress indicators) and API changes.
>
> Prior to branch, while the tree was closed, the QA team tested the changes from the 400+ blockers fixed to that point, and declared the nightly builds stable enough for release. If we wanted to ship off the Beta 7 relbranch, we would need to:
>
> - merge the JS "compartment" changes and regression fixes,
> - verify that all b7 blockers had landed in that codebase,
- verify that all the b7 blockers work in isolation from the non-b7 code
in tracemonkey and mozilla-central before attempting branded release
builds.
-- Do this by landing all changes from relbranch onto newly empty birch
branch, and trigger usual set of builds, unittest, talos. If all
builds/unittest/talos have problems, file new blockers.
> - generate builds, run some basic functional tests (or full smoketests if QA decides them to be necessary) and issue a release.
>
> // Trunk (mozilla-central)
>
> Since the October 6th branch point, fixes for over 300 beta8+, betaN+ and final+ blocker bugs have landed on mozilla-central:
>
> http://is.gd/giRLI
>
> These changes have not gone through the same rigorous QA testing, and would need additional time to stabilize prior to release. Additionally, the stability metrics for mozilla-central are still slightly worse than they were before the branch point. The majority of this instability is believed, however, to be due to the JS compartments changes, and so we'd expect to see it in builds generated from either mozilla-central or the Beta 7 relbranch.
>
> If we wanted to ship off mozilla-central, we would need to:
>
> - check recent changes for regressions and newly introduced instability,
> - back out or resolve outstanding work to stabilize the trunk,
> - generate builds (numbered 4.0b7, trunk would remain 4.0b8pre) run full QA smoketests and issue a release
>
> // Decision time!
>
> There are pros and cons to building Beta 7 from either source, but the key issue is to resolve the outstanding Beta 7 blockers. JagerMonkey is incomplete without the JS compartments work, and so our next milestone must include that set of changes. We can discuss the merits of the approaches at tomorrow's development meeting, taking place at 11am PT. At this point my recommendation is still to ship from the relbranch in order to limit the amount of code change that goes out in a milestone, but I can see rationale on either side.
>
> Looking forward to a discussion tomorrow.
>
> cheers,
> mike
> _______________________________________________
> dev-planning mailing list
> dev-pl...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-planning
Gavin Sharp
>If we wanted to ship off the Beta 7 relbranch, we would need to:
>
> - merge the JS "compartment" changes and regression fixes,
> - verify that all b7 blockers had landed in that codebase,
I think we'd definitely need to do more than just basic functional
tests, given the risk introduced by the merging (there are many
changes outside of js/src, and outside of Gecko, that were made on
trunk in response to compartments changes, so it isn't just a matter
of merging in tracemonkey wholesale). The merging applies to
non-compartments work as well (for example, for bug 598600). Add to
that the fact that there will have been no nightly testing on the
result of the merge prior to shipping, and the cost of doing that
merging and verifying that all blockers have landed, and I think the
costs of this approach far outweigh the benefit of the QA checks that
were done at the branch point.
> Since the October 6th branch point, fixes for over 300 beta8+, betaN+ and final+ blocker bugs have landed on mozilla-central:
>
> http://is.gd/giRLI
>
> These changes have not gone through the same rigorous QA testing
Neither has the result of the merge, though. At least these changes
have gone through continuous nightly testing, automated testing, and
have "settled" for much longer than the result of the merge will have.
Your list of tradeoffs also doesn't include the fact that shipping b7
from the relbranch would be delaying getting those 300 fixes to the
beta audience. I don't think shipping from the relbranch would get us
a more "stable" result, but even if it did, the feedback we'd receive
wouldn't be as useful as feedback from more current builds.
Gavin
Kyle Huey
> > Prior to branch, while the tree was closed, the QA team tested the
> changes from the 400+ blockers fixed to that point, and declared the nightly
> builds stable enough for release. If we wanted to ship off the Beta 7
> relbranch, we would need to:
> >
> > - merge the JS "compartment" changes and regression fixes,
> > - verify that all b7 blockers had landed in that codebase,
>
> - verify that all the b7 blockers work in isolation from the non-b7 code
> in tracemonkey and mozilla-central before attempting branded release
> builds.
> -- Do this by landing all changes from relbranch onto newly empty birch
> branch, and trigger usual set of builds, unittest, talos. If all
> builds/unittest/talos have problems, file new blockers.
>
IMO, if shipping off the b7 relbranch is even an option after the platform
meeting tomorrow this needs to be happening immediately. Given that the
relbranch didn't even build for several days last week there's a good chance
that there are fun lurking bugs there.
- Kyle
John J Barton
> Hi,
>
> Progress continues on the beta 7 blocker list:
>
> https://bugzilla.mozilla.org/buglist.cgi?quicksearch=blocking2:7
>
> In fact, the JS "compartments" changes have yet to land on this relbranch
...
>If we wanted to ship off the Beta 7 relbranch, we would need to:
>
> - merge the JS "compartment" changes and regression fixes,
Just a suggestion: do this step then decide. If it goes well, gathering
the fixes from the other branch will be tempting. If not, then you won't
have to decide yet ;-).
jjb
Mike Beltzner
> Since the October 6th branch point, fixes for over 300 beta8+, betaN+ and final+ blocker bugs have landed on mozilla-central:
>
> http://is.gd/giRLI
This URL should have been: http://is.gd/gk87b - apologies for the error.
cheers,
mike
Mike Beltzner
I'll be following up in a separate thread, but to close the loop, we
decided at the development planning meeting today that the best option
was to ship Beta 7 off of mozilla-central.
Thanks for your input and helping us make this decision.
cheers,
mike