Because information and proposals have been trickling out through the wiki pages and the weekly meeting notes, I'd like to follow up with a longer-form version of why I think we should do Firefox "Lorentz" in March 2010 based off of the 1.9.2 branch.
At the Firefox work week I sat at a whiteboard with Beltzner and some others thinking about the features that the Firefox team wants to ship in 2010:
* Crash-safe plugins (OOPP) * Transition to jetpack extensions * New UI "stuff": the new menu/toolbar structure and platform integration * Weave * Updater which doesn't interrupt the user * More responsive UI (async I/O, primarily) * Better startup time * Integrated developer tools
From a user point of view, some of these changes are non-disruptive, while some are definitely disruptive. The non-user-disruptive changes are:
* Crash-safe plugins (OOPP) * Updater which doesn't interrupt the user * More responsive UI (async I/O, primarily) * Better startup time
The desire is to get these features into the hands of users as quickly as pratical. These changes will still require a large beta with 600k+ users. The code name for this release is Firefox "Lorentz".
The other features, especially the new menu/toolbar UI structure and integrated jetpack, aren't going to be ready to go to beta until June, and are going to require extensive user feedback cycles after that.
Given these constraints, we have two options: we could ship the "Lorentz" release from 1.9.2, backporting the new changes. Or we could do a release off of mozilla-central/1.9.3.
I believe that backporting the OOPP work to 1.9.2 is a task whose scope and schedule implications are fairly well-known: the changes are limited in scope to a few existing files and the new IPC code. Most of the other changes (apart from the updater) have already landed on trunk and can be backported with minimal pain. In addition, we can mitigate risk for plugins by controlling which plugins are run out-of-process, using either a whitelist or a blacklist. With the project branch, I believe we could go to beta in the middle of January and release in late March/early April without disrupting the regular cycle of 3.6 security and stability releases. There are some issues to work out around localization changes on a stable branch, but I don't believe these will be insurmountable.
Doing a release from mozilla-central/1.9.3 presents a lot of schedule risk without matching reward. We would probably have to un-do changes that have already been made, such as the OJI plugin support removal as well as support for MacOS 10.4. Even if we went to beta in mid-January, the schedule needed to stabilize that branch for final release is much less clear. Doing a release from 1.9.3 also increases the risk of having to do a minor update due to extension compatibility from API and UI changes.
Do the content/layout/JS groups have similar lists of features to ship in 2010? It would be worthwhile to see whether/how they would fit into the proposed release structure. For example, the Direct2D rendering backend is fairly self-contained; could it land in the Lorentz timeframe on 1.9.2? SMIL and WebGL seem like possible candidates as well.
From a long-term perspective, I think that we'd love to be able to do quicker minor updates from the main "mozilla-central" tree, and transitioning from the current overlay/inject extension architecture to a jetpack is a key part of this plan. But until that is accomplished, I don't think short-cycle releases from mozilla-central are realistic.
It's also important to note that the current Lorentz plan doesn't have to be a single landing-point. If OOPP is ready in March but the updater isn't ready until April, they could land in different dot releases (Firefox 3.6.2 and 3.6.3).
With the 1.9.2-lorentz project branch, I think that we will have the best balance of controlling risk and gaining confidence in new features, and that ultimately that can help redefine our notion of what it means to land something on a stable branch.
> Because information and proposals have been trickling out through the > wiki pages and the weekly meeting notes, I'd like to follow up with a > longer-form version of why I think we should do Firefox "Lorentz" in > March 2010 based off of the 1.9.2 branch.
> Given these constraints, we have two options: we could ship the > "Lorentz" release from 1.9.2, backporting the new changes. Or we could > do a release off of mozilla-central/1.9.3.
Will the Firefox release be 3.6.X and Firefox 3.6 compatible addons work with the proposed release?
(BTW 'electrolysis' is chemistry, not physics, so we'd rather see the release named "Davy" or "Faraday" or "Nicholson" or "Carlisle").
> Do the content/layout/JS groups have similar lists of features to ship > in 2010? It would be worthwhile to see whether/how they would fit into > the proposed release structure. For example, the Direct2D rendering > backend is fairly self-contained; could it land in the Lorentz > timeframe on 1.9.2? SMIL and WebGL seem like possible candidates as well.
I have a list of things that were important for developers.
1. CSS Transitions (this is top of my list right now, esp as we reach into mobile.) 2. WebSockets (has patch, needs (final?) review.)
Plus performance, performance, performance.
dbaron has most of the Transitions work done, I believe, in a combination of stuff on -central and in his own tree - no idea what he thinks the impact and risk is there or if it's very self-contained.
WebSockets seems reasonably self-contained, at least in my understanding, with a small web-facing API.
I would love to have Direct2D on that train - as Shaver said, it's like a whole different browser.
We need to make sure this train doesn't get too big, though, or it will stretch out into a pretty long release.
> dbaron has most of the Transitions work done, I believe, in a > combination of stuff on -central and in his own tree - no idea what he > thinks the impact and risk is there or if it's very self-contained.
I'll let dbaron handle this (though I happen to think it's not that self-contained; I wouldn't want to try backporting it to 1.9.2).
> WebSockets seems reasonably self-contained, at least in my > understanding, with a small web-facing API.
And a pretty major refactoring of all the HTTP auth code to make it reusable for websockets. Not self-contained at all, in fact.
And a lot of the performance work in layout and DOM that's been happening on m-c is not exactly backportable easily either. I can't speak for jseng.
> From a long-term perspective, I think that we'd love to be able to do > quicker minor updates from the main "mozilla-central" tree, and > transitioning from the current overlay/inject extension architecture to > a jetpack is a key part of this plan. But until that is accomplished, I > don't think short-cycle releases from mozilla-central are realistic.
As Firebug is a overlay/injection extension I'm puzzled by any connection between this architecture and the release cycle. What about this architecture causes long cycles?
> With the 1.9.2-lorentz project branch, I think that we will have the > best balance of controlling risk and gaining confidence in new features, > and that ultimately that can help redefine our notion of what it means > to land something on a stable branch.
I'm worried about the opposite: redefinition of 'trunk'. The March release slips to June, important features are proposed for stable branch in Sept...
On Tuesday 2009-12-29 18:22 -0500, Boris Zbarsky wrote:
> On 12/29/09 5:21 PM, Christopher Blizzard wrote: > >dbaron has most of the Transitions work done, I believe, in a > >combination of stuff on -central and in his own tree - no idea what he > >thinks the impact and risk is there or if it's very self-contained.
> I'll let dbaron handle this (though I happen to think it's not that > self-contained; I wouldn't want to try backporting it to 1.9.2).
There are patches it depends on that touch other areas of code, and some of the transitions patches touch a bit of other code, but I think backporting is relatively doable if we're willing to also port over the patches it depends on or the relevant pieces of them.
>> For example, the Direct2D rendering >> backend is fairly self-contained; could it land in the Lorentz >> timeframe on 1.9.2? SMIL and WebGL seem like possible candidates as well.
> I have a list of things that were important for developers. > [...]
I think we need to be rather ruthless about limiting the scope of what goes into Lorentz.... For a few reasons: (1) prevent schedule slip (2) avoid slowing trunk development by spending time backporting and (3) reduce risk of breakage in a minor update.
Especially the last point. We've had issues with bad updates in our already tightly-scoped "security/stability only" releases. I think exploring ways to bring stuff to users quicker is fantastic, but we should resist the temptation -- especially for this first time -- to pile in extra stuff in while we're there.
My impression (and maybe I'm wrong, or this was lost in bsmedberg's post) was that in some respect we shouldn't think about Lorentz as a new release, but as backporting super-high-value features (like OOPP) to the stable release -- with all the Tread Carefully that entails. Maybe there are a few other such features we could do this with (as an experiment, if nothing else), but if OOPP is the only thing to ever ship this way I'd be ok with that.
> As Firebug is a overlay/injection extension I'm puzzled by any > connection between this architecture and the release cycle. What about > this architecture causes long cycles?
The breadth of the extension API (all XPCOM interfaces, plus the entire browser UI, basically), which means that we have to have a maxVersion setup with the maxVersion getting bumped on each release, which means that all the extensions have to be updated to be compatible with the new release. This takes a long time, typically.
The long-term goal is to be able to have major updates that don't break at least jetpack extensions (in the sense that whatever jetpacks the user has installed just keep working with no changes needed to them). Firebug would probabl need to be updated for every release, as now... (at least insofar as bumping up maxVersion).
> I'm worried about the opposite: redefinition of 'trunk'. The March > release slips to June, important features are proposed for stable branch > in Sept...
This is a somewhat separate problem, and yes one made worse by long release cycles...
> Do the content/layout/JS groups have similar lists of features to ship > in 2010? It would be worthwhile to see whether/how they would fit into > the proposed release structure. For example, the Direct2D rendering > backend is fairly self-contained; could it land in the Lorentz timeframe > on 1.9.2? SMIL and WebGL seem like possible candidates as well.
SMIL could be backportable, if we do CSS transitions. It's pretty self-contained, and a lot of it is already in 1.9.2 (but disabled with a build switch). The main not-self-contained part is the interpolation code in the style system, which is shared with CSS transitions.
Boris Zbarsky wrote: > On 12/29/09 6:52 PM, John J. Barton wrote: >> As Firebug is a overlay/injection extension I'm puzzled by any >> connection between this architecture and the release cycle. What about >> this architecture causes long cycles?
> The breadth of the extension API (all XPCOM interfaces, plus the entire > browser UI, basically), which means that we have to have a maxVersion > setup with the maxVersion getting bumped on each release, which means > that all the extensions have to be updated to be compatible with the new > release. This takes a long time, typically.
Then I would say that the problem in the architecture is mixing binary and JS compatibility. Adding a (deeper) shim JS layer would allow decoupling, increasing the amount of binary change before a maxVersion change above the shim layer.
> The long-term goal is to be able to have major updates that don't break > at least jetpack extensions (in the sense that whatever jetpacks the > user has installed just keep working with no changes needed to them).
I think jetpack is terrific innovation in better dynamics and modern packaging. It also re-interfaces the platform. In my opinion, that part way more effort than makes sense. In particular, using a shim JS layer would solve the problem faster and cheaper. Many parts of the current API could be improved, but not by arbitrarily introducing new API.
>> The breadth of the extension API (all XPCOM interfaces, plus the >> entire browser UI, basically), which means that we have to have a >> maxVersion setup with the maxVersion getting bumped on each release, >> which means that all the extensions have to be updated to be >> compatible with the new release. This takes a long time, typically.
> Then I would say that the problem in the architecture is mixing binary > and JS compatibility. Adding a (deeper) shim JS layer would allow > decoupling, increasing the amount of binary change before a maxVersion > change above the shim layer.
It's not a matter of binary compat. That's actually very rarely a problem. Right now any change to any .xul file in Firefox needs a maxVersion bump, because the way overlays and the existing extension JS code tend to work is by assuming things about the XUL DOM. As soon as it changes you lose compat.
The solution, though is right: a new extension API that is more limited in terms of what you can do and hence easier to preserve across underlying changes.
> I think jetpack is terrific innovation in better dynamics and modern > packaging. It also re-interfaces the platform. In my opinion, that part > way more effort than makes sense. In particular, using a shim JS layer > would solve the problem faster and cheaper. Many parts of the current > API could be improved, but not by arbitrarily introducing new API.
That sounds like a great topic for a jetpack-specific thread.
Boris Zbarsky wrote: > On 12/29/09 11:28 PM, John J. Barton wrote: >>> The breadth of the extension API (all XPCOM interfaces, plus the >>> entire browser UI, basically), which means that we have to have a >>> maxVersion setup with the maxVersion getting bumped on each release, >>> which means that all the extensions have to be updated to be >>> compatible with the new release. This takes a long time, typically.
>> Then I would say that the problem in the architecture is mixing binary >> and JS compatibility. Adding a (deeper) shim JS layer would allow >> decoupling, increasing the amount of binary change before a maxVersion >> change above the shim layer.
> It's not a matter of binary compat. That's actually very rarely a > problem. Right now any change to any .xul file in Firefox needs a > maxVersion bump, because the way overlays and the existing extension JS > code tend to work is by assuming things about the XUL DOM. As soon as > it changes you lose compat.
Excellent news, that means the problem is even smaller. We should take advantage of this. We only need only to replace the overlay mechanism with eg jetpack-isms. Not XPCOM, not the whole enchilada.
> The solution, though is right: a new extension API that is more limited > in terms of what you can do and hence easier to preserve across > underlying changes.
Yes, extensions that can't do anything useful can be preserved, no problem. For the rest, I think this is wishful thinking, and opposite to the CanDoAnything model that makes extensions in Mozilla successful.
>> I think jetpack is terrific innovation in better dynamics and modern >> packaging. It also re-interfaces the platform. In my opinion, that part >> way more effort than makes sense. In particular, using a shim JS layer >> would solve the problem faster and cheaper. Many parts of the current >> API could be improved, but not by arbitrarily introducing new API.
> That sounds like a great topic for a jetpack-specific thread.
But the proposal is to wait for jetpack to get faster trunk cycles. I say that we should not make that gamble. That is the opposite of jetpack-specific ;-)
On Tue, 29 Dec 2009 11:26:34 -0800, John J. Barton wrote: > Benjamin Smedberg wrote: > ... >> Given these constraints, we have two options: we could ship the >> "Lorentz" release from 1.9.2, backporting the new changes. Or we could >> do a release off of mozilla-central/1.9.3.
> Will the Firefox release be 3.6.X and Firefox 3.6 compatible addons work > with the proposed release?
> (BTW 'electrolysis' is chemistry, not physics, so we'd rather see the > release named "Davy" or "Faraday" or "Nicholson" or "Carlisle").
And here I was wondering which wildlife park/reserve was named "Lorentz".
Philip Chee wrote: > On Tue, 29 Dec 2009 11:26:34 -0800, John J. Barton wrote: >> Benjamin Smedberg wrote: >> ... >>> Given these constraints, we have two options: we could ship the >>> "Lorentz" release from 1.9.2, backporting the new changes. Or we could >>> do a release off of mozilla-central/1.9.3. >> Will the Firefox release be 3.6.X and Firefox 3.6 compatible addons work >> with the proposed release?
>> (BTW 'electrolysis' is chemistry, not physics, so we'd rather see the >> release named "Davy" or "Faraday" or "Nicholson" or "Carlisle").
> And here I was wondering which wildlife park/reserve was named "Lorentz".
Please pardon my aside, I should know better. How about the on-topic question?
Benjamin Smedberg wrote: > Doing a release from mozilla-central/1.9.3 presents a lot of schedule > risk without matching reward.
Not doing it is a big schedule risk for anyone who had already planned to base their work on a 1.9.3 release in June/July. And this is my fear - I don't see SeaMonkey ready for a release before May or June at least, and we have no 1.9.2 testing whatsoever yet, we also will (optimistically) need about 3 months before we even have build machine capacity to even cover an additional branch, given the experience I have of requesting machines and getting them after long discussions. And we want to get nearer to being in sync with Firefox with new releases on new Gecko/Mozilla platforms. With the original plan for 1.9.3, that looked possible, with the "Lorentz" plans I see us failing there and staying a few months behind everything FF does, which is clearly not what we want.
In a greater picture, if we want others to build on the Mozilla platform, we need to provide somewhat reliable plans, and dropping the January branch and June release for some time-expensive stuffing of things into the current stable branch and making it more unstable, putting the next bigger release in very late 2010 or even 2011 is not really what helps those parties.
> We would probably have to un-do changes > that have already been made, such as the OJI plugin support removal as > well as support for MacOS 10.4.
Then either those steps were probably premature in the first place or you are painting the old plans black to make yours look shinier.
> I think we need to be rather ruthless about limiting the scope of what > goes into Lorentz.... For a few reasons: (1) prevent schedule slip (2) > avoid slowing trunk development by spending time backporting and (3) > reduce risk of breakage in a minor update.
> SMIL could be backportable, if we do CSS transitions. It's pretty > self-contained, and a lot of it is already in 1.9.2 (but disabled with a > build switch). The main not-self-contained part is the interpolation > code in the style system, which is shared with CSS transitions.
I don't think we should backport SMIL or CSS transitions, personally. In the long run, backporting is a lose, so we should only backport things that are extremely high value. IMHO the excitement around new Web platform features focuses on what's on trunk, not so much what we're actually shipping in stable releases, so generally backporting them is not high value.
> I believe that backporting the OOPP work to 1.9.2 is a task whose scope > and schedule implications are fairly well-known: the changes are limited > in scope to a few existing files and the new IPC code. Most of the other > changes (apart from the updater) have already landed on trunk and can be > backported with minimal pain. In addition, we can mitigate risk for > plugins by controlling which plugins are run out-of-process, using > either a whitelist or a blacklist.
As I just wrote in another message, if we whitelist --- especially if the list contains only Flash --- I think this is an OK plan. I suspect making "unknown" plugins OOP would end up requiring a much longer test and stabilization cycle than you want.
> For example, the Direct2D rendering backend is fairly self-contained;
> could it land in the Lorentz timeframe
> on 1.9.2? SMIL and WebGL seem like possible candidates as well.
I think we should consider D2D, although there are some potential issues, like which drivers and hardware we would enable it for.
I don't think SMIL, CSS Transitions, WebGL or Web Sockets are worth considering. As I just wrote in another message, I think the difference in value between "on trunk" and "shipped to users" is a lot lower for Web platform features than for user-facing features. WebGL also has unresolved issues that we can't ship with.
> Because information and proposals have been trickling out through > the wiki pages and the weekly meeting notes, I'd like to follow up > with a longer-form version of why I think we should do Firefox > "Lorentz" in March 2010 based off of the 1.9.2 branch.
> At the Firefox work week I sat at a whiteboard with Beltzner and > some others thinking about the features that the Firefox team wants > to ship in 2010:
> * Crash-safe plugins (OOPP) > * Transition to jetpack extensions > * New UI "stuff": the new menu/toolbar structure and platform integration > * Weave > * Updater which doesn't interrupt the user > * More responsive UI (async I/O, primarily) > * Better startup time > * Integrated developer tools
> From a user point of view, some of these changes are non-disruptive, > while some are definitely disruptive. The non-user-disruptive > changes are:
> * Crash-safe plugins (OOPP) > * Updater which doesn't interrupt the user > * More responsive UI (async I/O, primarily) > * Better startup time
I think when we discuss our plans for Firefox releases, we should also think about what platform features we want to ship, and when. Platform features are important because (1) the capabilities supported by the Web platform affect people's choices between developing Web applications and developing applications on other platforms and (2) perception of our leadership in Web platform features affects our ability to influence the design of the Web platform in directions we care about (user control, security, etc.).
I have bad feelings about this plan based on the last time we did this: Firefox 2.0 sucked resources away from the trunk and allowed it to become extremely unstable, and it look a long time to get things back together for Firefox 3.
If we're serious about not doing that this time, then we need to put significant resources into moving mozilla-central development forward simultaneously with Lorentz: this means planning the next release, shipping alphas (I think we should do 1.9.3a1 in January), and setting a clear target for when we expect to ship (I'd hope summer of 2010 at the latest).
I think having a clear plan for shipping mozilla-central is also very important to show all the contributors who aren't working on one of the small number of things that are part of Lorentz that their work is valued.
It's also important because the bigger the gaps we make between releases (especially unexpected gaps), the harder we make it to ship any individual release, because people try to cram more stuff into it (given that the next one is indefinitely far away).
> Do the content/layout/JS groups have similar lists of features to ship > in 2010?
I just looked at the post-branch changes I've made so far on m-c, more or less, [1] and those that aren't security fixes tend to be performance and architecture work. It's possible to backport some of the latter, with enough pain, but I don't think moving it up by 3 months is worth the time investment. That assumes, of course, that we still plan to be shipping Gecko 1.9.3 as planned sometime in early summer 2010. Which I sincerely hope we are.
My general feeling is that most of the recent content/layout work falls into similar buckets.
> It would be worthwhile to see whether/how they would fit into > the proposed release structure.
This would have been a good question to ask 6 months ago when planning and time prioritization for this various work was happening...
In general, I think getting our newest-and-best layout features into an alpha and out there is almost as important as getting it into a shipping release. Possibly more important in some cases in terms of the hype and marketing effects (sad, but true). Getting in our performance improvements is in a similar position, but with betas, not alphas -- a lot of the comparisons people are making are between betas, not shipping releases. A lot of the web developer hype seems to be about alphas, not shipping releases.
> But until that is accomplished, I > don't think short-cycle releases from mozilla-central are realistic.
Shorter than the 1.9.1 to 1.9.2 cycle? In general, the current plan is for the 1.9.2 to 1.9.3 cycle to be about the same length as that, right?
> It's also important to note that the current Lorentz plan doesn't > have to be a single landing-point. If OOPP is ready in March but > the updater isn't ready until April, they could land in different dot > releases (Firefox 3.6.2 and 3.6.3).
This seems perfectly reasonable to me, sure.
To be clear, I can see why we want to land OOPP on 1.9.2.x. I think it'll take some work, but it might be worth it (probably is) for the immediate boost to stability and user experience. I don't think backporting random layout/DOM architecture work makes sense, and I don't think that it makes sense to try to disentangle features that landed after architecture work from said architecture work, in terms of time investment. Better to aggressively work on developing and stabilizing 1.9.3, with the explicit goal of shipping it in early summer 2010. David's proposal of a 1.9.3a1 in January makes a lot of sense to me.
One other note: one issue as I see it is that we have several different audiences we're targeting: the mass of our users, early-adopter users, and web developers (some overlap between these last two). They're looking for slightly different things, and are actually best served by different release schedules... Hence the tension we're seeing here.
-Boris
[1] http://hg.mozilla.org/mozilla-central/pushloghtml?startdate=2009-08-13&endd ate=2010-01-01&user=bzbar...@mozilla.com is the set of changes I pushed; some of these were not authored by me.
> One other note: one issue as I see it is that we have several different > audiences we're targeting: the mass of our users, early-adopter users, > and web developers (some overlap between these last two). They're > looking for slightly different things, and are actually best served by > different release schedules... Hence the tension we're seeing here.
Enterprise customer deployments? They won't follow a major-release-every-9-month schedule very long. Capacity divided by the number of still supported branches gives an estimate of a maximum lifetime from branching to EoL which is ALWAYS too short for conservative Enterprise customers. Maybe something like the Ubuntu or Fedora/RedHat model with shortlived branches and only few long term support branches with security fixes only?
