I wanted to kick off a thread about what Gecko.next timing and scope should look like.
As I see it, we have the following general desiderata for Gecko (in no particular order):
1) Ship a Fennec release somewhere in June. This release would ideally include out-of-process tabs, which is not an easy thing to backport to 1.9.2. 2) Ship web-facing features and layout performance improvements to web developers (more on this later). 3) Ship a Firefox release (or maybe two) this year. When is a good question. 4) Ship js performance improvements.
A significant motivation for the second proposal is item 1 on the above list. That proposal also helps somewhat with shipping item 2 a bit earlier, keeping us more competitive. The drawback is that item 3 becomes more complicated (possibly pushing out Firefox features to later in the year). Furthermore, it looks like the general timeframe for further significant JS performance improvements is somewhat like this:
End of March: arch changes needed in place, but probably no significant performance changes yet. Start of summer: More stable; some speedups After end of summer: payoff
In summary, if we follow the second proposal we will not pick up these JS performance changes in the resulting June release.
The other important piece of data is that https://wiki.mozilla.org/User:Dbaron/1.9.3_Alpha has a list of things that have either already landed or are very close to landing on trunk and could use some more-widespread testing and that would be ready to ship in June to users. I've gone through our content/layout checkins and put the relevant things on this list. Not sure whether there's something else that should be on there.
Open questions: 1) Is that list enough to justify an alpha in the near future? 2) Can we make 1.9.3 (current m-c) a minor update to 1.9.2 (so rename it as 1.9.2.something)? 3) If not, what if anything would we want to backport to 1.9.2?
Partial answers:
1) I think it is. 2a) The list of IDL interfaces changed since 1.9.2 is at https://wiki.mozilla.org/User:Bzbarsky/1.9.3_Interface_Changes and there have of course been changes to things like nsIContent since then. 2b) I've looked through the 1500 content/layout changesets and nothing else there jumped out at me as "probably breaking extensions", but who knows. We have about 4500 post-branch changesets in other parts of the code that I have NOT looked at. 2c) There is no Java support on Mac on m-c right now. The won't be until at least two weeks from now. After that, we'll see; there's a good chance there won't be Java support on Mac until March. 2d) Trunk has no OS X Tiger (10.4) support. 2e) Backing stuff out from trunk so as to restore compatibility to the point of shipping it as 1.9.2.x would presumably involve undoing the changes from (2a) and restoring support for 10.4, at least. The latter is estimated as several weeks of work, with nontrivial regression risk and involves a switch back to an older compiler that generates slower code. 3) Not sure anything quite jumps out at me.
My overall gut feeling is that the answer to (2) above is "No" as things stand...
This seems to be about it for the current state of things. The things we should decide ASAP are:
* Should we ship an alpha next week? * What is the June plan, if any? * What is the post-June plan?
> I wanted to kick off a thread about what Gecko.next timing and scope > should look like.
> As I see it, we have the following general desiderata for Gecko (in > no particular order):
> 1) Ship a Fennec release somewhere in June. This release would > ideally include out-of-process tabs, which is not an easy thing to > backport to 1.9.2. > 2) Ship web-facing features and layout performance improvements to > web > developers (more on this later). > 3) Ship a Firefox release (or maybe two) this year. When is a good > question. > 4) Ship js performance improvements.
> Are there others?
Two things that I'd like to see are the Direct2D work and the initial layers work.
Boris Zbarsky wrote: > I wanted to kick off a thread about what Gecko.next timing and scope > should look like.
> As I see it, we have the following general desiderata for Gecko (in no > particular order):
> 1) Ship a Fennec release somewhere in June. This release would > ideally include out-of-process tabs, which is not an easy thing to > backport to 1.9.2.
The second sentence is very surprising. The cost/benefit of out of process tabs is dubious on desktop and very odd as the top goal for Fennec that then drives the rest of the project.
> 2) Ship web-facing features and layout performance improvements to web > developers (more on this later). > 3) Ship a Firefox release (or maybe two) this year. When is a good > question.
Target Jun 24, land mid Sept. No matter what else.
The information that page is puzzling, a mix of things that seem sensible with things that are not; a mix of mythical 4.0 and practical 3.6 OPP.
I think more information on the items would help. Some of the items on the list can't make 2010 let alone June.
Since add-on compatibility is blocking release cycles, encourage AMO to change to a test-and-notify approach for addons.
> A significant motivation for the second proposal is item 1 on the above > list.
The second proposal in the web site Roadmap I guess? The motivation does not make sense and you can't hit the dates with that much change. But the second proposal makes a lot of sense nevertheless!
> That proposal also helps somewhat with shipping item 2 a bit > earlier, keeping us more competitive. The drawback is that item 3 > becomes more complicated (possibly pushing out Firefox features to later > in the year). Furthermore, it looks like the general timeframe for > further significant JS performance improvements is somewhat like this:
> End of March: arch changes needed in place, but probably no > significant performance changes yet. > Start of summer: More stable; some speedups > After end of summer: payoff
> In summary, if we follow the second proposal we will not pick up these > JS performance changes in the resulting June release.
The JS team is awesome and they are in a tight competitive battle. But just my observation: the JS issues have been the last to land in the last two releases. So that makes another plus for the 2nd proposal: less likely to slip.
On Fri, Jan 22, 2010 at 2:52 PM, Boris Zbarsky <bzbar...@mit.edu> wrote: > I wanted to kick off a thread about what Gecko.next timing and scope should > look like.
I think we risk getting a fair bit ahead of ourselves with respect to speculative stuff if we scope the conversation that broadly.
It would be great for the mobile guys to talk more about what they need and want in a June Fennec release (which platforms, how much e10s parallelism, etc.). They will know the major motivators for that, and whether it's more of a time-oriented release (in which case we should scope down the amount of e10s parallelism we undertake to have in the first milestone, IMO) or whether it's more "as soon as e10s-fennec works really well, and we'll take whatever else is along at that point" in which case I think the planning works out a little differently.
> 1) Ship a Fennec release somewhere in June. This release would > ideally include out-of-process tabs, which is not an easy thing to > backport to 1.9.2. > 2) Ship web-facing features and layout performance improvements to web > developers (more on this later). > 3) Ship a Firefox release (or maybe two) this year. When is a good > question. > 4) Ship js performance improvements.
For all of those things, we greatly increase our chances of success if we focus on keeping the trunk shippable, which means aggressively reacting to regressions. "That will be fixed by (the next tracemonkey merge|the html5 parser|a new version of libtheora|a big patch that's waiting for review" will probably not suffice. Things that regress sites or add frequent crashes need to be met with band-aids or backouts (or early merges) pretty much the same day they're found, or we risk ending up behind a pile of blockers too big to let us estimate effectively or adjust schedule for market or other reasons.
I think it is basically always worth shipping an alpha, to be honest. The bar on it is quite low, especially if we stay on top of regressions, and it lets us get more critical site-compatibility feedback earlier, if only from a relatively small set of people. We should be looking to use alpha shipping as a forcing function for getting work from "yeah, it's almost done, just need to X, Y, Z (deal with perf regression from X, figure out flaky test added with Z, get review on Y)" to "it's in the tree and we're on top of regressions, so we're a known distance from this phase being Done".
I think the best discussion to have right now is "what are the regressions that need to be addressed before we can ship an alpha off m-c" (let's ignore the version numbering of it for now, beltzner and I can walk across those coals next week or something), and then look at the work that could go in another alpha 3-4 weeks later. Pieces of work that can get backported to 1.9.2 should get listed somewhere for evaluation: not everything will (and things that are API-tweaking almost certainly won't), but we've taken new features in "minor updates" before (http-only cookie support springs to mind) and we'll do so again (OOPP).
Whether we ship a given upcoming release as a "minor" (default-yes) or not, and when, is something that requires us to know a lot more about the context in which we'd ship it than we do now. For now, let's focus on what is already in the tree and ready to go in an alpha, and spend some time getting it put to bed.
Next week I want to start a series of threads about the major (important, perhaps not large or invasive) pieces of work we have planned or underway, who we need to get those changes in front of, and how we might most effectively do that. The world of possibility is much bigger than 1.9.2.x-is-minor, 1.9.3-is-major, monolithic planning, and we should be looking more from first principles. Then we can decide if that means we want a beta or an alpha or a special parallel preview build or an extension or something shipped-with-a-pref-off or what.
For now, I very much think we want to ship an alpha in the next 2 weeks, and that we mostly need to get our heads around what the current regression space looks like. Our (well-proven!) ability to ship pre-release updates every week or two means that we don't need to rush additional things into a given alpha, though see above about the forcing function as well.
>> 1) Ship a Fennec release somewhere in June. This release would >> ideally include out-of-process tabs, which is not an easy thing to >> backport to 1.9.2.
> The second sentence is very surprising. The cost/benefit of out of > process tabs is dubious on desktop
It is?
> and very odd as the top goal for Fennec that then drives the rest of the project.
I don't follow.
> Target Jun 24, land mid Sept. No matter what else.
You know... sometimes I get the feel that you're trolling.
> On Fri, Jan 22, 2010 at 2:52 PM, Boris Zbarsky<bzbar...@mit.edu> wrote: >> I wanted to kick off a thread about what Gecko.next timing and scope should >> look like.
> I think we risk getting a fair bit ahead of ourselves with respect to > speculative stuff if we scope the conversation that broadly.
Well, I'd be happy to just scope it to June and mobile, but that ends up drawing in Firefox and then Firefox plans for later this year... :(
> It would be great for the mobile guys to talk more about what they > need and want in a June Fennec release (which platforms, how much e10s > parallelism, etc.).
Agreed. Waiting on this data.
> For all of those things, we greatly increase our chances of success if > we focus on keeping the trunk shippable
Agreed.
> I think it is basically always worth shipping an alpha, to be honest.
Excellent. Then let's do it. How/when do we make this call and what does it take to make it happen? ;)
> I think the best discussion to have right now is "what are the > regressions that need to be addressed before we can ship an alpha off > m-c"
Offhand, I'm not actually aware of any in the modules I follow, modulo the Java situation on Mac (there isn't any). But I think we'll just have to ship an alpha that way; waiting on Java will take too long. I do know of some regressions; I don't think they should hold an alpha.
> Pieces of work that can get backported to 1.9.2 should get listed somewhere for > evaluation: not everything will (and things that are API-tweaking > almost certainly won't), but we've taken new features in "minor > updates" before (http-only cookie support springs to mind) and we'll > do so again (OOPP).
That's the thing... the desirability of backporting something depends on how soon the next release is. If we're shipping off trunk in June, I wouldn't backport anything on that 1.9.3_Alpha list, period. If we're shipping off trunk in Sept/Oct I might have to think about it a bit more, but still err on the side of not backporting. If we're shipping off trunk in 2011, then I think we need to do some backporting.
> Whether we ship a given upcoming release as a "minor" (default-yes) or > not, and when, is something that requires us to know a lot more about > the context in which we'd ship it than we do now.
The thing is that if we're considering making the release off trunk a minor update to 1.9.2 then we need to:
1) Work on reverting some changes that break compat. 2) Perhaps hold off on other changes that would further break compat.
The sooner that decision is made (and I take your point that we're not sure what June will look like), the less pain those two items will be.
> Next week I want to start a series of threads about the major > (important, perhaps not large or invasive) pieces of work we have > planned or underway, who we need to get those changes in front of, and > how we might most effectively do that.
That was in fact one main purpose of starting this thread.
> The world of possibility is much bigger than 1.9.2.x-is-minor, 1.9.3-is-major
Is it, though? In terms of Firefox extension compat those seem to be our options.... Either we rev the toolkit version and then can have interface changes, or we don't and we can't. Is there a middle ground we're willing to take there?
> For now, I very much think we want to ship an alpha in the next 2 > weeks
If nothing unexpected comes up, what stops us from doing it next Wednesday (Jan 27), say?
From a SeaMonkey POV, the alternative proposal looks much more workable than the first one on that page.
> * Should we ship an alpha next week?
Not sure if next week can be done from a resources POV, but I think the sooner, the better - if we follow the alternative proposal. And if we don't want to let 1.9.3 grow too large, I think that would be the better option.
> * What is the June plan, if any?
SeaMonkey will probably plan for a stable release around that, so teaming up there might be a good idea.
> * What is the post-June plan?
I'd really like to see us come to regular 6-8 month off-trunk release cycles in the future.
The largest problem I'm seeing is add-on compatibility - not actual incompatibilities with code, though, actually, but more the compatibility settings in install.rdf or AMO's developer panel.
Still, I hope there are possibilities to deal with that, it would really fit the plans on our side for sure. :)
Boris Zbarsky wrote: > On 1/22/10 4:03 PM, John J Barton wrote: >>> 1) Ship a Fennec release somewhere in June. This release would >>> ideally include out-of-process tabs, which is not an easy thing to >>> backport to 1.9.2.
>> The second sentence is very surprising. The cost/benefit of out of >> process tabs is dubious on desktop
> It is?
I think the costs are very high, in terms of impact on the developers, ecosystem, and final runtime resources. Benefits are not as big. Much more in the nice-to-have some day than the drop-everything now category.
Plus I've not seen any discussion of how extensions fit in multiprocess.
>> and very odd as the top goal for Fennec that then drives the rest of >> the project.
> I don't follow.
It was your first item and it figures strongly in the rest of the analysis.
>> Target Jun 24, land mid Sept. No matter what else.
> You know... sometimes I get the feel that you're trolling.
I meant: don't let feature arrival dates drive the schedule. I'm agreeing with proposal 2.
>> The second proposal in the web site Roadmap I guess? The motivation does >> not make sense and you can't hit the dates with that much change.
> The list of changes is a list of possibilities. It's not an "all of > these" list. The key part of the second proposal is the June ship date.
>> I think it is basically always worth shipping an alpha, to be honest.
> Excellent. Then let's do it. How/when do we make this call and what does it take to make it happen? ;)
See dbaron's later message do dev-planning, "Planning (and triaging) for the 1.9.3 alphas." Everyone needs to triage for the alpha, get the bugs in line with the right flags. We crank through them, and release an alpha as usual. No magic here.
>> 1) Ship a Fennec release somewhere in June. This release would >> ideally include out-of-process tabs, which is not an easy thing to >> backport to 1.9.2.
> The second sentence is very surprising. The cost/benefit of out of > process tabs is dubious on desktop and very odd as the top goal for > Fennec that then drives the rest of the project.
The big reason for process separation on mobile is responsiveness. We frequently see situations where the XUL-driven user interface of Fennec isn't able to respond to user input quickly enough because time is being taken by HTML rendering, content scripting, or plugins. In comparison, the C++-based UI of MicroB (the default browser on Maemo 5) uses a separation model already with the Gecko engine running in a background process and page surfaces being send across processes to the browser user interface. While latency is added to actually manipulate the page, the UI is able to give feedback to the user much more quickly. We feel that the e10s work should give us similar results while keeping the UI implemented in extensible JS/XUL.
> I think the costs are very high, in terms of impact on the developers, > ecosystem, and final runtime resources. Benefits are not as big.
I think you're wrong on that. The responsiveness and crash-recovery benefits are big.
> Much more in the nice-to-have some day than the drop-everything now category.
It's not drop-everything-now, but need-to-get-this-working-sooner-rather-than-later.
> Plus I've not seen any discussion of how extensions fit in multiprocess.
Depends on the extension. One of the goals is to make existing extensions largely supported, iirc.
>>> and very odd as the top goal for Fennec that then drives the rest of >>> the project.
>> I don't follow.
> It was your first item and it figures strongly in the rest of the analysis.
I just said that one of the main Gecko consumers is looking to release in June and would really like a Gecko feature that's not on 1.9.2 and won't be on 1.9.2 to be ready by then if possible.
Boris Zbarsky wrote: >> I think the costs are very high, in terms of impact on the developers, >> ecosystem, and final runtime resources. Benefits are not as big.
> I think you're wrong on that. The responsiveness and crash-recovery > benefits are big.
Perhaps this assessment will be reconsidered after the out-of-process plugins ship. That was a great idea, a way of getting lots of benefit at a lower total cost. Are there other incremental steps?
>> Much more in the nice-to-have some day than the drop-everything now >> category.
> It's not drop-everything-now, but > need-to-get-this-working-sooner-rather-than-later.
>> Plus I've not seen any discussion of how extensions fit in multiprocess.
> Depends on the extension. One of the goals is to make existing > extensions largely supported, iirc.
Some discussion of the technical approach to that goal would be welcome. Absent any information, extensions have to made decisions in a vacuum. We estimated that converting Firebug to multiprocess will require 2 years. We are almost a year into that now. If we are wrong about our design, we are in big trouble.
>>>> and very odd as the top goal for Fennec that then drives the rest of >>>> the project.
>>> I don't follow.
>> It was your first item and it figures strongly in the rest of the >> analysis.
> I just said that one of the main Gecko consumers is looking to release > in June and would really like a Gecko feature that's not on 1.9.2 and > won't be on 1.9.2 to be ready by then if possible.
But the feature is also not in 3.7, so I don't understand how matters.
Ben Combee wrote: >>> 1) Ship a Fennec release somewhere in June. This release would >>> ideally include out-of-process tabs, which is not an easy thing to >>> backport to 1.9.2.
>> The second sentence is very surprising. The cost/benefit of out of >> process tabs is dubious on desktop and very odd as the top goal for >> Fennec that then drives the rest of the project.
> The big reason for process separation on mobile is responsiveness. We > frequently see situations where the XUL-driven user interface of Fennec > isn't able to respond to user input quickly enough because time is being > taken by HTML rendering, content scripting, or plugins. In comparison, > the C++-based UI of MicroB (the default browser on Maemo 5) uses a > separation model already with the Gecko engine running in a background > process and page surfaces being send across processes to the browser > user interface. While latency is added to actually manipulate the page, > the UI is able to give feedback to the user much more quickly. We feel > that the e10s work should give us similar results while keeping the UI > implemented in extensible JS/XUL.
Shawn Wilsher wrote: > On 1/23/2010 11:41 AM, John J. Barton wrote: >> Thanks! But why can't this be done with threads? > Largely the perils of shared memory.
> Cheers,
> Shawn
Certainly something to worry about! But it's too late, you already have threads. In any case, I shouldn't push back, this decision is taken. The roll out is what I should focus on.
> Certainly something to worry about! But it's too late, you already > have threads. In any case, I shouldn't push back, this decision is > taken. The roll out is what I should focus on.
Sure, we already have them, and that means we have expedience in knowing how painful they can be. I've fixed a number of correctness bugs due to race conditions in threads. I know I'm not the only one to fix a bunch either. They are simply no fun to deal with.
As I understand things, our IPC communication protocol is stateful (with the possible exception of the RPC stuff), which makes it all much easier to work with. One of the e10s folks should be able to better answer that though.
On Jan 23, 11:38 am, "John J. Barton" <johnjbar...@johnjbarton.com> wrote:
> Boris Zbarsky wrote: > >> I think the costs are very high, in terms of impact on the developers, > >> ecosystem, and final runtime resources. Benefits are not as big.
> > I think you're wrong on that. The responsiveness and crash-recovery > > benefits are big.
I have Firefox lock up enough and Windows turn the window gray to really look forward to this...
> Perhaps this assessment will be reconsidered after the out-of-process > plugins ship. That was a great idea, a way of getting lots of benefit at > a lower total cost. Are there other incremental steps?
OK, I guess I'm trolling here, but won't out of process plugins allow Firefox to ship in 64bit?
> Perhaps this assessment will be reconsidered after the out-of-process > plugins ship. That was a great idea, a way of getting lots of benefit at > a lower total cost. Are there other incremental steps?
Yes, for example getting oop tabs working for Fennec. ;)
> Some discussion of the technical approach to that goal would be welcome.
I believe there has been some already. The goal is to provide a JS-level wrapper that can "synchronously" proxy to the content process, so as to support current code without huge changes. Ideally, of course, code would be rewritten without using this wrapper, since that would allow it to not block the UI on its RPC calls. What the APIs would look like for that is still unclear.
>> I just said that one of the main Gecko consumers is looking to release >> in June and would really like a Gecko feature that's not on 1.9.2 and >> won't be on 1.9.2 to be ready by then if possible.
> But the feature is also not in 3.7, so I don't understand how matters.
It's not in 3.7 _yet_. The point would be to have it ready by June (ready in the sense that Fennec can use it, not that Firefox can; the latter would take a lot more work).
Boris Zbarsky wrote: > On 1/23/10 2:38 PM, John J. Barton wrote: >> Perhaps this assessment will be reconsidered after the out-of-process >> plugins ship. That was a great idea, a way of getting lots of benefit at >> a lower total cost. Are there other incremental steps?
> Yes, for example getting oop tabs working for Fennec. ;)
>> Some discussion of the technical approach to that goal would be welcome.
> I believe there has been some already. The goal is to provide a > JS-level wrapper that can "synchronously" proxy to the content process, > so as to support current code without huge changes.
But extension UIs are where? Presumably in another content process. So the call goes from one content process through something then to another content process?
> Ideally, of course, > code would be rewritten without using this wrapper, since that would > allow it to not block the UI on its RPC calls. What the APIs would look > like for that is still unclear.
Firebug is working on the assumption that we have to do the remote protocol.
> But extension UIs are where? Presumably in another content process. So > the call goes from one content process through something then to > another content process?
Running extensions in a remote process seems rather difficult, so I suspect they'll run in the chrome process at first. That may change in the future, of course.
I do believe JetPack is to have a process per thing though (somebody please correct me if I'm wrong).
Shawn Wilsher wrote: > On 1/24/2010 1:09 PM, John J. Barton wrote: >> But extension UIs are where? Presumably in another content process. So >> the call goes from one content process through something then to >> another content process? > Running extensions in a remote process seems rather difficult, so I > suspect they'll run in the chrome process at first. That may change in > the future, of course.
Well the extensions are remote to the content, so they are in a remote process. I suppose the extensive work to isolate content and chrome helps make this division seem more approachable.
But what live in the 'content' vs 'chrome' processes? Is the content process like a single-tab browser, with complete network stack, JS engine, parser, UI rendering? And thus the chrome process just supervises and manages the screen sharing? Or?
> I do believe JetPack is to have a process per thing though (somebody > please correct me if I'm wrong).
Firebug's interaction with chrome is tiny compared to interaction with content.
> Well the extensions are remote to the content, so they are in a remote > process. I suppose the extensive work to isolate content and chrome > helps make this division seem more approachable.
Well, more precisely by default extensions will live in chrome. There will perhaps need to be ways for extensions to put code in the content process, imo. Unclear how yet, or whether it's desirable. I, for one, would rather never have code with chrome privileges running in the content process.
> But what live in the 'content' vs 'chrome' processes? Is the content > process like a single-tab browser, with complete network stack, JS > engine, parser, UI rendering?
General plan is that the network stack is in the chrome. Both processes have parser and js engine and layout. Painting (and in particular how to ship the stuff the content paints into the chrome process and paint it onscreen there) is still being fully sorted out, in general.
Boris Zbarsky wrote: >> I think the costs are very high, in terms of impact on the developers, >> ecosystem, and final runtime resources. Benefits are not as big.
> I think you're wrong on that. The responsiveness and crash-recovery > benefits are big.
But what is the memory cost of OOP tabs on mobile ? There's a lot of devices that are severely memory constrained. More memory use means it simply won't run on those devices.
