Firefox front-end features, time-to-market, and coordination.
Dietrich Ayala
Panorama and F1 that front-end features developed outside of the
Firefox team require massive refactorings/rewrites and long
development cycles to transition from an add-on experiment to a
feature in the core, and sometimes the feature doesn't land at all.
Also, there can be friction between groups as issues of performance,
quality, ownership, conflicting timelines and mismatched expectations
crop up during these transitions.
We need to reduce the friction and time that it takes to get these
types of projects into Firefox.
Currently both the Apps and Identity teams are using Jetpack as their
development platform. Jetpack is fantastic for rapid prototyping but
is targeted at lightweight add-on development, not ship-quality
browser core features. It's also not in Firefox.
The product road map for Apps and Identity is not specific about
timeline, but says they're included in what the Web looks like in 24 -
36 months. Given the competitive atmosphere and the historical
evidence for adding many months onto the schedule to get projects like
these shipping in Firefox, I think we need to look at ways to reduce
the time-to-integration for these kinds of projects.
Maybe this means immediately putting resources behind embedding
Jetpack in Firefox such that we can write core features on top of it.
Maybe it means we ship new features as add-ons that are included with
the release (killswitch, anyone?). Maybe it means we move these
projects off of Jetpack and into the core at the earliest possible
stage.
But no matter when the code lands in core, we're still hampered by a
fragile ecosystem: The mix of XUL/XPCOM/XBL/JS/CSS that we currently
use for core feature development is powerful yet extremely
regression-prone, and painstaking to develop against. Writing
shippable code is only possible by specialists with experiential
knowledge of a ton of undocumented code and a host of implicit
behaviors that are impossible to predict.
My vote is for embedding Jetpack inside of Firefox for use for
developing core front-end features. Having a stable and sane pure-JS
API layer to build new features against would:
* make it easier to share code between desktop and mobile
* ease integration of projects developed by external teams (and allow
possible code-sharing there too)
* ease contribution by volunteer developers, and ramp-up by new employees
* allow e10s-ification of core features using the same mechanisms and
patterns as add-ons
* strengthen Jetpack by ensuring it's APIs are up to the level of
stability and performance required to be in the core
Sure, it wouldn't work for everything - we'd still have to dive under
the covers for a bunch of things. Sure, we'll have to figure out a
host of issues around the interplay between shipped-SDK and
embedded-SDK. But I think it will allow front-end development to move
faster and more effectively in the long term.
Axel Hecht
Yes to most of all of the above.
Sadly, there's a pitfall, and that's that l10n in jetpack is, if at all,
in its infancy. I'm not sure how far we'll get anytime soon there for
beyond-easy UI, in particular as long as we don't have chrome registry
support (AFAICT, we don't).
Not sure how RTL and a11y go here, too.
Axel
Dao
> We've learned through experience with Sync, Personas, Account Manager,
> Panorama and F1 that front-end features developed outside of the
> Firefox team require massive refactorings/rewrites and long
> development cycles to transition from an add-on experiment to a
> feature in the core, and sometimes the feature doesn't land at all.
> My vote is for embedding Jetpack inside of Firefox for use for
> developing core front-end features.
For which of the cited projects would Jetpack have been an option? I'd
say none of Sync, Personas, Account Manager and Panorama. F1? I guess so.
Rob Campbell
On 2011-07-29, at 14:11 , Dietrich Ayala wrote:
> Maybe this means immediately putting resources behind embedding
> Jetpack in Firefox such that we can write core features on top of it.
> Maybe it means we ship new features as add-ons that are included with
> the release (killswitch, anyone?). Maybe it means we move these
> projects off of Jetpack and into the core at the earliest possible
> stage.
+1.
We looked hard at using Jetpack for devtools features and determined that it would've been just too difficult to do 6 months ago, and most of the issues we had then are still in place today.
We looked at what it would take to package a jetpack into the browser without having a baseline add-ons codebase in the browser itself and that, coupled with testing uncertainties and build-time packaging just made the whole process that much more daunting.
It meant most of the features we would write as Jetpacks would either remain as dissociated jetpacks living on AMO (not necessarily a bad thing, but that came with a set of limitations) or we'd have to build up a bunch of test infrastructure and coordinate with the addons teams to attempt to integrate features back into the browser.
Having an included version of Jetpack in the browser means a lot of that test-infrastructure and build-time uncertainty becomes a lot easier to deal with. Not to mention eliminates a lot of the compatibility concerns you run into with multiple versions of the SDK floating around in the ecosphere.
> But no matter when the code lands in core, we're still hampered by a
> fragile ecosystem: The mix of XUL/XPCOM/XBL/JS/CSS that we currently
> use for core feature development is powerful yet extremely
> regression-prone, and painstaking to develop against. Writing
> shippable code is only possible by specialists with experiential
> knowledge of a ton of undocumented code and a host of implicit
> behaviors that are impossible to predict.
>
> My vote is for embedding Jetpack inside of Firefox for use for
> developing core front-end features. Having a stable and sane pure-JS
> API layer to build new features against would:
>
> * make it easier to share code between desktop and mobile
> * ease integration of projects developed by external teams (and allow
> possible code-sharing there too)
> * ease contribution by volunteer developers, and ramp-up by new employees
> * allow e10s-ification of core features using the same mechanisms and
> patterns as add-ons
> * strengthen Jetpack by ensuring it's APIs are up to the level of
> stability and performance required to be in the core
And
* allow easy integration of existing test and build infrastructure.
* Make it possible to write well-tested code based on jetpack that ships in the browser.
* Make it possible to integrate jetpacks "as-is" without doing a total rewrite at integration time.
> Sure, it wouldn't work for everything - we'd still have to dive under
> the covers for a bunch of things. Sure, we'll have to figure out a
> host of issues around the interplay between shipped-SDK and
> embedded-SDK. But I think it will allow front-end development to move
> faster and more effectively in the long term.
All for it!
~ rob
Myk Melez
> We need to reduce the friction and time that it takes to get these
> types of projects into Firefox.
> Currently both the Apps and Identity teams are using Jetpack as their
> development platform. Jetpack is fantastic for rapid prototyping but
> is targeted at lightweight add-on development, not ship-quality
> browser core features.
That's true, but it's changing over time.
About six months ago, philikon asked me if he should use the SDK to
build an experimental feature that he eventually intended to land in
Firefox, and I told him "no," because our primary target audience was
casual developers building simple addons.
When Apps folks asked me the same question last month, my answer was
"maybe," since our secondary target audience is professional developers
building complex addons, and we're starting to expand our focus to their
use cases as part of our push to get traditional addon developers to
transition to the SDK.
And in six months, I hope the answer will be "yes," because we will have
addressed the various issues associated with SDK development of complex
addons.
> Maybe it means we ship new features as add-ons that are included with
> the release (killswitch, anyone?).
This is a particularly promising avenue for integration. We've done this
with Test Pilot for beta releases, and Mozilla China has done it with a
bunch of addons for stable releases. And Mozilla has helped to develop,
release, and market high-quality addons alongside Firefox, including
Personas, F1, and Kodak EasyShare Gallery Companion.
> My vote is for embedding Jetpack inside of Firefox for use for
> developing core front-end features.
I agree, but it's important to share an understanding of what "embedding
Jetpack" means and to schedule this work carefully.
For me it means landing the core API implementations and possibly the
module loader and the bootstrapper into core Firefox, and I think we
should do it after we finish moving addon execution out-of-process (i.e.
e10s), which is scheduled for Q3/4 of this year, because e10s is a
sea-change for our API implementations, and the team will be more
productive doing that transformative work in the project's own repository.
-myk
David Mandelin
> We've learned through experience with Sync, Personas, Account Manager,
> Panorama and F1 that front-end features developed outside of the
> Firefox team require massive refactorings/rewrites and long
> development cycles to transition from an add-on experiment to a
> feature in the core, and sometimes the feature doesn't land at all.
> quality, ownership, conflicting timelines and mismatched expectations
> crop up during these transitions.
technology problem. How about putting teams that develop core features
inside the Firefox team? The Jetpack proposal below is great, but beyond
that, I think "Team A does all the cool work inventing a new feature,
then Team B does all the grunt work to make it actually work" is a bad
model for all kinds of reasons.
> We need to reduce the friction and time that it takes to get these
> types of projects into Firefox.
> Maybe this means immediately putting resources behind embedding
> Jetpack in Firefox such that we can write core features on top of it.
> projects off of Jetpack and into the core at the earliest possible
> stage.
>
> fragile ecosystem: The mix of XUL/XPCOM/XBL/JS/CSS that we currently
> use for core feature development is powerful yet extremely
> regression-prone, and painstaking to develop against. Writing
> shippable code is only possible by specialists with experiential
> knowledge of a ton of undocumented code and a host of implicit
> behaviors that are impossible to predict.
> My vote is for embedding Jetpack inside of Firefox for use for
> developing core front-end features.
> Having a stable and sane pure-JS
> API layer to build new features against would:
>
> * make it easier to share code between desktop and mobile
> * ease integration of projects developed by external teams (and allow
> possible code-sharing there too)
> * ease contribution by volunteer developers, and ramp-up by new employees
> * allow e10s-ification of core features using the same mechanisms and
> patterns as add-ons
> * strengthen Jetpack by ensuring it's APIs are up to the level of
> stability and performance required to be in the core
* get Jetpack development properly integrated with Gecko and on
Tinderbox, so Jetpack doesn't get randomly broken by changes to Gecko
and SpiderMonkey.
> Sure, it wouldn't work for everything - we'd still have to dive under
> the covers for a bunch of things.
Can we try to extend Jetpack instead, as much as possible? As you say
above, Jetpack has the potential to be a far superior addon API to what
we have now.
> Sure, we'll have to figure out a
> host of issues around the interplay between shipped-SDK and
> embedded-SDK. But I think it will allow front-end development to move
> faster and more effectively in the long term.
Dave
Dave Townsend
> We need to reduce the friction and time that it takes to get these
> types of projects into Firefox.
>
Wholeheartedly agree.
> Currently both the Apps and Identity teams are using Jetpack as their
> development platform. Jetpack is fantastic for rapid prototyping but
> is targeted at lightweight add-on development, not ship-quality
> browser core features.
I'd love to hear what you think are the differences there. Ignoring the
embedding piece for now what is Jetpack missing that is required for core
feature development?
> The product road map for Apps and Identity is not specific about
> timeline, but says they're included in what the Web looks like in 24 -
> 36 months. Given the competitive atmosphere and the historical
> evidence for adding many months onto the schedule to get projects like
> these shipping in Firefox, I think we need to look at ways to reduce
> the time-to-integration for these kinds of projects.
>
> Maybe this means immediately putting resources behind embedding
> Jetpack in Firefox such that we can write core features on top of it.
> Maybe it means we ship new features as add-ons that are included with
> the release (killswitch, anyone?). Maybe it means we move these
> projects off of Jetpack and into the core at the earliest possible
> stage.
>
We've been actively talking with many of the teams you mention about this
very issue over the past few days, so far I haven't heard their dependence
on Jetpack as being a part that delays their release, but of course when
they are ready Jetpack needs to be ready for them too.
As you mention there are multiple ways we could ship features based on
Jetpack but I don't think we need to tie ourselves to just one of those
ways. My guess is that a fine first step would be to ship these features as
add-ons and then move on from there. It's my plan to spend some time over
the next week or so figuring out what barriers might be in place for us to
be able to do that.
David Dahl
> From: "Myk Melez" <m...@mozilla.org>
> To: "Dietrich Ayala" <auto...@gmail.com>
> Cc: "Todd Simpson" <to...@mozilla.com>, "mozilla. dev. planning group" <dev-pl...@lists.mozilla.org>, "Mike Connor"
> <mco...@mozilla.com>, "Jay Sullivan" <jsul...@mozilla.com>, "Dave Townsend" <mos...@mozilla.com>, "David Mason"
> <d...@mozilla.com>
> Sent: Friday, July 29, 2011 1:01:12 PM
> Subject: Re: Firefox front-end features, time-to-market, and coordination.
> And in six months, I hope the answer will be "yes," because we will
> have
> addressed the various issues associated with SDK development of
> complex
> addons.
Is there a roadmap/wikipage/list somewhere that enumerates the holes in the API to make the Addon SDK an avenue for core features? (I realize this has been discussed here and there, and may not have been a goal for the Addon SDK). Has anyone thought about running some analytics/datamining tools on the source of the most used interfaces on the top x number of traditional addons?
Regards,
David
Philipp von Weitershausen
do a brownbag or something simliar about this for a while. The
Services team was involved in some of the projects you mentioned below
(specifically Sync, F1). Feel free to rope us in if you need some
detailed feedback!
On Fri, Jul 29, 2011 at 10:11 AM, Dietrich Ayala <auto...@gmail.com> wrote:
> We've learned through experience with Sync, Personas, Account Manager,
> Panorama and F1 that front-end features developed outside of the
> Firefox team require massive refactorings/rewrites and long
> development cycles to transition from an add-on experiment to a
> feature in the core, and sometimes the feature doesn't land at all.
> Also, there can be friction between groups as issues of performance,
> quality, ownership, conflicting timelines and mismatched expectations
> crop up during these transitions.
Like others have suggested, some of this could be a management
problem. And to a certain degree it seems to be a development attitude
problem. Things like coding style and unit tests shouldn't be
optional. Throwing away experimental code, rewriting things from
scratch, etc. is OK.
> The product road map for Apps and Identity is not specific about
> timeline, but says they're included in what the Web looks like in 24 -
> 36 months. Given the competitive atmosphere and the historical
> evidence for adding many months onto the schedule to get projects like
> these shipping in Firefox, I think we need to look at ways to reduce
> the time-to-integration for these kinds of projects.
Absolutely. And without compromising on performance and stability.
> Maybe this means immediately putting resources behind embedding
> Jetpack in Firefox such that we can write core features on top of it.
> Maybe it means we ship new features as add-ons that are included with
> the release (killswitch, anyone?). Maybe it means we move these
> projects off of Jetpack and into the core at the earliest possible
> stage.
>
> But no matter when the code lands in core, we're still hampered by a
> fragile ecosystem: The mix of XUL/XPCOM/XBL/JS/CSS that we currently
> use for core feature development is powerful yet extremely
> regression-prone, and painstaking to develop against. Writing
> shippable code is only possible by specialists with experiential
> knowledge of a ton of undocumented code and a host of implicit
> behaviors that are impossible to predict.
I don't think all of the problems we've had in the past were due to
the fact that we were lacking a JS-friendly (or friendlier) API layer.
To speak from my experience, Sync needed (and still needs) performant
APIs that drill way down into our core components. Web Apps, Identity,
etc. will need stuff like extensible DOM APIs, desktop notifications,
etc. All of this functionality can be achieved today, in Labs, but it
would probably not meet the high standards we rightly have for shipped
code.
That said, I'm all for creating JS-friendly abstraction layers on top
of clunky and/or unstable APIs. Jetpack has done a fantastic job of
prototyping in this space, but I have my doubts it is ready as-is for
trunk -- not that you were implying it was. As far as I have
understood Myk in the past, Jetpack was developed mainly with
developer convenience in mind. That makes sense for an add-on SDK. For
core stuff, however, the main focus must shift more towards
performance and efficiency. This means no synchronous I/O, careful use
of object allocations, etc. This isn't particularly hard to do, it's
mainly just a change of attitude.
> My vote is for embedding Jetpack inside of Firefox for use for
> developing core front-end features. Having a stable and sane pure-JS
> API layer to build new features against would:
+1 to this and all of the points below.
Mike Connor
> My vote is for embedding Jetpack inside of Firefox for use for
> developing core front-end features.
I'm in favour of this, as a long-term play. But unsurprisingly, I'm
naturally skeptical of any assertion that $future_unfinished_product
will be so much better that we should commit to building on top of that
abstraction layer well before it's done.
Don't get me wrong, I want Jetpack to get done, and to get integrated,
and to kick ass. But until that day comes, I am not convinced we should
adopt a Jetpack-centric feature development plan. If that's an end
state we want, we should define requirements and ensure they're on the
Jetpack roadmap, and shift to that model once the platform pieces are
ready. But we have major pieces to ship before that day.
Myk says this work can/should come after e10s, so we're likely talking
about 6-9 months at least to have a minimum viable platform, and then
some amount of time after that for someone to build a viable, complex
feature on the new APIs to prove out the new platform. I still think
it's worth committing significant resources to this plan, but I'd also
like to see us define some sort of interim best practice that we can
apply right now, without adding a dependency on Jetpack to getting
critical projects done this year.
For the "until Jetpack is deemed ready" timeframe, I'd like to
counter-propose the following:
1. Prototype/iterate in whatever way is easiest/fastest to solidify the
product plan.
2. Once we have product requirements and an understanding of what we
need to build:
* Build features as restartless add-ons, optionally shipping these
via a LabKit/"Services Beta Channel" type of experimental feature
channel to get feedback.
* Enforce m-c grade test coverage, code review, and quality bars from
the start of this phase
* Provide a generic test framework that works in the add-on space,
where the tests are compatible with the main tree's test frameworks
* Use modern coding styles, JSMs, sane API design
* Write UI code in a way that it can be trivially ported into Firefox
proper
3. Once a feature is done enough to include in the mainline, dropping
that code into Firefox proper shouldn't take more than a few days for
one developer, since tests, code, and UI bits will need minimal changes.
This is an improvement on what we did late in the Sync endgame, and
applies the lessons we learned from that experience. And, again, we can
start applying this approach today, and transition over time to better
technology stacks as they become available.
-- Mike
Robert Kaiser
> My vote is for embedding Jetpack inside of Firefox for use for
> developing core front-end features. Having a stable and sane pure-JS
> API layer to build new features against would:
Removing markup (XUL/XBL right now, hopefully nobody even thinks about
removing CSS, or do you?) and making everything JS does surely
incredibly slow down any somewhat larger UI and makes code almost
unreadable and uncustomizable. I don't think that's a useful path to
anything. You also don't write large websites as one large JS script
that builds the DOM and styles all dynamically, right?
I'm not at all against bringing Jetpack in, but strongly against making
all UI elements build solely in JS. That sounds like a big mess and
would make contribution much harder. Markup like XUL is way easier to
read than figuring out how things get structured from a large JS script.
Robert Kaiser
--
Note that any statements of mine - no matter how passionate - are never
meant to be offensive but very often as food for thought or possible
arguments that we as a community should think about. And most of the
time, I even appreciate irony and fun! :)
David Dahl
> From: "Robert Kaiser" <ka...@kairo.at>
> To: dev-pl...@lists.mozilla.org
> Sent: Friday, July 29, 2011 2:08:06 PM
> Subject: Re: Firefox front-end features, time-to-market, and coordination.
> > My vote is for embedding Jetpack inside of Firefox for use for
> > developing core front-end features. Having a stable and sane pure-JS
> > API layer to build new features against would:
>
> Removing markup (XUL/XBL right now, hopefully nobody even thinks about
> removing CSS, or do you?) and making everything JS does surely
> incredibly slow down any somewhat larger UI and makes code almost
> unreadable and uncustomizable. I don't think that's a useful path to
> anything. You also don't write large websites as one large JS script
> that builds the DOM and styles all dynamically, right?
>
> I'm not at all against bringing Jetpack in, but strongly against
> making
> all UI elements build solely in JS. That sounds like a big mess and
> would make contribution much harder. Markup like XUL is way easier to
> read than figuring out how things get structured from a large JS
> script.
I'm pretty sure Dietrich means "Non-native code" when he referenced "pure-JS". You can load CSS, html and other types of files into a Jetpack Addon.
Regards,
David
Nicholas Nethercote
<phil...@googlemail.com> wrote:
> For core stuff, however, the main focus must shift more towards
> performance and efficiency.
Yes. For example, Jetpack's memory usage isn't great at the moment, see
https://bugzilla.mozilla.org/show_bug.cgi?id=672443 for an example.
There's also a known leak but I can't find the bug now.
Nick
Ehsan Akhgari
one day we get Jetpack integrated into the core Firefox, and for that to
happen, a number of architectural changes (like e10s awareness) and quality
changes (addressing memory-related problems, and stuff like that) need to
happen first. I don't think that has anything to do with developing
prototypes for new projects with a clear integration path with the core
Firefox.
I would like to speak from my experience helping the Panorama team integrate
their code into Firefox. Panorama (which was called TabCandy back in the
day) was developed in a very small team. There were a bunch of problems
that we faced during the integration phase:
1. No automated testing framework. This meant that the code had a lot of
bugs, and nobody knew about them. The basic stuff worked, but lots of edge
cases had weird problems. It took us a while before we managed to fix the
pile of bugs that people filed initially after landing and getting unit
tests for them to bring the test coverage to an acceptable level.
2. Performance problems. The TabCandy add-on was written without much
attention to performance. It affected startup time even if you never used
the feature, and it also had performance problems in some operations (most
notably, animations).
3. Coding style. The coding style used in TabCandy was *completely*
different to what the front-end people used in Firefox/Toolkit. This caused
a lot of pain for people who wanted to help (like me) to understand the
code, and also for the reviewers to make sense out of the code. There was a
basic level of documentation, but everything was so different that looking
at TabCandy code was not easy by the average front-end person.
Here's how we addressed these problems. The lack of automated testing hurt
us really badly. The Panorama team spent a lot of time to write unit tests
to cover the basic functionality, and since the time that Panorama landed on
mozilla-central, we required unit tests per every check-in, similar to the
rest of Firefox code. The performance problem we mostly addressed by
putting the Panorama team in contact with a Platform person (me in this
case). I had enough experience with the platform in general that I could
help the Panorama team understand why their code has performance problems,
and what to do about them. Part of this effort was focused before landing
Panorama for the first time (such as bringing down the Ts regression to 0)
and part of this was performed after the code had landed (such as improving
the animation performance). The third problem was never really addressed.
If I have to share the lessons that I learned from this efforts, I can
summarize them as:
1. Getting feedback from users/developers early on is valuable. As mconnor
mentions, creating new features as restartless add-ons with the
release-early-release-often model is extremely valuable. I think that
feedback should be collected from both users and developers, since the
latter group can help the team working on New Shiny Feature to understand
some of the potential integration/performance problems from early on, and
work on addressing them before it's too late.
2. Code quality should never be an after-thought. While I understand that
having unit tests per check-in might be too strict of a requirement for
early prototyping, I think automated testing is a must at least after the
goals for a project are set. Sure, things will change, and tests need to be
rewritten from scratch, but I think that there is very real value in the
getting higher quality code by our common testing practices, at least from
the point where we get serious about the project.
Hope this helps,
--
Ehsan
<http://ehsanakhgari.org/>
On Fri, Jul 29, 2011 at 2:42 PM, Mike Connor <mco...@mozilla.com> wrote:
> On 29/07/2011 1:11 PM, Dietrich Ayala wrote:
>
>> My vote is for embedding Jetpack inside of Firefox for use for
>> developing core front-end features.
>>
>
> ______________________________**_________________
> dev-planning mailing list
> dev-pl...@lists.mozilla.org
> https://lists.mozilla.org/**listinfo/dev-planning<https://lists.mozilla.org/listinfo/dev-planning>
>
Dietrich Ayala
> Sadly, there's a pitfall, and that's that l10n in jetpack is, if at all, in
> its infancy. I'm not sure how far we'll get anytime soon there for
> beyond-easy UI, in particular as long as we don't have chrome registry
> support (AFAICT, we don't).
>
> Not sure how RTL and a11y go here, too.
>
> Axel
Yes, l10n is one of holes in Jetpack today. If we jump into using it
for core feature development, we'll surely be leaning on you and the
rest of the l10n experts for input as we expand and refine Jetpack's
capabilities there.
Dietrich Ayala
>> Maybe it means we ship new features as add-ons that are included with
>> the release (killswitch, anyone?).
>
> with Test Pilot for beta releases, and Mozilla China has done it with a
> bunch of addons for stable releases. And Mozilla has helped to develop,
> release, and market high-quality addons alongside Firefox, including
> Personas, F1, and Kodak EasyShare Gallery Companion.
These are using Jetpack?
>> My vote is for embedding Jetpack inside of Firefox for use for
>> developing core front-end features.
>
> I agree, but it's important to share an understanding of what "embedding
> Jetpack" means and to schedule this work carefully.
>
> For me it means landing the core API implementations and possibly the module
> loader and the bootstrapper into core Firefox, and I think we should do it
> after we finish moving addon execution out-of-process (i.e. e10s), which is
> scheduled for Q3/4 of this year, because e10s is a sea-change for our API
> implementations, and the team will be more productive doing that
> transformative work in the project's own repository.
Yes, there's a bunch of design thinking about this that needs to be done.
Regarding e10s: Feels like we could do a lot of the work side-by-side
the e10s work, instead of waiting until e10s is complete. But as you
said, we can't really tell until we sit down and plot out what
"Jetpack embedded in Firefox" actually means.
Dietrich Ayala
<phil...@googlemail.com> wrote:
> On Fri, Jul 29, 2011 at 10:11 AM, Dietrich Ayala <auto...@gmail.com> wrote:
>> We've learned through experience with Sync, Personas, Account Manager,
>> Panorama and F1 that front-end features developed outside of the
>> Firefox team require massive refactorings/rewrites and long
>> development cycles to transition from an add-on experiment to a
>> feature in the core, and sometimes the feature doesn't land at all.
>> Also, there can be friction between groups as issues of performance,
>> quality, ownership, conflicting timelines and mismatched expectations
>> crop up during these transitions.
>
> Like others have suggested, some of this could be a management
> problem. And to a certain degree it seems to be a development attitude
> problem. Things like coding style and unit tests shouldn't be
> optional. Throwing away experimental code, rewriting things from
> scratch, etc. is OK.
Yes! Jetpack is not a panacea for solving inter-group friction. But
external groups do not need to work under the stringent requirements
that core code does for a variety of reasons, and I don't see that
going away. My hope is that having a modern and desirable development
platform in common with external groups and add-on devs will mitigate
some of these unavoidable friction points.
> I don't think all of the problems we've had in the past were due to
> the fact that we were lacking a JS-friendly (or friendlier) API layer.
> To speak from my experience, Sync needed (and still needs) performant
> APIs that drill way down into our core components. Web Apps, Identity,
> etc. will need stuff like extensible DOM APIs, desktop notifications,
> etc. All of this functionality can be achieved today, in Labs, but it
> would probably not meet the high standards we rightly have for shipped
> code.
Jetpack has all of the above: it can reach through into chrome-space.
What Jetpack brings (among other things) is a module and package
system that help reinforce cleaner abstractions when developing these
features.
> That said, I'm all for creating JS-friendly abstraction layers on top
> of clunky and/or unstable APIs. Jetpack has done a fantastic job of
> prototyping in this space, but I have my doubts it is ready as-is for
> trunk -- not that you were implying it was. As far as I have
> understood Myk in the past, Jetpack was developed mainly with
> developer convenience in mind. That makes sense for an add-on SDK. For
> core stuff, however, the main focus must shift more towards
> performance and efficiency. This means no synchronous I/O, careful use
> of object allocations, etc. This isn't particularly hard to do, it's
> mainly just a change of attitude.
Yes, in order for Jetpack to land, the performance requirements of any
core code will still apply.
Dietrich Ayala
> On 29/07/2011 1:11 PM, Dietrich Ayala wrote:
>>
>> developing core front-end features.
>
> naturally skeptical of any assertion that $future_unfinished_product will be
> so much better that we should commit to building on top of that abstraction
> layer well before it's done.
Correct, I'm not advocating halting all development to make this
happen today. This thread is about planning for a more nimble
development landscape for core feature development in the future,
Jetpack or not.
> Don't get me wrong, I want Jetpack to get done, and to get integrated, and
> to kick ass. But until that day comes, I am not convinced we should adopt a
> Jetpack-centric feature development plan. If that's an end state we want,
> we should define requirements and ensure they're on the Jetpack roadmap, and
> shift to that model once the platform pieces are ready. But we have major
> pieces to ship before that day.
Sure. I don't think Jetpack is the assumed heir for core front-end
development. It's got my vote, but there also aren't any obvious
alternatives to talk about
and I'm biased because I've worked on it a bunch :)
This thread is becoming about Jetpack, but the root problem is that
front-end development is pretty broken. I'd love to hear other visions
of what it could look like to build Firefox (desktop, mobile, and
wherever else we go) in 2-3 years.
And yes, we should do all the things normally done when starting a
project, same as has been done for Jetpack so far :)
> Myk says this work can/should come after e10s, so we're likely talking about
> 6-9 months at least to have a minimum viable platform, and then some amount
> of time after that for someone to build a viable, complex feature on the new
> APIs to prove out the new platform. I still think it's worth committing
> significant resources to this plan, but I'd also like to see us define some
> sort of interim best practice that we can apply right now, without adding a
> dependency on Jetpack to getting critical projects done this year.
Waiting until e10s Jetpack arrives is an example of what this thread
is trying to head off. We can do all of what you recommended above
without waiting for e10s Jetpack to land and without writing any code.
> For the "until Jetpack is deemed ready" timeframe, I'd like to
> counter-propose the following:
>
> 1. Prototype/iterate in whatever way is easiest/fastest to solidify the
> product plan.
> 2. Once we have product requirements and an understanding of what we need to
> build:
> * Build features as restartless add-ons, optionally shipping these via a
> LabKit/"Services Beta Channel" type of experimental feature channel to get
> feedback.
> * Enforce m-c grade test coverage, code review, and quality bars from the
> start of this phase
> * Provide a generic test framework that works in the add-on space, where
> the tests are compatible with the main tree's test frameworks
> * Use modern coding styles, JSMs, sane API design
> * Write UI code in a way that it can be trivially ported into Firefox
> proper
> 3. Once a feature is done enough to include in the mainline, dropping that
> code into Firefox proper shouldn't take more than a few days for one
> developer, since tests, code, and UI bits will need minimal changes.
>
> This is an improvement on what we did late in the Sync endgame, and applies
> the lessons we learned from that experience. And, again, we can start
> applying this approach today, and transition over time to better technology
> stacks as they become available.
This sounds fantastic. However, Identity and Apps are already
in-motion with Jetpack. I guess the next step would be to meet with
those groups to talk about this in context of those specific projects?
David Mason
in to what everyone has said - but I'll give it some thought over the
weekend and add anything I might feel hasn't been mentioned.
Thanks,
Dave
Dietrich Ayala
There is no answer to that question. Jetpack of today, 6 months ago or
6 months from now? Personas or Sync before or after their massive
refactorings?
We are all charged with deciding what Firefox development looks like.
I think it looks pretty painful and slow right now, and Jetpack is one
option for improving it. If you have suggestions for improving Firefox
development, I encourage you to share them.
Dietrich Ayala
> On 7/29/2011 10:11 AM, Dietrich Ayala wrote:
>> We've learned through experience with Sync, Personas, Account Manager,
>> Panorama and F1 that front-end features developed outside of the
>> Firefox team require massive refactorings/rewrites and long
>> development cycles to transition from an add-on experiment to a
>> feature in the core, and sometimes the feature doesn't land at all.
>> quality, ownership, conflicting timelines and mismatched expectations
>> crop up during these transitions.
>
> technology problem. How about putting teams that develop core features
> inside the Firefox team?
Management problem and more! We have groups that have different
approaches, different goals and different timelines, and all are valid
differences. So yeah the same inter-group dynamics that you'll find at
any large organization :)
As I said in my response to Philipp, Jetpack would mitigate some of
these issues, but is not a panacea. Mike Connor's recommendations
would help things in the short term. Even before these
development-related things, we could probably improve communication
between groups with these projects that we are fairly sure will end up
in the product.
> The Jetpack proposal below is great, but beyond
> that, I think "Team A does all the cool work inventing a new feature,
> then Team B does all the grunt work to make it actually work" is a bad
> model for all kinds of reasons.
We've had variations throughout the years. Some projects are handed to
Firefox team to implement "for reals". Some are converted from add-on
to patch, landed and still maintained by the original team that
developed the add-on. We need to be able to handle all permutations of
these scenarios.
Dietrich Ayala
> How about putting teams that develop core features
> inside the Firefox team?
Missed answering this directly on the first round.
That just doesn't scale. We need to be able to handle contributions
from all quarters.
Also see my earlier response to your comment - some of these groups
have valid reasons for working under different sets of requirements.
Changing manager and instituting top-down requirements is not a
reasonable approach in those instances.
Dao
> On Sat, Jul 30, 2011 at 2:27 AM, Dao<d...@design-noir.de> wrote:
>> On 29.07.2011 19:11, Dietrich Ayala wrote:
>>>
>>> We've learned through experience with Sync, Personas, Account Manager,
>>> Panorama and F1 that front-end features developed outside of the
>>> Firefox team require massive refactorings/rewrites and long
>>> development cycles to transition from an add-on experiment to a
>>> feature in the core, and sometimes the feature doesn't land at all.
>>
>>> My vote is for embedding Jetpack inside of Firefox for use for
>>> developing core front-end features.
>>
>> For which of the cited projects would Jetpack have been an option? I'd say
>> none of Sync, Personas, Account Manager and Panorama. F1? I guess so.
>
> There is no answer to that question. Jetpack of today, 6 months ago or
> 6 months from now? Personas or Sync before or after their massive
> refactorings?
Let's say 6 months from now. I'm actually interested in both before and
after their refactorings. The refactorings happened for certain reasons
and I'd like to know if using Jetpack from the start would have ruled
these problems out. The post-refactoring state alone is also interesting
as I'd like to know if a Jetpack-based implementation could even
theoretically deliver the same quality with the same or less effort.
Personas was implemented from scratch for stock Firefox because:
1) the parts of the add-on's back-end we would have been interested in
were tied to things we weren't interested in, and
2) for Personas to reach product quality, it needed integrated theme
support:
http://mxr.mozilla.org/mozilla-central/search?string=%3A-moz-lwtheme
(It also needed integrated add-on manager support, as the manager wasn't
as extensible as it supposedly is today.)
The Sync refactoring I'm not sure about, but AFAIK it was first and
foremost about the back-end rather than the front-end, and a Jetpack
would probably have used a very similar back-end.
Generally, I think you're saying we want less redundant and wasted work
but the same quality we would have gotten otherwise, but I don't see how
Jetpack gets us there for cases like those experienced by us.
Myk Melez
> On Sat, Jul 30, 2011 at 3:01 AM, Myk Melez<m...@mozilla.org> wrote:
>> This is a particularly promising avenue for integration. We've done this
>> with Test Pilot for beta releases, and Mozilla China has done it with a
>> bunch of addons for stable releases. And Mozilla has helped to develop,
>> release, and market high-quality addons alongside Firefox, including
>> Personas, F1, and Kodak EasyShare Gallery Companion.
> These are using Jetpack?
Firefox generally.
-myk
Jean-Marc Desperrier
> The mix of XUL/XPCOM/XBL/JS/CSS that we currently
> use for core feature development is powerful yet extremely
> regression-prone, and painstaking to develop against.
I recently was wondering if, for the short term, rewritting the core of
Firefox on top of HTML5 wouldn't make more sense and be easier to reach
than B2G for Mozilla ...
BTW isn't this whole discussion more platform than planning ?
Asa Dotzler
Jean-Marc Desperrier wrote:
> On 29/07/2011 19:11, Dietrich Ayala wrote:
>> The mix of XUL/XPCOM/XBL/JS/CSS that we currently
>> use for core feature development is powerful yet extremely
>> regression-prone, and painstaking to develop against.
>
> I recently was wondering if, for the short term, rewritting the core of
> Firefox on top of HTML5 wouldn't make more sense and be easier to reach
> than B2G for Mozilla ...
Firefox and B2G are different things. B2G isn't trying to be a Web
browser. B2G is plan for a platform and not a plan for a piece of client
software. Firefox is client software and re-writing it with purely Web
technologies would be a pretty huge undertaking and not something "short
term".
> BTW isn't this whole discussion more platform than planning ?
Planning is a fine place for people to share what their plans are and I
think that's what the B2G folks were trying to do here.
Asa Dotzler
Oops, that second part didn't involve B2G. Sorry. I've had too many
threads going at once here.
Dietrich's thread was very much in line with "planning". It may have
wandered off some into platform talk, but it started as a discussion
around coordination and tools and how we want to design and build Firefox.
- A
Blair McBride
> We've learned through experience with Sync, Personas, Account Manager,
> Panorama and F1 that front-end features developed outside of the
> Firefox team require massive refactorings/rewrites and long
> development cycles to transition from an add-on experiment to a
> feature in the core, and sometimes the feature doesn't land at all.
> quality, ownership, conflicting timelines and mismatched expectations
> crop up during these transitions.
In all of those cases, they started out as experiments and prototypes
that added something on top of Firefox, and tried to directly transition
to a production-ready feature. Shipping prototype code doesn't work.
Jetpack is great, and it should be in Firefox - but it's not going to
make prototypes shippable.
- Blair
Robert Kaiser
> There were a bunch of problems
> that we faced during the integration phase
All of which would IMHO not have been too different if you would have
been based on Jetpack.
All of those are important points I have seen with a lot of things that
have been prototyped in some way, usually as add-ons, and then later
integrated. Even Firefox itself suffered a lot from a number of those
points in its early stage, before it became a mainline project.
IMHO, those are all points of both culture and rapid prototyping, and
independent of the technology.
> If I have to share the lessons that I learned from this efforts, I can
> summarize them as:
>
> 1. Getting feedback from users/developers early on is valuable. [...]
>
> 2. Code quality should never be an after-thought. [...]
Good lessons for all of us, right.
I for myself spent a lot of time writing automated tests for my Tahoe
Data Manager add-on, and that tremendously helped code quality by
finding a number of bugs I probably would not have seen else, and it (in
addition to following common Mozilla code style from the beginning)
surely made the integration into SeaMonkey smoother.
And I think those things are more to the core of getting stuff
integrated into mainline code more easily. Of course, spending a lot of
time writing tests and writing your code to be clean and possibly even
performant from the beginning may conflict with rapid prototyping, but
you always need to keep in mind that you need to get there some time
anyhow if your target is to include the feature in the mainline.
Those things are independent of the technology though, no matter if we
have Jetpack in the mainline or not, you can prototype fast without
tests and with ugly hacks and later rewrite or start off a bit slower
but with tests and clean code from the beginning. Your choice as a
developer - just don't complain afterwards that you haven't been warned. ;-)
Ron Hunter
Hasn't it always been the plan to incorporate only those features into
the base code of Firefox that would be used by the majority of users?
Is this no longer the policy? I surely don't want to see Firefox become
another ADA which was designed by a large committee, and they put in
everything anyone suggested. How many 'users' would ever use the
Jetpack code? I already have to tote the load for all of the 3% of
users who find the incorporation of Tab Candy into the mainline code.
What's next?
Back to lurking.
