Firefox front-end features, time-to-market, and coordination.

402 views
Skip to first unread message

Dietrich Ayala

unread,
Jul 29, 2011, 1:11:33 PM7/29/11
to mozilla. dev. planning group, Todd Simpson, David Mason, Mike Connor, Jay Sullivan, Dave Townsend, Myk Melez
We've learned through experience with Sync, Personas, Account Manager,
Panorama and F1 that front-end features developed outside of the
Firefox team require massive refactorings/rewrites and long
development cycles to transition from an add-on experiment to a
feature in the core, and sometimes the feature doesn't land at all.
Also, there can be friction between groups as issues of performance,
quality, ownership, conflicting timelines and mismatched expectations
crop up during these transitions.

We need to reduce the friction and time that it takes to get these
types of projects into Firefox.

Currently both the Apps and Identity teams are using Jetpack as their
development platform. Jetpack is fantastic for rapid prototyping but
is targeted at lightweight add-on development, not ship-quality
browser core features. It's also not in Firefox.

The product road map for Apps and Identity is not specific about
timeline, but says they're included in what the Web looks like in 24 -
36 months. Given the competitive atmosphere and the historical
evidence for adding many months onto the schedule to get projects like
these shipping in Firefox, I think we need to look at ways to reduce
the time-to-integration for these kinds of projects.

Maybe this means immediately putting resources behind embedding
Jetpack in Firefox such that we can write core features on top of it.
Maybe it means we ship new features as add-ons that are included with
the release (killswitch, anyone?). Maybe it means we move these
projects off of Jetpack and into the core at the earliest possible
stage.

But no matter when the code lands in core, we're still hampered by a
fragile ecosystem: The mix of XUL/XPCOM/XBL/JS/CSS that we currently
use for core feature development is powerful yet extremely
regression-prone, and painstaking to develop against. Writing
shippable code is only possible by specialists with experiential
knowledge of a ton of undocumented code and a host of implicit
behaviors that are impossible to predict.

My vote is for embedding Jetpack inside of Firefox for use for
developing core front-end features. Having a stable and sane pure-JS
API layer to build new features against would:

* make it easier to share code between desktop and mobile
* ease integration of projects developed by external teams (and allow
possible code-sharing there too)
* ease contribution by volunteer developers, and ramp-up by new employees
* allow e10s-ification of core features using the same mechanisms and
patterns as add-ons
* strengthen Jetpack by ensuring it's APIs are up to the level of
stability and performance required to be in the core

Sure, it wouldn't work for everything - we'd still have to dive under
the covers for a bunch of things. Sure, we'll have to figure out a
host of issues around the interplay between shipped-SDK and
embedded-SDK. But I think it will allow front-end development to move
faster and more effectively in the long term.

Axel Hecht

unread,
Jul 29, 2011, 1:24:44 PM7/29/11
to

Yes to most of all of the above.

Sadly, there's a pitfall, and that's that l10n in jetpack is, if at all,
in its infancy. I'm not sure how far we'll get anytime soon there for
beyond-easy UI, in particular as long as we don't have chrome registry
support (AFAICT, we don't).

Not sure how RTL and a11y go here, too.

Axel

Dao

unread,
Jul 29, 2011, 1:27:05 PM7/29/11
to
On 29.07.2011 19:11, Dietrich Ayala wrote:
> We've learned through experience with Sync, Personas, Account Manager,
> Panorama and F1 that front-end features developed outside of the
> Firefox team require massive refactorings/rewrites and long
> development cycles to transition from an add-on experiment to a
> feature in the core, and sometimes the feature doesn't land at all.

> My vote is for embedding Jetpack inside of Firefox for use for
> developing core front-end features.

For which of the cited projects would Jetpack have been an option? I'd
say none of Sync, Personas, Account Manager and Panorama. F1? I guess so.

Rob Campbell

unread,
Jul 29, 2011, 1:33:17 PM7/29/11
to Dietrich Ayala, Todd Simpson, mozilla. dev. planning group, Mike Connor, Jay Sullivan, Dave Townsend, David Mason, Myk Melez

On 2011-07-29, at 14:11 , Dietrich Ayala wrote:

> Maybe this means immediately putting resources behind embedding
> Jetpack in Firefox such that we can write core features on top of it.
> Maybe it means we ship new features as add-ons that are included with
> the release (killswitch, anyone?). Maybe it means we move these
> projects off of Jetpack and into the core at the earliest possible
> stage.

+1.

We looked hard at using Jetpack for devtools features and determined that it would've been just too difficult to do 6 months ago, and most of the issues we had then are still in place today.

We looked at what it would take to package a jetpack into the browser without having a baseline add-ons codebase in the browser itself and that, coupled with testing uncertainties and build-time packaging just made the whole process that much more daunting.

It meant most of the features we would write as Jetpacks would either remain as dissociated jetpacks living on AMO (not necessarily a bad thing, but that came with a set of limitations) or we'd have to build up a bunch of test infrastructure and coordinate with the addons teams to attempt to integrate features back into the browser.

Having an included version of Jetpack in the browser means a lot of that test-infrastructure and build-time uncertainty becomes a lot easier to deal with. Not to mention eliminates a lot of the compatibility concerns you run into with multiple versions of the SDK floating around in the ecosphere.

> But no matter when the code lands in core, we're still hampered by a
> fragile ecosystem: The mix of XUL/XPCOM/XBL/JS/CSS that we currently
> use for core feature development is powerful yet extremely
> regression-prone, and painstaking to develop against. Writing
> shippable code is only possible by specialists with experiential
> knowledge of a ton of undocumented code and a host of implicit
> behaviors that are impossible to predict.
>

> My vote is for embedding Jetpack inside of Firefox for use for

> developing core front-end features. Having a stable and sane pure-JS
> API layer to build new features against would:
>
> * make it easier to share code between desktop and mobile
> * ease integration of projects developed by external teams (and allow
> possible code-sharing there too)
> * ease contribution by volunteer developers, and ramp-up by new employees
> * allow e10s-ification of core features using the same mechanisms and
> patterns as add-ons
> * strengthen Jetpack by ensuring it's APIs are up to the level of
> stability and performance required to be in the core

And
* allow easy integration of existing test and build infrastructure.
* Make it possible to write well-tested code based on jetpack that ships in the browser.
* Make it possible to integrate jetpacks "as-is" without doing a total rewrite at integration time.

> Sure, it wouldn't work for everything - we'd still have to dive under
> the covers for a bunch of things. Sure, we'll have to figure out a
> host of issues around the interplay between shipped-SDK and
> embedded-SDK. But I think it will allow front-end development to move
> faster and more effectively in the long term.

All for it!

~ rob

Myk Melez

unread,
Jul 29, 2011, 2:01:12 PM7/29/11
to Dietrich Ayala, Todd Simpson, mozilla. dev. planning group, Mike Connor, Jay Sullivan, Dave Townsend, David Mason
On 2011-07-29 10:11, Dietrich Ayala wrote:
> We need to reduce the friction and time that it takes to get these
> types of projects into Firefox.
++

> Currently both the Apps and Identity teams are using Jetpack as their
> development platform. Jetpack is fantastic for rapid prototyping but
> is targeted at lightweight add-on development, not ship-quality
> browser core features.

That's true, but it's changing over time.

About six months ago, philikon asked me if he should use the SDK to
build an experimental feature that he eventually intended to land in
Firefox, and I told him "no," because our primary target audience was
casual developers building simple addons.

When Apps folks asked me the same question last month, my answer was
"maybe," since our secondary target audience is professional developers
building complex addons, and we're starting to expand our focus to their
use cases as part of our push to get traditional addon developers to
transition to the SDK.

And in six months, I hope the answer will be "yes," because we will have
addressed the various issues associated with SDK development of complex
addons.

> Maybe it means we ship new features as add-ons that are included with
> the release (killswitch, anyone?).

This is a particularly promising avenue for integration. We've done this
with Test Pilot for beta releases, and Mozilla China has done it with a
bunch of addons for stable releases. And Mozilla has helped to develop,
release, and market high-quality addons alongside Firefox, including
Personas, F1, and Kodak EasyShare Gallery Companion.

> My vote is for embedding Jetpack inside of Firefox for use for
> developing core front-end features.

I agree, but it's important to share an understanding of what "embedding
Jetpack" means and to schedule this work carefully.

For me it means landing the core API implementations and possibly the
module loader and the bootstrapper into core Firefox, and I think we
should do it after we finish moving addon execution out-of-process (i.e.
e10s), which is scheduled for Q3/4 of this year, because e10s is a
sea-change for our API implementations, and the team will be more
productive doing that transformative work in the project's own repository.

-myk

David Mandelin

unread,
Jul 29, 2011, 2:05:28 PM7/29/11
to dev-pl...@lists.mozilla.org
On 7/29/2011 10:11 AM, Dietrich Ayala wrote:
> We've learned through experience with Sync, Personas, Account Manager,
> Panorama and F1 that front-end features developed outside of the
> Firefox team require massive refactorings/rewrites and long
> development cycles to transition from an add-on experiment to a
> feature in the core, and sometimes the feature doesn't land at all.
> Also, there can be friction between groups as issues of performance,
> quality, ownership, conflicting timelines and mismatched expectations
> crop up during these transitions.
To me, the above reads as even more of a management problem than a
technology problem. How about putting teams that develop core features
inside the Firefox team? The Jetpack proposal below is great, but beyond
that, I think "Team A does all the cool work inventing a new feature,
then Team B does all the grunt work to make it actually work" is a bad
model for all kinds of reasons.

> We need to reduce the friction and time that it takes to get these
> types of projects into Firefox.
Yes.

> Maybe this means immediately putting resources behind embedding
> Jetpack in Firefox such that we can write core features on top of it.
> Maybe it means we ship new features as add-ons that are included with
> the release (killswitch, anyone?). Maybe it means we move these
> projects off of Jetpack and into the core at the earliest possible
> stage.
>
> But no matter when the code lands in core, we're still hampered by a
> fragile ecosystem: The mix of XUL/XPCOM/XBL/JS/CSS that we currently
> use for core feature development is powerful yet extremely
> regression-prone, and painstaking to develop against. Writing
> shippable code is only possible by specialists with experiential
> knowledge of a ton of undocumented code and a host of implicit
> behaviors that are impossible to predict.
That's terrible. Let's correct that now.

> My vote is for embedding Jetpack inside of Firefox for use for
> developing core front-end features.
I agree very much.

> Having a stable and sane pure-JS
> API layer to build new features against would:
>
> * make it easier to share code between desktop and mobile
> * ease integration of projects developed by external teams (and allow
> possible code-sharing there too)
> * ease contribution by volunteer developers, and ramp-up by new employees
> * allow e10s-ification of core features using the same mechanisms and
> patterns as add-ons
> * strengthen Jetpack by ensuring it's APIs are up to the level of
> stability and performance required to be in the core
Embedding Jetpack would also:

* get Jetpack development properly integrated with Gecko and on
Tinderbox, so Jetpack doesn't get randomly broken by changes to Gecko
and SpiderMonkey.


> Sure, it wouldn't work for everything - we'd still have to dive under
> the covers for a bunch of things.

Can we try to extend Jetpack instead, as much as possible? As you say
above, Jetpack has the potential to be a far superior addon API to what
we have now.


> Sure, we'll have to figure out a
> host of issues around the interplay between shipped-SDK and
> embedded-SDK. But I think it will allow front-end development to move
> faster and more effectively in the long term.

Dave

Dave Townsend

unread,
Jul 29, 2011, 2:06:16 PM7/29/11
to Dietrich Ayala, Todd Simpson, mozilla. dev. planning group, Mike Connor, Jay Sullivan, David Mason, Myk Melez
On 29 July 2011 10:12, Dietrich Ayala <auto...@gmail.com> wrote:

> We need to reduce the friction and time that it takes to get these
> types of projects into Firefox.
>

Wholeheartedly agree.


> Currently both the Apps and Identity teams are using Jetpack as their
> development platform. Jetpack is fantastic for rapid prototyping but
> is targeted at lightweight add-on development, not ship-quality
> browser core features.


I'd love to hear what you think are the differences there. Ignoring the
embedding piece for now what is Jetpack missing that is required for core
feature development?


> The product road map for Apps and Identity is not specific about
> timeline, but says they're included in what the Web looks like in 24 -
> 36 months. Given the competitive atmosphere and the historical
> evidence for adding many months onto the schedule to get projects like
> these shipping in Firefox, I think we need to look at ways to reduce
> the time-to-integration for these kinds of projects.
>

> Maybe this means immediately putting resources behind embedding
> Jetpack in Firefox such that we can write core features on top of it.
> Maybe it means we ship new features as add-ons that are included with
> the release (killswitch, anyone?). Maybe it means we move these
> projects off of Jetpack and into the core at the earliest possible
> stage.
>

We've been actively talking with many of the teams you mention about this
very issue over the past few days, so far I haven't heard their dependence
on Jetpack as being a part that delays their release, but of course when
they are ready Jetpack needs to be ready for them too.

As you mention there are multiple ways we could ship features based on
Jetpack but I don't think we need to tie ourselves to just one of those
ways. My guess is that a fine first step would be to ship these features as
add-ons and then move on from there. It's my plan to spend some time over
the next week or so figuring out what barriers might be in place for us to
be able to do that.

David Dahl

unread,
Jul 29, 2011, 2:08:27 PM7/29/11
to Myk Melez, Todd Simpson, David Mason, Mike Connor, Dietrich Ayala, Jay Sullivan, Dave Townsend, mozilla. dev. planning group
----- Original Message -----
> From: "Myk Melez" <m...@mozilla.org>
> To: "Dietrich Ayala" <auto...@gmail.com>
> Cc: "Todd Simpson" <to...@mozilla.com>, "mozilla. dev. planning group" <dev-pl...@lists.mozilla.org>, "Mike Connor"
> <mco...@mozilla.com>, "Jay Sullivan" <jsul...@mozilla.com>, "Dave Townsend" <mos...@mozilla.com>, "David Mason"
> <d...@mozilla.com>
> Sent: Friday, July 29, 2011 1:01:12 PM
> Subject: Re: Firefox front-end features, time-to-market, and coordination.

> And in six months, I hope the answer will be "yes," because we will
> have
> addressed the various issues associated with SDK development of
> complex
> addons.

Is there a roadmap/wikipage/list somewhere that enumerates the holes in the API to make the Addon SDK an avenue for core features? (I realize this has been discussed here and there, and may not have been a goal for the Addon SDK). Has anyone thought about running some analytics/datamining tools on the source of the most used interfaces on the top x number of traditional addons?

Regards,

David

Philipp von Weitershausen

unread,
Jul 29, 2011, 2:34:44 PM7/29/11
to Dietrich Ayala, Todd Simpson, mozilla. dev. planning group, Mike Connor, Jay Sullivan, Dave Townsend, David Mason, Myk Melez
Thanks for kicking off this discussion, Dietrich. I've been wanting to
do a brownbag or something simliar about this for a while. The
Services team was involved in some of the projects you mentioned below
(specifically Sync, F1). Feel free to rope us in if you need some
detailed feedback!


On Fri, Jul 29, 2011 at 10:11 AM, Dietrich Ayala <auto...@gmail.com> wrote:
> We've learned through experience with Sync, Personas, Account Manager,
> Panorama and F1 that front-end features developed outside of the
> Firefox team require massive refactorings/rewrites and long
> development cycles to transition from an add-on experiment to a
> feature in the core, and sometimes the feature doesn't land at all.
> Also, there can be friction between groups as issues of performance,
> quality, ownership, conflicting timelines and mismatched expectations
> crop up during these transitions.

Like others have suggested, some of this could be a management
problem. And to a certain degree it seems to be a development attitude
problem. Things like coding style and unit tests shouldn't be
optional. Throwing away experimental code, rewriting things from
scratch, etc. is OK.

> The product road map for Apps and Identity is not specific about
> timeline, but says they're included in what the Web looks like in 24 -
> 36 months. Given the competitive atmosphere and the historical
> evidence for adding many months onto the schedule to get projects like
> these shipping in Firefox, I think we need to look at ways to reduce
> the time-to-integration for these kinds of projects.

Absolutely. And without compromising on performance and stability.

> Maybe this means immediately putting resources behind embedding
> Jetpack in Firefox such that we can write core features on top of it.
> Maybe it means we ship new features as add-ons that are included with
> the release (killswitch, anyone?). Maybe it means we move these
> projects off of Jetpack and into the core at the earliest possible
> stage.
>

> But no matter when the code lands in core, we're still hampered by a
> fragile ecosystem: The mix of XUL/XPCOM/XBL/JS/CSS that we currently
> use for core feature development is powerful yet extremely
> regression-prone, and painstaking to develop against. Writing
> shippable code is only possible by specialists with experiential
> knowledge of a ton of undocumented code and a host of implicit
> behaviors that are impossible to predict.

I don't think all of the problems we've had in the past were due to
the fact that we were lacking a JS-friendly (or friendlier) API layer.
To speak from my experience, Sync needed (and still needs) performant
APIs that drill way down into our core components. Web Apps, Identity,
etc. will need stuff like extensible DOM APIs, desktop notifications,
etc. All of this functionality can be achieved today, in Labs, but it
would probably not meet the high standards we rightly have for shipped
code.

That said, I'm all for creating JS-friendly abstraction layers on top
of clunky and/or unstable APIs. Jetpack has done a fantastic job of
prototyping in this space, but I have my doubts it is ready as-is for
trunk -- not that you were implying it was. As far as I have
understood Myk in the past, Jetpack was developed mainly with
developer convenience in mind. That makes sense for an add-on SDK. For
core stuff, however, the main focus must shift more towards
performance and efficiency. This means no synchronous I/O, careful use
of object allocations, etc. This isn't particularly hard to do, it's
mainly just a change of attitude.

> My vote is for embedding Jetpack inside of Firefox for use for

> developing core front-end features. Having a stable and sane pure-JS


> API layer to build new features against would:

+1 to this and all of the points below.

Mike Connor

unread,
Jul 29, 2011, 2:42:22 PM7/29/11
to Dietrich Ayala, Todd Simpson, mozilla. dev. planning group, Jay Sullivan, Dave Townsend, David Mason, Myk Melez
On 29/07/2011 1:11 PM, Dietrich Ayala wrote:
> My vote is for embedding Jetpack inside of Firefox for use for
> developing core front-end features.

I'm in favour of this, as a long-term play. But unsurprisingly, I'm
naturally skeptical of any assertion that $future_unfinished_product
will be so much better that we should commit to building on top of that
abstraction layer well before it's done.

Don't get me wrong, I want Jetpack to get done, and to get integrated,
and to kick ass. But until that day comes, I am not convinced we should
adopt a Jetpack-centric feature development plan. If that's an end
state we want, we should define requirements and ensure they're on the
Jetpack roadmap, and shift to that model once the platform pieces are
ready. But we have major pieces to ship before that day.

Myk says this work can/should come after e10s, so we're likely talking
about 6-9 months at least to have a minimum viable platform, and then
some amount of time after that for someone to build a viable, complex
feature on the new APIs to prove out the new platform. I still think
it's worth committing significant resources to this plan, but I'd also
like to see us define some sort of interim best practice that we can
apply right now, without adding a dependency on Jetpack to getting
critical projects done this year.

For the "until Jetpack is deemed ready" timeframe, I'd like to
counter-propose the following:

1. Prototype/iterate in whatever way is easiest/fastest to solidify the
product plan.
2. Once we have product requirements and an understanding of what we
need to build:
* Build features as restartless add-ons, optionally shipping these
via a LabKit/"Services Beta Channel" type of experimental feature
channel to get feedback.
* Enforce m-c grade test coverage, code review, and quality bars from
the start of this phase
* Provide a generic test framework that works in the add-on space,
where the tests are compatible with the main tree's test frameworks
* Use modern coding styles, JSMs, sane API design
* Write UI code in a way that it can be trivially ported into Firefox
proper
3. Once a feature is done enough to include in the mainline, dropping
that code into Firefox proper shouldn't take more than a few days for
one developer, since tests, code, and UI bits will need minimal changes.

This is an improvement on what we did late in the Sync endgame, and
applies the lessons we learned from that experience. And, again, we can
start applying this approach today, and transition over time to better
technology stacks as they become available.

-- Mike

Robert Kaiser

unread,
Jul 29, 2011, 3:08:06 PM7/29/11
to
Dietrich Ayala schrieb:

> My vote is for embedding Jetpack inside of Firefox for use for
> developing core front-end features. Having a stable and sane pure-JS
> API layer to build new features against would:

Removing markup (XUL/XBL right now, hopefully nobody even thinks about
removing CSS, or do you?) and making everything JS does surely
incredibly slow down any somewhat larger UI and makes code almost
unreadable and uncustomizable. I don't think that's a useful path to
anything. You also don't write large websites as one large JS script
that builds the DOM and styles all dynamically, right?

I'm not at all against bringing Jetpack in, but strongly against making
all UI elements build solely in JS. That sounds like a big mess and
would make contribution much harder. Markup like XUL is way easier to
read than figuring out how things get structured from a large JS script.

Robert Kaiser


--
Note that any statements of mine - no matter how passionate - are never
meant to be offensive but very often as food for thought or possible
arguments that we as a community should think about. And most of the
time, I even appreciate irony and fun! :)

David Dahl

unread,
Jul 29, 2011, 3:53:56 PM7/29/11
to Robert Kaiser, dev-pl...@lists.mozilla.org
----- Original Message -----
> From: "Robert Kaiser" <ka...@kairo.at>
> To: dev-pl...@lists.mozilla.org
> Sent: Friday, July 29, 2011 2:08:06 PM
> Subject: Re: Firefox front-end features, time-to-market, and coordination.
> Dietrich Ayala schrieb:
> > My vote is for embedding Jetpack inside of Firefox for use for
> > developing core front-end features. Having a stable and sane pure-JS
> > API layer to build new features against would:
>
> Removing markup (XUL/XBL right now, hopefully nobody even thinks about
> removing CSS, or do you?) and making everything JS does surely
> incredibly slow down any somewhat larger UI and makes code almost
> unreadable and uncustomizable. I don't think that's a useful path to
> anything. You also don't write large websites as one large JS script
> that builds the DOM and styles all dynamically, right?
>
> I'm not at all against bringing Jetpack in, but strongly against
> making
> all UI elements build solely in JS. That sounds like a big mess and
> would make contribution much harder. Markup like XUL is way easier to
> read than figuring out how things get structured from a large JS
> script.

I'm pretty sure Dietrich means "Non-native code" when he referenced "pure-JS". You can load CSS, html and other types of files into a Jetpack Addon.

Regards,

David

Nicholas Nethercote

unread,
Jul 29, 2011, 5:03:00 PM7/29/11
to Philipp von Weitershausen, Todd Simpson, mozilla. dev. planning group, Mike Connor, Dietrich Ayala, Jay Sullivan, Dave Townsend, David Mason, Myk Melez
On Sat, Jul 30, 2011 at 4:34 AM, Philipp von Weitershausen
<phil...@googlemail.com> wrote:

> For core stuff, however, the main focus must shift more towards
> performance and efficiency.

Yes. For example, Jetpack's memory usage isn't great at the moment, see
https://bugzilla.mozilla.org/show_bug.cgi?id=672443 for an example.
There's also a known leak but I can't find the bug now.

Nick

Ehsan Akhgari

unread,
Jul 29, 2011, 5:58:06 PM7/29/11
to Mike Connor, Todd Simpson, David Mason, Dietrich Ayala, Jay Sullivan, Dave Townsend, mozilla. dev. planning group, Myk Melez
Firstly, I would like to second mconnor's point below. I really hope that
one day we get Jetpack integrated into the core Firefox, and for that to
happen, a number of architectural changes (like e10s awareness) and quality
changes (addressing memory-related problems, and stuff like that) need to
happen first. I don't think that has anything to do with developing
prototypes for new projects with a clear integration path with the core
Firefox.

I would like to speak from my experience helping the Panorama team integrate
their code into Firefox. Panorama (which was called TabCandy back in the
day) was developed in a very small team. There were a bunch of problems
that we faced during the integration phase:

1. No automated testing framework. This meant that the code had a lot of
bugs, and nobody knew about them. The basic stuff worked, but lots of edge
cases had weird problems. It took us a while before we managed to fix the
pile of bugs that people filed initially after landing and getting unit
tests for them to bring the test coverage to an acceptable level.

2. Performance problems. The TabCandy add-on was written without much
attention to performance. It affected startup time even if you never used
the feature, and it also had performance problems in some operations (most
notably, animations).

3. Coding style. The coding style used in TabCandy was *completely*
different to what the front-end people used in Firefox/Toolkit. This caused
a lot of pain for people who wanted to help (like me) to understand the
code, and also for the reviewers to make sense out of the code. There was a
basic level of documentation, but everything was so different that looking
at TabCandy code was not easy by the average front-end person.

Here's how we addressed these problems. The lack of automated testing hurt
us really badly. The Panorama team spent a lot of time to write unit tests
to cover the basic functionality, and since the time that Panorama landed on
mozilla-central, we required unit tests per every check-in, similar to the
rest of Firefox code. The performance problem we mostly addressed by
putting the Panorama team in contact with a Platform person (me in this
case). I had enough experience with the platform in general that I could
help the Panorama team understand why their code has performance problems,
and what to do about them. Part of this effort was focused before landing
Panorama for the first time (such as bringing down the Ts regression to 0)
and part of this was performed after the code had landed (such as improving
the animation performance). The third problem was never really addressed.

If I have to share the lessons that I learned from this efforts, I can
summarize them as:

1. Getting feedback from users/developers early on is valuable. As mconnor
mentions, creating new features as restartless add-ons with the
release-early-release-often model is extremely valuable. I think that
feedback should be collected from both users and developers, since the
latter group can help the team working on New Shiny Feature to understand
some of the potential integration/performance problems from early on, and
work on addressing them before it's too late.

2. Code quality should never be an after-thought. While I understand that
having unit tests per check-in might be too strict of a requirement for
early prototyping, I think automated testing is a must at least after the
goals for a project are set. Sure, things will change, and tests need to be
rewritten from scratch, but I think that there is very real value in the
getting higher quality code by our common testing practices, at least from
the point where we get serious about the project.

Hope this helps,
--
Ehsan
<http://ehsanakhgari.org/>


On Fri, Jul 29, 2011 at 2:42 PM, Mike Connor <mco...@mozilla.com> wrote:

> On 29/07/2011 1:11 PM, Dietrich Ayala wrote:
>

>> My vote is for embedding Jetpack inside of Firefox for use for
>> developing core front-end features.
>>
>

> ______________________________**_________________
> dev-planning mailing list
> dev-pl...@lists.mozilla.org
> https://lists.mozilla.org/**listinfo/dev-planning<https://lists.mozilla.org/listinfo/dev-planning>
>

Dietrich Ayala

unread,
Jul 30, 2011, 9:00:58 AM7/30/11
to Axel Hecht, dev-pl...@lists.mozilla.org
On Sat, Jul 30, 2011 at 2:24 AM, Axel Hecht <l1...@mozilla.com> wrote:
> Sadly, there's a pitfall, and that's that l10n in jetpack is, if at all, in
> its infancy. I'm not sure how far we'll get anytime soon there for
> beyond-easy UI, in particular as long as we don't have chrome registry
> support (AFAICT, we don't).
>
> Not sure how RTL and a11y go here, too.
>
> Axel

Yes, l10n is one of holes in Jetpack today. If we jump into using it
for core feature development, we'll surely be leaning on you and the
rest of the l10n experts for input as we expand and refine Jetpack's
capabilities there.

Dietrich Ayala

unread,
Jul 30, 2011, 9:10:02 AM7/30/11
to Myk Melez, Todd Simpson, mozilla. dev. planning group, Mike Connor, Jay Sullivan, Dave Townsend, David Mason
On Sat, Jul 30, 2011 at 3:01 AM, Myk Melez <m...@mozilla.org> wrote:
>> Maybe it means we ship new features as add-ons that are included with
>> the release (killswitch, anyone?).
>
> This is a particularly promising avenue for integration. We've done this
> with Test Pilot for beta releases, and Mozilla China has done it with a
> bunch of addons for stable releases. And Mozilla has helped to develop,
> release, and market high-quality addons alongside Firefox, including
> Personas, F1, and Kodak EasyShare Gallery Companion.

These are using Jetpack?

>> My vote is for embedding Jetpack inside of Firefox for use for
>> developing core front-end features.
>

> I agree, but it's important to share an understanding of what "embedding
> Jetpack" means and to schedule this work carefully.
>
> For me it means landing the core API implementations and possibly the module
> loader and the bootstrapper into core Firefox, and I think we should do it
> after we finish moving addon execution out-of-process (i.e. e10s), which is
> scheduled for Q3/4 of this year, because e10s is a sea-change for our API
> implementations, and the team will be more productive doing that
> transformative work in the project's own repository.

Yes, there's a bunch of design thinking about this that needs to be done.

Regarding e10s: Feels like we could do a lot of the work side-by-side
the e10s work, instead of waiting until e10s is complete. But as you
said, we can't really tell until we sit down and plot out what
"Jetpack embedded in Firefox" actually means.

Dietrich Ayala

unread,
Jul 30, 2011, 9:47:01 AM7/30/11
to Philipp von Weitershausen, Todd Simpson, mozilla. dev. planning group, Mike Connor, Jay Sullivan, Dave Townsend, David Mason, Myk Melez
On Sat, Jul 30, 2011 at 3:34 AM, Philipp von Weitershausen
<phil...@googlemail.com> wrote:
> On Fri, Jul 29, 2011 at 10:11 AM, Dietrich Ayala <auto...@gmail.com> wrote:
>> We've learned through experience with Sync, Personas, Account Manager,
>> Panorama and F1 that front-end features developed outside of the
>> Firefox team require massive refactorings/rewrites and long
>> development cycles to transition from an add-on experiment to a
>> feature in the core, and sometimes the feature doesn't land at all.
>> Also, there can be friction between groups as issues of performance,
>> quality, ownership, conflicting timelines and mismatched expectations
>> crop up during these transitions.
>
> Like others have suggested, some of this could be a management
> problem. And to a certain degree it seems to be a development attitude
> problem. Things like coding style and unit tests shouldn't be
> optional. Throwing away experimental code, rewriting things from
> scratch, etc. is OK.

Yes! Jetpack is not a panacea for solving inter-group friction. But
external groups do not need to work under the stringent requirements
that core code does for a variety of reasons, and I don't see that
going away. My hope is that having a modern and desirable development
platform in common with external groups and add-on devs will mitigate
some of these unavoidable friction points.

> I don't think all of the problems we've had in the past were due to
> the fact that we were lacking a JS-friendly (or friendlier) API layer.
> To speak from my experience, Sync needed (and still needs) performant
> APIs that drill way down into our core components. Web Apps, Identity,
> etc. will need stuff like extensible DOM APIs, desktop notifications,
> etc. All of this functionality can be achieved today, in Labs, but it
> would probably not meet the high standards we rightly have for shipped
> code.

Jetpack has all of the above: it can reach through into chrome-space.
What Jetpack brings (among other things) is a module and package
system that help reinforce cleaner abstractions when developing these
features.

> That said, I'm all for creating JS-friendly abstraction layers on top
> of clunky and/or unstable APIs. Jetpack has done a fantastic job of
> prototyping in this space, but I have my doubts it is ready as-is for
> trunk -- not that you were implying it was. As far as I have
> understood Myk in the past, Jetpack was developed mainly with

> developer convenience in mind. That makes sense for an add-on SDK. For


> core stuff, however, the main focus must shift more towards

> performance and efficiency. This means no synchronous I/O, careful use
> of object allocations, etc. This isn't particularly hard to do, it's
> mainly just a change of attitude.

Yes, in order for Jetpack to land, the performance requirements of any
core code will still apply.

Dietrich Ayala

unread,
Jul 30, 2011, 10:35:17 AM7/30/11
to Mike Connor, Todd Simpson, mozilla. dev. planning group, Jay Sullivan, Dave Townsend, David Mason, Myk Melez
On Sat, Jul 30, 2011 at 3:42 AM, Mike Connor <mco...@mozilla.com> wrote:
> On 29/07/2011 1:11 PM, Dietrich Ayala wrote:
>>
>> My vote is for embedding Jetpack inside of Firefox for use for
>> developing core front-end features.
>
> I'm in favour of this, as a long-term play.   But unsurprisingly, I'm
> naturally skeptical of any assertion that $future_unfinished_product will be
> so much better that we should commit to building on top of that abstraction
> layer well before it's done.

Correct, I'm not advocating halting all development to make this
happen today. This thread is about planning for a more nimble
development landscape for core feature development in the future,
Jetpack or not.

> Don't get me wrong, I want Jetpack to get done, and to get integrated, and
> to kick ass.  But until that day comes, I am not convinced we should adopt a
> Jetpack-centric feature development plan.  If that's an end state we want,
> we should define requirements and ensure they're on the Jetpack roadmap, and
> shift to that model once the platform pieces are ready.  But we have major
> pieces to ship before that day.

Sure. I don't think Jetpack is the assumed heir for core front-end
development. It's got my vote, but there also aren't any obvious
alternatives to talk about
and I'm biased because I've worked on it a bunch :)

This thread is becoming about Jetpack, but the root problem is that
front-end development is pretty broken. I'd love to hear other visions
of what it could look like to build Firefox (desktop, mobile, and
wherever else we go) in 2-3 years.

And yes, we should do all the things normally done when starting a
project, same as has been done for Jetpack so far :)

> Myk says this work can/should come after e10s, so we're likely talking about
> 6-9 months at least to have a minimum viable platform, and then some amount
> of time after that for someone to build a viable, complex feature on the new
> APIs to prove out the new platform.  I still think it's worth committing
> significant resources to this plan, but I'd also like to see us define some
> sort of interim best practice that we can apply right now, without adding a
> dependency on Jetpack to getting critical projects done this year.

Waiting until e10s Jetpack arrives is an example of what this thread
is trying to head off. We can do all of what you recommended above
without waiting for e10s Jetpack to land and without writing any code.

> For the "until Jetpack is deemed ready" timeframe, I'd like to
> counter-propose the following:
>
> 1. Prototype/iterate in whatever way is easiest/fastest to solidify the
> product plan.
> 2. Once we have product requirements and an understanding of what we need to
> build:
>  * Build features as restartless add-ons, optionally shipping these via a
> LabKit/"Services Beta Channel" type of experimental feature channel to get
> feedback.
>  * Enforce m-c grade test coverage, code review, and quality bars from the
> start of this phase
>  * Provide a generic test framework that works in the add-on space, where
> the tests are compatible with the main tree's test frameworks
>  * Use modern coding styles, JSMs, sane API design
>  * Write UI code in a way that it can be trivially ported into Firefox
> proper
> 3. Once a feature is done enough to include in the mainline, dropping that
> code into Firefox proper shouldn't take more than a few days for one
> developer, since tests, code, and UI bits will need minimal changes.
>
> This is an improvement on what we did late in the Sync endgame, and applies
> the lessons we learned from that experience.  And, again, we can start
> applying this approach today, and transition over time to better technology
> stacks as they become available.

This sounds fantastic. However, Identity and Apps are already
in-motion with Jetpack. I guess the next step would be to meet with
those groups to talk about this in context of those specific projects?

David Mason

unread,
Jul 29, 2011, 11:36:32 PM7/29/11
to Ehsan Akhgari, Todd Simpson, mozilla. dev. planning group, Mike Connor, Dietrich Ayala, Jay Sullivan, Dave Townsend, Myk Melez
Hey guys, I've been on a plane all day and haven't had a chance to dive
in to what everyone has said - but I'll give it some thought over the
weekend and add anything I might feel hasn't been mentioned.


Thanks,


Dave

Dietrich Ayala

unread,
Jul 30, 2011, 10:51:29 AM7/30/11
to Dao, dev-pl...@lists.mozilla.org

There is no answer to that question. Jetpack of today, 6 months ago or
6 months from now? Personas or Sync before or after their massive
refactorings?

We are all charged with deciding what Firefox development looks like.
I think it looks pretty painful and slow right now, and Jetpack is one
option for improving it. If you have suggestions for improving Firefox
development, I encourage you to share them.

Dietrich Ayala

unread,
Jul 30, 2011, 11:05:15 AM7/30/11
to David Mandelin, dev-pl...@lists.mozilla.org
On Sat, Jul 30, 2011 at 3:05 AM, David Mandelin <dman...@mozilla.com> wrote:

> On 7/29/2011 10:11 AM, Dietrich Ayala wrote:
>> We've learned through experience with Sync, Personas, Account Manager,
>> Panorama and F1 that front-end features developed outside of the
>> Firefox team require massive refactorings/rewrites and long
>> development cycles to transition from an add-on experiment to a
>> feature in the core, and sometimes the feature doesn't land at all.
>> Also, there can be friction between groups as issues of performance,
>> quality, ownership, conflicting timelines and mismatched expectations
>> crop up during these transitions.
>
> To me, the above reads as even more of a management problem than a
> technology problem. How about putting teams that develop core features
> inside the Firefox team?

Management problem and more! We have groups that have different
approaches, different goals and different timelines, and all are valid
differences. So yeah the same inter-group dynamics that you'll find at
any large organization :)

As I said in my response to Philipp, Jetpack would mitigate some of
these issues, but is not a panacea. Mike Connor's recommendations
would help things in the short term. Even before these
development-related things, we could probably improve communication
between groups with these projects that we are fairly sure will end up
in the product.

> The Jetpack proposal below is great, but beyond
> that, I think "Team A does all the cool work inventing a new feature,
> then Team B does all the grunt work to make it actually work" is a bad
> model for all kinds of reasons.

We've had variations throughout the years. Some projects are handed to
Firefox team to implement "for reals". Some are converted from add-on
to patch, landed and still maintained by the original team that
developed the add-on. We need to be able to handle all permutations of
these scenarios.

Dietrich Ayala

unread,
Jul 30, 2011, 11:37:32 AM7/30/11
to David Mandelin, dev-pl...@lists.mozilla.org
> On Sat, Jul 30, 2011 at 3:05 AM, David Mandelin <dman...@mozilla.com> wrote:
> How about putting teams that develop core features
> inside the Firefox team?

Missed answering this directly on the first round.

That just doesn't scale. We need to be able to handle contributions
from all quarters.

Also see my earlier response to your comment - some of these groups
have valid reasons for working under different sets of requirements.
Changing manager and instituting top-down requirements is not a
reasonable approach in those instances.

Dao

unread,
Jul 30, 2011, 12:03:57 PM7/30/11
to auto...@gmail.com
On 30.07.2011 16:51, Dietrich Ayala wrote:
> On Sat, Jul 30, 2011 at 2:27 AM, Dao<d...@design-noir.de> wrote:
>> On 29.07.2011 19:11, Dietrich Ayala wrote:
>>>
>>> We've learned through experience with Sync, Personas, Account Manager,
>>> Panorama and F1 that front-end features developed outside of the
>>> Firefox team require massive refactorings/rewrites and long
>>> development cycles to transition from an add-on experiment to a
>>> feature in the core, and sometimes the feature doesn't land at all.
>>
>>> My vote is for embedding Jetpack inside of Firefox for use for
>>> developing core front-end features.
>>
>> For which of the cited projects would Jetpack have been an option? I'd say
>> none of Sync, Personas, Account Manager and Panorama. F1? I guess so.
>
> There is no answer to that question. Jetpack of today, 6 months ago or
> 6 months from now? Personas or Sync before or after their massive
> refactorings?

Let's say 6 months from now. I'm actually interested in both before and
after their refactorings. The refactorings happened for certain reasons
and I'd like to know if using Jetpack from the start would have ruled
these problems out. The post-refactoring state alone is also interesting
as I'd like to know if a Jetpack-based implementation could even
theoretically deliver the same quality with the same or less effort.

Personas was implemented from scratch for stock Firefox because:
1) the parts of the add-on's back-end we would have been interested in
were tied to things we weren't interested in, and
2) for Personas to reach product quality, it needed integrated theme
support:
http://mxr.mozilla.org/mozilla-central/search?string=%3A-moz-lwtheme
(It also needed integrated add-on manager support, as the manager wasn't
as extensible as it supposedly is today.)

The Sync refactoring I'm not sure about, but AFAIK it was first and
foremost about the back-end rather than the front-end, and a Jetpack
would probably have used a very similar back-end.

Generally, I think you're saying we want less redundant and wasted work
but the same quality we would have gotten otherwise, but I don't see how
Jetpack gets us there for cases like those experienced by us.

Myk Melez

unread,
Jul 30, 2011, 3:00:11 PM7/30/11
to Dietrich Ayala, Todd Simpson, mozilla. dev. planning group, Mike Connor, Jay Sullivan, Dave Townsend, David Mason
On 2011-07-30 06:10, Dietrich Ayala wrote:
> On Sat, Jul 30, 2011 at 3:01 AM, Myk Melez<m...@mozilla.org> wrote:
>> This is a particularly promising avenue for integration. We've done this
>> with Test Pilot for beta releases, and Mozilla China has done it with a
>> bunch of addons for stable releases. And Mozilla has helped to develop,
>> release, and market high-quality addons alongside Firefox, including
>> Personas, F1, and Kodak EasyShare Gallery Companion.
> These are using Jetpack?
Erm, no, my point is about shipping addons bundled with and alongside
Firefox generally.

-myk

Jean-Marc Desperrier

unread,
Jul 30, 2011, 8:57:54 PM7/30/11
to
On 29/07/2011 19:11, Dietrich Ayala wrote:
> The mix of XUL/XPCOM/XBL/JS/CSS that we currently
> use for core feature development is powerful yet extremely
> regression-prone, and painstaking to develop against.

I recently was wondering if, for the short term, rewritting the core of
Firefox on top of HTML5 wouldn't make more sense and be easier to reach
than B2G for Mozilla ...

BTW isn't this whole discussion more platform than planning ?

Asa Dotzler

unread,
Jul 30, 2011, 9:26:31 PM7/30/11
to Jean-Marc Desperrier

Jean-Marc Desperrier wrote:
> On 29/07/2011 19:11, Dietrich Ayala wrote:
>> The mix of XUL/XPCOM/XBL/JS/CSS that we currently
>> use for core feature development is powerful yet extremely
>> regression-prone, and painstaking to develop against.
>
> I recently was wondering if, for the short term, rewritting the core of
> Firefox on top of HTML5 wouldn't make more sense and be easier to reach
> than B2G for Mozilla ...

Firefox and B2G are different things. B2G isn't trying to be a Web
browser. B2G is plan for a platform and not a plan for a piece of client
software. Firefox is client software and re-writing it with purely Web
technologies would be a pretty huge undertaking and not something "short
term".

> BTW isn't this whole discussion more platform than planning ?

Planning is a fine place for people to share what their plans are and I
think that's what the B2G folks were trying to do here.

Asa Dotzler

unread,
Jul 30, 2011, 9:32:10 PM7/30/11
to Jean-Marc Desperrier

Oops, that second part didn't involve B2G. Sorry. I've had too many
threads going at once here.

Dietrich's thread was very much in line with "planning". It may have
wandered off some into platform talk, but it started as a discussion
around coordination and tools and how we want to design and build Firefox.

- A

Blair McBride

unread,
Jul 31, 2011, 10:26:48 PM7/31/11
to Dietrich Ayala, Todd Simpson, mozilla. dev. planning group, Mike Connor, Jay Sullivan, Dave Townsend, David Mason, Myk Melez
On 30/07/2011 5:11 a.m., Dietrich Ayala wrote:
> We've learned through experience with Sync, Personas, Account Manager,
> Panorama and F1 that front-end features developed outside of the
> Firefox team require massive refactorings/rewrites and long
> development cycles to transition from an add-on experiment to a
> feature in the core, and sometimes the feature doesn't land at all.
> Also, there can be friction between groups as issues of performance,
> quality, ownership, conflicting timelines and mismatched expectations
> crop up during these transitions.

In all of those cases, they started out as experiments and prototypes
that added something on top of Firefox, and tried to directly transition
to a production-ready feature. Shipping prototype code doesn't work.
Jetpack is great, and it should be in Firefox - but it's not going to
make prototypes shippable.

- Blair

Robert Kaiser

unread,
Aug 1, 2011, 9:36:45 AM8/1/11
to
Ehsan Akhgari schrieb:

> There were a bunch of problems
> that we faced during the integration phase

All of which would IMHO not have been too different if you would have
been based on Jetpack.
All of those are important points I have seen with a lot of things that
have been prototyped in some way, usually as add-ons, and then later
integrated. Even Firefox itself suffered a lot from a number of those
points in its early stage, before it became a mainline project.
IMHO, those are all points of both culture and rapid prototyping, and
independent of the technology.

> If I have to share the lessons that I learned from this efforts, I can
> summarize them as:
>

> 1. Getting feedback from users/developers early on is valuable. [...]
>
> 2. Code quality should never be an after-thought. [...]

Good lessons for all of us, right.
I for myself spent a lot of time writing automated tests for my Tahoe
Data Manager add-on, and that tremendously helped code quality by
finding a number of bugs I probably would not have seen else, and it (in
addition to following common Mozilla code style from the beginning)
surely made the integration into SeaMonkey smoother.

And I think those things are more to the core of getting stuff
integrated into mainline code more easily. Of course, spending a lot of
time writing tests and writing your code to be clean and possibly even
performant from the beginning may conflict with rapid prototyping, but
you always need to keep in mind that you need to get there some time
anyhow if your target is to include the feature in the mainline.

Those things are independent of the technology though, no matter if we
have Jetpack in the mainline or not, you can prototype fast without
tests and with ugly hacks and later rewrite or start off a bit slower
but with tests and clean code from the beginning. Your choice as a
developer - just don't complain afterwards that you haven't been warned. ;-)

Ron Hunter

unread,
Aug 1, 2011, 11:30:43 AM8/1/11
to
I am going to stick my 'user oar' into this discussion.

Hasn't it always been the plan to incorporate only those features into
the base code of Firefox that would be used by the majority of users?
Is this no longer the policy? I surely don't want to see Firefox become
another ADA which was designed by a large committee, and they put in
everything anyone suggested. How many 'users' would ever use the
Jetpack code? I already have to tote the load for all of the 3% of
users who find the incorporation of Tab Candy into the mainline code.
What's next?
Back to lurking.

Alex Limi

unread,
Aug 1, 2011, 12:38:26 PM8/1/11
to Dietrich Ayala, Todd Simpson, David Mason, Mike Connor, Jay Sullivan, Dave Townsend, mozilla. dev. planning group, Myk Melez
On Sat, Jul 30, 2011 at 7:35 AM, Dietrich Ayala <auto...@gmail.com> wrote:

> I'd love to hear other visions
> of what it could look like to build Firefox (desktop, mobile, and
> wherever else we go) in 2-3 years.
>

The #1 thing I'd like to see from the UX side of things over the next years
is dropping (or minimizing the use of) XUL if possible.

If contributors could hack something like straight-up HTML5 & CSS, our
contributions (and quality!) would increase, and I suspect our time to
implement front-end improvements and designs would be reduced greatly. Have
a part of our stack that works the same way as the web.

(and yes, I'm sure there are millions of details and dependencies here)

The other thing I'd like us to seriously look in to is hosting in a system
like Github. The amount of visibility and developer engagement, willingness
to hack, etc this brings us shouldn't be underestimated.

Hopefully this isn't too off-topic, I'm passionately interested in lowering
the barrier to getting involved with Firefox.

--
Alex Limi · Firefox UX Team · +limi <http://profiles.google.com/limi> ·
@limi <http://twitter.com/limi/> · limi.net

Dao

unread,
Aug 1, 2011, 1:36:20 PM8/1/11
to
On 01.08.2011 17:30, Ron Hunter wrote:
> How many 'users' would ever use the Jetpack code?

None. Jetpack is not a user-facing feature, it's a framework for add-ons.

Boris Zbarsky

unread,
Aug 1, 2011, 1:51:17 PM8/1/11
to
On 8/1/11 11:30 AM, Ron Hunter wrote:
> How many 'users' would ever use the Jetpack code?

About as many as ever use the XPI installer code. Way more than ever
use the MathML code or probably the SVG code.

-Boris

Daniel Veditz

unread,
Aug 1, 2011, 2:43:01 PM8/1/11
to dev-pl...@lists.mozilla.org
On 8/1/11 10:51 AM, Boris Zbarsky wrote:
> On 8/1/11 11:30 AM, Ron Hunter wrote:
>> How many 'users' would ever use the Jetpack code?
>
> About as many as ever use the XPI installer code.

Way more if we eventually develop core Firefox features on top of
it. Maybe a big "if" but that's how Jetpack came into this thread in
the first place.

-Dan Veditz

Daniel Veditz

unread,
Aug 1, 2011, 2:45:39 PM8/1/11
to dev-pl...@lists.mozilla.org
On 7/29/11 12:08 PM, Robert Kaiser wrote:
> Removing markup [...] and making everything JS does surely
> incredibly slow down any somewhat larger UI [....] You also

> don't write large websites as one large JS script that builds the
> DOM and styles all dynamically, right?

Facebook and Twitter do ;-)

-Dan Veditz

Daniel Veditz

unread,
Aug 1, 2011, 2:50:00 PM8/1/11
to dev-pl...@lists.mozilla.org
On 7/31/11 7:26 PM, Blair McBride wrote:
> Shipping prototype code doesn't work. Jetpack is great, and it
> should be in Firefox - but it's not going to make prototypes
> shippable.

"Build one to throw away" is still true.

Axel Hecht

unread,
Aug 2, 2011, 8:14:56 AM8/2/11
to
On 01.08.11 18:38, Alex Limi wrote:
> On Sat, Jul 30, 2011 at 7:35 AM, Dietrich Ayala<auto...@gmail.com> wrote:
>
>> I'd love to hear other visions
>> of what it could look like to build Firefox (desktop, mobile, and
>> wherever else we go) in 2-3 years.
>>
>
> The #1 thing I'd like to see from the UX side of things over the next years
> is dropping (or minimizing the use of) XUL if possible.
>
> If contributors could hack something like straight-up HTML5& CSS, our

> contributions (and quality!) would increase, and I suspect our time to
> implement front-end improvements and designs would be reduced greatly. Have
> a part of our stack that works the same way as the web.
>
> (and yes, I'm sure there are millions of details and dependencies here)

Actually, I think this very thread is about the problems we have with
getting code that's straight-up HTML5 etc into a shippable product.

The "straight-up" piece is just not about writing something that's
localizable, accessible, RTL-ready, comes with tests etc, blends into
all platforms, etc

Once you have all those code-quality things on top of HTML, the bit of
markup differences may not be all that much of a difference.

> The other thing I'd like us to seriously look in to is hosting in a system
> like Github. The amount of visibility and developer engagement, willingness
> to hack, etc this brings us shouldn't be underestimated.

Do you think we have problems with any of these? I don't, really.

I do think we're bad at turning the willingness and engagement and ideas
that are there into try-server builds, and explaining what that is.
Becoming more of a hosting service to lower the barrier to those
systems, and figure out ways to make those discoverable would help a
ton, I guess. "discoverable" on both ways, as in, make it discoverable
that you can do a try server build, where they go etc from your repo.
And make it discoverable for mentors that there are new contributions
out to field, and maybe test results to help with or something.

> Hopefully this isn't too off-topic, I'm passionately interested in lowering
> the barrier to getting involved with Firefox.

I personally prefer to design the learning curve vs lowering the barrier
to entry. Like, I don't think there's a barrier of entry per se, but
there are a few points on your way to becoming a mozilla contributor
where you have to learn a whole lot, and you don't really understand why
you're learning that at that point.

If we get that to work for a handful of use cases, we'd probably win a lot.

Axel

Robert Kaiser

unread,
Aug 2, 2011, 10:11:50 AM8/2/11
to
Daniel Veditz schrieb:

And I thought the only thing that sucked about them was being
centralistic silos. Now I know anther reason why I would never think to
register with them. ;-)

Robert Kaiser

unread,
Aug 2, 2011, 10:22:25 AM8/2/11
to
Alex Limi schrieb:

> The #1 thing I'd like to see from the UX side of things over the next years
> is dropping (or minimizing the use of) XUL if possible.

Why from a UX side? Right now, XUL excels at looking well UX-wise
(unless you design an alien-looking piece with it like the add-ons
manager, sorry to say that) while HTML sucks in terms of
cross-application consistency and positioning of UI elements, flexible
layout, and availability of easy UI controls (and I won't go into the
hell of using <input> for about a dozen completely different kinds of
controls). Also, being able to use overlays and defining new controls
(with XBL atm) are other things where XUL excels compared to HTML right now.

We need to put a ton of work into HTML to make it match XUL UX-wise. I'm
all for us doing that, and actually we've been holding back on doing
that a bit too long IMHO. We need to bring HTML up to speed there, and
with our ton of experience in XUL land we're in the best position on the
whole web planet to work on the new standards needed, to avoid mistakes
we made in XUL land, and to lead implementation-wise.

I agree on the long-term target, but we still have a *ton* of work to
get there - and I really don't understand the UX argument at this point.

Alex Limi

unread,
Aug 2, 2011, 1:58:28 PM8/2/11
to Robert Kaiser, dev-pl...@lists.mozilla.org
On Tue, Aug 2, 2011 at 7:22 AM, Robert Kaiser <ka...@kairo.at> wrote:

> Why from a UX side?


Because most of the people that would/could contribute (including people on
the UX team) have pretty deep knowledge of HTML/CSS — and occasionally JS —
but XUL is too narrow in its focus, so it's unlikely that they'll ever pick
that up, since it isn't a reusable skill.

Also, the browser should be written in the language of the web.

PS: If you think the add-ons manager is alien-looking, that really has
nothing to do with XUL vs. HTML. It's really not that hard to make a UI in
HTML/CSS that looks "native", whatever your definition of native is.
Performance, animations and effects are a different matter, and need to
improve — but we're getting there.

Alex Limi

unread,
Aug 2, 2011, 1:58:28 PM8/2/11
to Robert Kaiser, dev-pl...@lists.mozilla.org
On Tue, Aug 2, 2011 at 7:22 AM, Robert Kaiser <ka...@kairo.at> wrote:

> Why from a UX side?

Because most of the people that would/could contribute (including people on
the UX team) have pretty deep knowledge of HTML/CSS — and occasionally JS —
but XUL is too narrow in its focus, so it's unlikely that they'll ever pick
that up, since it isn't a reusable skill.

Also, the browser should be written in the language of the web.

PS: If you think the add-ons manager is alien-looking, that really has
nothing to do with XUL vs. HTML. It's really not that hard to make a UI in
HTML/CSS that looks "native", whatever your definition of native is.
Performance, animations and effects are a different matter, and need to
improve — but we're getting there.

--

Dan Mills

unread,
Aug 2, 2011, 2:28:58 PM8/2/11
to mozilla. dev. planning group, Todd Simpson, David Mason, Mike Connor, Jay Sullivan, Dave Townsend, Myk Melez
Hey all,

Jumping late into this thread (and sorry if it doesn't thread right, I
had some mailing list fail and the headers aren't right).

Thanks for kicking off this discussion. I've been talking with Ragavan
about this and want to give you our $.02 from the Identity & Apps POV.

First, I'm happy to see Jetpack being considered as a possible vehicle
for feature integration. We (the Identity and Apps teams) have been
able to iterate unbelievably quickly using Jetpack, while there are
certainly areas for improvement I can't overstate how valuable it has
been to us so far.

I wanted to take the discussion a step further, though, and talk about
a couple of things we'd like to see from the platform. I don't mean to
negate the pain points discussed before (like l10n tools,
perf/automated testing, etc)--those will certainly need to be worked
on. But I think you need to think bigger:

* We want to "dial up" feature deployment

We'd like to be able to progressively deploy features based on
criteria such as geographic area, language, or just to a percentage of
users (e.g., have a knob where we can say "12% of users in France" and
roll features out progressively).

Using such a system we could also maintain regional sets of features
on a permanent or semi-permanent basis. For example, the China team's
Firefox build with bundled add-ons could be rolled out/maintained in
this way.

Websites do this all the time, and it's a huge advantage that we
currently don't have.

* We need to iterate independently, on an ongoing basis

Something that seems to get ignored is iteration *after* landing a
feature. I don't mean maintenance, rather already-planned changes
(further milestones or new sub-features), or just response to user
feedback. We need the ability to iterate quickly, ship, then iterate
quickly some more.

Note that I don't mean we want to ship sub-par quality code earlier
and fix it later. I'm talking about shipping a kernel of a feature
which grows/changes over time.

Hope this gives you an idea of the direction we'd like to go in. I
think that Jetpack has the best potential for giving us features like
these, and that's why we're rooting for it.

Dan

> We've learned through experience with Sync, Personas, Account Manager,
> Panorama and F1 that front-end features developed outside of the
> Firefox team require massive refactorings/rewrites and long
> development cycles to transition from an add-on experiment to a
> feature in the core, and sometimes the feature doesn't land at all.
> Also, there can be friction between groups as issues of performance,
> quality, ownership, conflicting timelines and mismatched expectations
> crop up during these transitions.
>

> We need to reduce the friction and time that it takes to get these
> types of projects into Firefox.
>
> Currently both the Apps and Identity teams are using Jetpack as their
> development platform. Jetpack is fantastic for rapid prototyping but
> is targeted at lightweight add-on development, not ship-quality
> browser core features. It's also not in Firefox.
>
> The product road map for Apps and Identity is not specific about
> timeline, but says they're included in what the Web looks like in 24 -
> 36 months. Given the competitive atmosphere and the historical
> evidence for adding many months onto the schedule to get projects like
> these shipping in Firefox, I think we need to look at ways to reduce
> the time-to-integration for these kinds of projects.
>
> Maybe this means immediately putting resources behind embedding
> Jetpack in Firefox such that we can write core features on top of it.
> Maybe it means we ship new features as add-ons that are included with
> the release (killswitch, anyone?). Maybe it means we move these
> projects off of Jetpack and into the core at the earliest possible
> stage.
>
> But no matter when the code lands in core, we're still hampered by a
> fragile ecosystem: The mix of XUL/XPCOM/XBL/JS/CSS that we currently


> use for core feature development is powerful yet extremely

> regression-prone, and painstaking to develop against. Writing
> shippable code is only possible by specialists with experiential
> knowledge of a ton of undocumented code and a host of implicit
> behaviors that are impossible to predict.


>
> My vote is for embedding Jetpack inside of Firefox for use for

> developing core front-end features. Having a stable and sane pure-JS
> API layer to build new features against would:
>
> * make it easier to share code between desktop and mobile
> * ease integration of projects developed by external teams (and allow
> possible code-sharing there too)
> * ease contribution by volunteer developers, and ramp-up by new employees
> * allow e10s-ification of core features using the same mechanisms and
> patterns as add-ons
> * strengthen Jetpack by ensuring it's APIs are up to the level of
> stability and performance required to be in the core
>
> Sure, it wouldn't work for everything - we'd still have to dive under
> the covers for a bunch of things. Sure, we'll have to figure out a
> host of issues around the interplay between shipped-SDK and
> embedded-SDK. But I think it will allow front-end development to move
> faster and more effectively in the long term.
>

Mike Shaver

unread,
Aug 2, 2011, 2:30:49 PM8/2/11
to Dietrich Ayala, Todd Simpson, mozilla. dev. planning group, Mike Connor, Jay Sullivan, Dave Townsend, David Mason, Myk Melez
On Fri, Jul 29, 2011 at 1:11 PM, Dietrich Ayala <auto...@gmail.com> wrote:
> We've learned through experience with Sync, Personas, Account Manager,
> Panorama and F1 that front-end features developed outside of the
> Firefox team require massive refactorings/rewrites and long
> development cycles to transition from an add-on experiment to a
> feature in the core, and sometimes the feature doesn't land at all.
> Also, there can be friction between groups as issues of performance,
> quality, ownership, conflicting timelines and mismatched expectations
> crop up during these transitions.

Sorry for not quoting the things I agree with or am directly
referencing; this thread got away from me.

We need to build on the platform that we tell others they should build
on. That means HTML and it means Jetpack. It will take time and
platform improvements to get there, but if it were easy everyone would
be doing it. If HTML applications can't be localized well (which
would seem odd, since I see lots of localized web sites), then we just
need to fix that. If you can't use a Jetpack to write the download
manager, we just need to fix that. Every time someone writes XUL
instead of HTML, they're throwing away some leverage against our
mission (and against some of our other projects, like the services
we're developing and B2G). Sometimes the right thing is to throw away
some leverage for the sake of expediency, but we need to change our
default mode from "XUL unless someone decides otherwise".

Prototypes are like writing a paper, not like writing the first chunk
of a product. You learn things in the context of the paper's research
and then you figure out how to use those lessons in production code.
The things that make prototyping a good way to learn -- cutting
corners, solving only the problems that answer questions, simplifying
assumptions about security/scalability/etc. -- make them exactly the
wrong way to build towards a product. Taking prototype code as the
basis of a product is a terribly false economy and we need to stop
doing it, IMO.

Mike

Robert Kaiser

unread,
Aug 3, 2011, 9:58:04 AM8/3/11
to
Alex Limi schrieb:

> On Tue, Aug 2, 2011 at 7:22 AM, Robert Kaiser<ka...@kairo.at> wrote:
>
>> Why from a UX side?
>
> Because most of the people that would/could contribute (including people on
> the UX team) have pretty deep knowledge of HTML/CSS — and occasionally JS —
> but XUL is too narrow in its focus, so it's unlikely that they'll ever pick
> that up, since it isn't a reusable skill.

OK, sounds good, still we need to do a lot of work to get HTML be really
in a usable state for that.

> PS: If you think the add-ons manager is alien-looking, that really has
> nothing to do with XUL vs. HTML. It's really not that hard to make a UI in
> HTML/CSS that looks "native", whatever your definition of native is.
> Performance, animations and effects are a different matter, and need to
> improve — but we're getting there.

Well, I know it has nothing to do with XUL vs. HTML, as it actually is
XUL and still looks like a amateur website to me instead of like
professional browser UI in a lot of pieces. I even have read comments
from a number of people who thought it was a website and therefore
didn't really trust what it said.

Robert Kaiser

unread,
Aug 3, 2011, 10:01:31 AM8/3/11
to
Mike Shaver schrieb:

> We need to build on the platform that we tell others they should build
> on. That means HTML and it means Jetpack. It will take time and
> platform improvements to get there, but if it were easy everyone would
> be doing it.

I agree and we need to start working on it (actually I think we're even
late in some of it), but we're unfortunately not there yet. Should not
stop us to get there, though!

> Prototypes are like writing a paper, not like writing the first chunk
> of a product.

True, even though a good number of comments here sounded like they want
to make it easier for us to ship prototypes as products and quickly make
them into it.

Dietrich Ayala

unread,
Aug 8, 2011, 8:30:46 PM8/8/11
to Dan Mills, Todd Simpson, mozilla. dev. planning group, Mike Connor, Jay Sullivan, Dave Townsend, David Mason, Myk Melez
> First, I'm happy to see Jetpack being considered as a possible vehicle
> for feature integration. We (the Identity and Apps teams) have been
> able to iterate unbelievably quickly using Jetpack, while there are
> certainly areas for improvement I can't overstate how valuable it has
> been to us so far.

Can you talk about why this is? What characteristics of the framework
itself, or the development flow are so useful? Whether we bundle
add-ons, or embed Jetpack or do nothing revolutionary at all, this
would be really good to know.

> * We want to "dial up" feature deployment
>
> We'd like to be able to progressively deploy features based on
> criteria such as geographic area, language, or just to a percentage of
> users (e.g., have a knob where we can say "12% of users in France" and
> roll features out progressively).
>
> Using such a system we could also maintain regional sets of features
> on a permanent or semi-permanent basis. For example, the China team's
> Firefox build with bundled add-ons could be rolled out/maintained in
> this way.
>
> Websites do this all the time, and it's a huge advantage that we
> currently don't have.

In the short term, the idea of shipping a bunch of versions of Firefox
that are different from each other probably scares the living
daylights out of a lot of people. But the reality is that add-on
install numbers show that the Firefox that most people *have* is *not*
the one we shipped. And that's a reality we need to embrace.

However, I think this need is pretty far outside the scope of this
thread (which is overbroad already!). For now, what you want could be
done inside an add-on, or at least prototyped that way. Maybe spin up
a labs project around this idea? And get the conversation started with
a blog post or something that'll get project-wide input from all the
affected groups (QA, Support, RelEng, Automation, L10n, etc). An
all-hands session on this might not be a bad idea.

> * We need to iterate independently, on an ongoing basis
>
> Something that seems to get ignored is iteration *after* landing a
> feature. I don't mean maintenance, rather already-planned changes
> (further milestones or new sub-features), or just response to user
> feedback. We need the ability to iterate quickly, ship, then iterate
> quickly some more.
>
> Note that I don't mean we want to ship sub-par quality code earlier
> and fix it later. I'm talking about shipping a kernel of a feature
> which grows/changes over time.

We've done this to some extent with Personas and Panorama for example.
The add-ons manager is another example of a major feature that changed
pretty significantly over time.

However, there are very high costs to small changes to the core
product when your user-base is a half billion people. Talk to SUMO for
more information about this :)

Maybe what you want here is a world where we ship with these types of
add-ons pre-installed (Jetpack or not is irrelevant) and those add-ons
can iterate independently of the core browser.

Steve Wendt

unread,
Aug 8, 2011, 8:43:36 PM8/8/11
to
On 8/8/2011 5:30 PM, Dietrich Ayala wrote:

> Maybe what you want here is a world where we ship with these types of
> add-ons pre-installed (Jetpack or not is irrelevant) and those add-ons
> can iterate independently of the core browser.

Note that this already happens with SeaMonkey.

Dietrich Ayala

unread,
Aug 8, 2011, 8:43:09 PM8/8/11
to Blair McBride, Todd Simpson, mozilla. dev. planning group, Mike Connor, Jay Sullivan, Dave Townsend, David Mason, Myk Melez
On Sun, Jul 31, 2011 at 7:26 PM, Blair McBride <bmcb...@mozilla.com> wrote:

> On 30/07/2011 5:11 a.m., Dietrich Ayala wrote:
>>
>> We've learned through experience with Sync, Personas, Account Manager,
>> Panorama and F1 that front-end features developed outside of the
>> Firefox team require massive refactorings/rewrites and long
>> development cycles to transition from an add-on experiment to a
>> feature in the core, and sometimes the feature doesn't land at all.
>> Also, there can be friction between groups as issues of performance,
>> quality, ownership, conflicting timelines and mismatched expectations
>> crop up during these transitions.
>
> In all of those cases, they started out as experiments and prototypes that
> added something on top of Firefox, and tried to directly transition to a
> production-ready feature. Shipping prototype code doesn't work. Jetpack is

> great, and it should be in Firefox - but it's not going to make prototypes
> shippable.

While the "ship my prototype" scenario has occurred, some of the teams
above were targeting Firefox directly, and definitely did not think of
their code as prototype code.

But the prototype scenario is peripheral to my main point. We need to
make Firefox development faster and less fragile for people in Firefox
team and outside of it.

The Identity team are developing a core Firefox feature, not a
prototype. We need to allow that kind of work to happen at web-speed.
Part of the solution is having them oriented around our shipping
requirements WRT performance, review, tests, etc. But another part is
making Firefox development not suck. Jetpack is a potential route to
the latter.

Robert Kaiser

unread,
Aug 9, 2011, 11:43:31 AM8/9/11
to
Steve Wendt schrieb:

Where exactly that has been the cause of a lot of grief due to the sad
state in which everything regarding the shipping of add-ons with the
product is in shared code, from the installer via the build system to
the add-ons manager backend.

Kevin Dangoor

unread,
Aug 9, 2011, 1:14:26 PM8/9/11
to Dietrich Ayala, Todd Simpson, mozilla. dev. planning group, Mike Connor, Jay Sullivan, Blair McBride, Dave Townsend, David Mason, Myk Melez
On Aug 8, 2011, at 8:43 PM, Dietrich Ayala wrote:

> But the prototype scenario is peripheral to my main point. We need to
> make Firefox development faster and less fragile for people in Firefox
> team and outside of it.
>
> The Identity team are developing a core Firefox feature, not a
> prototype. We need to allow that kind of work to happen at web-speed.
> Part of the solution is having them oriented around our shipping
> requirements WRT performance, review, tests, etc. But another part is
> making Firefox development not suck. Jetpack is a potential route to
> the latter.

I am not a Firefox developer, but I am a web developer and managed a team of Firefox developers. I read some of their patches and watched them work... it's painful. Here we have this amazing, dynamic platform that has grown tremendously in power over the past few years, but contributing to it requires way more pain than people want from a 2011 platform.

Shaver's comments about building on the same platform we're telling everyone to build on are spot on. Dietrich's comments above about using those tools but with the same quality characteristics we expect from our development process today also resonate very strongly with me.

Allowing for fast, web-like prototyping of new features and a path to build on those same development techniques (HTML/JS/CSS + Jetpack-style APIs) to create final, production-quality (tested and localized) features would be a huge win.

At a risk of going a little off-topic, the "Principles of Product Development Flow" [1] cites research that shows a huge improvement in product quality that comes from faster iteration. I think rapid release is an awesome step forward for us, but I think there's a lot more we can do and I'm happy to see what the Identity team is doing there.

This discussion crosses product and engineering boundaries in interesting ways. This is a conversation that I personally would love to see have a tangible outcome. But, we have a lot of infrastructurey work underway now (e10s!) and I fear that if "doing something about this" does not appear on anyone's roadmap then nothing will happen until the next time this conversation comes up. Or until Identity forces us to act.

Generalizing, the product team wants:

- even faster iteration than our 18-weeks-and-out-the-door cycles
- ability to distribute test features among various audiences
- more web-like development that makes it easier to bring new contributors up to speed

Engineering (or, at least Dietrich :) wants:

- easier development with no measurable loss of quality
- more web-like development that makes it easier to bring new contributors up to speed

... or something like that.

Perhaps we should get a feature page going with product and engineering people attached to it so that solutions for the various issues brought up on this thread can be investigated and taken down.

Kevin

[1] Warning: this book is dense and dry. But, the material is useful nonetheless. http://www.amazon.com/Principles-Product-Development-Flow-Generation/dp/1935401009

--
Kevin Dangoor
product manager, developer tools

work: http://mozilla.com/
blog: http://blueskyonmars.com/
@dangoor

Mike Connor

unread,
Aug 9, 2011, 7:23:35 PM8/9/11
to Mike Shaver, Todd Simpson, mozilla. dev. planning group, Dietrich Ayala, Jay Sullivan, Dave Townsend, David Mason, Myk Melez

On 2011-08-02, at 2:30 PM, Mike Shaver wrote:

> Prototypes are like writing a paper, not like writing the first chunk

> of a product. You learn things in the context of the paper's research
> and then you figure out how to use those lessons in production code.
> The things that make prototyping a good way to learn -- cutting
> corners, solving only the problems that answer questions, simplifying
> assumptions about security/scalability/etc. -- make them exactly the
> wrong way to build towards a product. Taking prototype code as the
> basis of a product is a terribly false economy and we need to stop
> doing it, IMO.

This is the most important point of this thread. I've personally spent the last 18 months second-guessing the "second rewrite or harden what we have" call we made for Sync, and while there were major time-to-market pressures, we ran up a pretty significant amount of technical debt in the process. We're only now seeing the light at the end of the tunnel there, and it'll likely be two years after Sync 1.0 that we'll actually be on a product-quality codebase, by the standards we'd use if we were building now.

Prototypes and rapid iteration are critical pieces, and I think that we've been great at doing that in a Labs context. But we have to avoid the trap of treating a solid prototype as something ready to ship and maintain for years. At some point in the process we need to be able to stop prototyping and build a product. A product, as opposed to a prototype, needs to have defined goals (and non-goals) and a solid, maintainable implementation we can ship and support for years. The latter means solid API and architecture design, comprehensive test coverage, and well-written and understandable code.

I think we can and should go faster where we can, but we need to go faster through smarter, better engineering, not through shortcuts. They don't work, and incur long-term costs that we can't afford if we also want to go fast with follow-up work.

tl;dr Build awesome prototypes fast and loose, but when it's time to make them into products build with discipline and quality, from a clean base.

-- Mike

Robert Kaiser

unread,
Aug 10, 2011, 10:49:10 AM8/10/11
to
Kevin Dangoor schrieb:

> I am not a Firefox developer, but I am a web developer and managed a team of Firefox developers. I read some of their patches and watched them work... it's painful. Here we have this amazing, dynamic platform that has grown tremendously in power over the past few years, but contributing to it requires way more pain than people want from a 2011 platform.

From all I've seen, the vast majority of the problems are not the
technology used but that fact that it's really hard to write really
performant, lean-on-memory, non-leaking, production-quality and readable
features. If that's done with XUL-based or HTML-based stacks is in the
end not a real difference, IMHO.

Add-on API and HTML are nice once they are capable of replacing some of
the previous APIs and XUL (but we still have a long way to go there,
more on the latter than on the former), but they nothing near a panacea
for needing to re-think almost every piece of code when going from
getting things working nicely to making it production-ready in a product
that ships to hundreds of millions of wholly different people.

David Mandelin

unread,
Aug 10, 2011, 12:08:54 PM8/10/11
to Robert Kaiser, dev-pl...@lists.mozilla.org
----- Original Message -----
> Kevin Dangoor schrieb:
> > I am not a Firefox developer, but I am a web developer and managed a
> > team of Firefox developers. I read some of their patches and watched
> > them work... it's painful. Here we have this amazing, dynamic
> > platform that has grown tremendously in power over the past few
> > years, but contributing to it requires way more pain than people
> > want from a 2011 platform.
>
> From all I've seen, the vast majority of the problems are not the
> technology used but that fact that it's really hard to write really
> performant, lean-on-memory, non-leaking, production-quality and
> readable
> features. If that's done with XUL-based or HTML-based stacks is in the
> end not a real difference, IMHO.

That sounds like a list of problems to be solved. I realize that to some extent, high-quality software is Just Hard, but I'd bet we can come up with ways to make it easier. Also, I have some experience writing front-end software in Visual Basic and Python, which I would think are not too fundamentally different, and it took care to do well, but not extraordinary care, and I rarely found perf and memory to be a burden. Are XUL/HTML in fact fundamentally different? If so, how?

"performant, lean-on-memory" -- Why are these hard? How much would solid perf tools in the browser help, like CPU and memory profilers? Steve Fink is already working on the engine side of a CPU profiler for JS...

"non-leaking" -- What about this one? What kind of leaks--is it things that get held on to for way too long, or something else? Can tools help here? What about education?

"production-quality and readable" -- Is this basically non-bugginess, UX, polish, that sort of thing? There is a lot of detail work there no matter what. But I would have to think there are differences between XUL and HTML: one or the other has got to have more control over appearance or less semantic corner cases. Which one is really better?

Dave

beltzner

unread,
Aug 10, 2011, 12:28:21 PM8/10/11
to dev-pl...@lists.mozilla.org
> tl;dr Build awesome prototypes fast and loose, but when it's time to make them into products
> build with discipline and quality, from a clean base.

This, a million times yes.

It's an incredibly hard thing to do, of course, but also an incredibly
important one. As such, it needs recognition and understanding from
the product stakeholders and the prototyping team that what's being
built is a proof of concept or prototype, not a line of code that will
*ever* see direct inclusion in a mainstream product.

I think that will also, perhaps counter-intuitively, allow us to move
faster. If groups doing prototyping focused less on deep levels of
polish, presentation, and focused on building prototypes that answer
questions about performance, experience and proving of technology,
then we'll be able to run these prototypes as "fast multiple"
experiments giving us a better base of knowledge on which to base the
actual product-quality code.

If we can get everyone to buy into this attitude the way that the
Mozilla community has bought into the rapid-release cycle and
associated up-front planning (and protip: they're all related!) then I
think we'll win huge, and avoid large amounts of head- and heartache.

cheers,
mike

Dave Mandelin

unread,
Aug 10, 2011, 12:33:12 PM8/10/11
to dev-pl...@lists.mozilla.org
And a billion times yes to that.

A related point is that sometimes it's good to declare something a
prototype that you originally might have expected to ship. In JM, we
started building the compiler while fatvals was still underway. We knew
that the result might not be usable with the post-fatvals engine, but we
kind of hoped it would, and built it reasonably solidly. But once
fatvals got done, David Anderson declared the first version a prototype,
scrapped it, and started anew, using everything he had learned there to
make the final result better.

Dave

Benjamin Stover

unread,
Aug 10, 2011, 1:49:20 PM8/10/11
to dev-pl...@lists.mozilla.org
> "non-leaking" -- What about this one? What kind of leaks--is it things
> that get held on to for way too long, or something else? Can tools help
> here? What about education?
>
>
Finding Mozilla leaks is confusing and hard, at least for me. Especially in
contrast to a Chrome tool that came up during the last memshrink meeting:
http://gent.ilcore.com/2011/08/finding-memory-leaks.html

The two nice things there:
1) seeing a graph over time about the amount of memory used by a website (in
this case, our chrome code)
2) being able to snapshot and compare

With this tool, it was easy to see any DOM nodes that were still around but
no longer in the DOM.

> "production-quality and readable" -- Is this basically non-bugginess, UX,
> polish, that sort of thing? There is a lot of detail work there no matter
> what. But I would have to think there are differences between XUL and HTML:
> one or the other has got to have more control over appearance or less
> semantic corner cases. Which one is really better?
>
>

It's less about the technology, more about writing tests. My personal,
uncomfortably honest anecdote ahead.

I've been bitten too many times trying to land some major refactoring like
e10s, replacing canvas tiles with cross-process async scrolling, and sidebar
panning for Fennec. Inevitably there would be months of fixing regressions
and a general gut-wrenching feeling that the code is becoming ungainly with
all the workarounds that get glued on top as an afterthought. Fixing some
regressions would cause new regressions. I quickly learned a bad lesson:
stop trying to rewrite code by making it better, faster, smaller. Just fix
the problem with as little change as possible. There are better things to
spend my time on, and they'll make me feel a lot happier.

This is because we have poor test coverage. Our "net" that tells us what has
changed is full of giant holes, so we step lightly and try not to touch
anything we don't understand. And so comes the nugget of wisdom: we need
more test coverage!

What I've learned is that we can tell ourselves to write more tests until
we're blue in the face. That doesn't mean we will.

I find writing frontend tests to be an exercise in frustration. I feel that
the test infrastructure is 1) unreliable (land something, random orange,
backout) 2) takes a long time to write and run tests (could be days and days
of sending to try because there's some leak on Windows) 3) no proper line
between functional and unit tests and 4) just plain no fun.

The salt in the wound is that I've done it the right way with websites
before working on browsers, and I know what I'm missing. Things like pymock,
inline tests, nose, proper fixtures, and twill made it fun and fast to
refactor code. If there was a regression, I wrote a functional test and they
could always detect failure, even if I completely gutted the code and
started over (mochitests most of the time need lots of API calls or access
of implementation-specific JS objects). While I was writing new code, I did
mostly unit tests. Test driven development was amazing. I'd write a few
lines, run a test suite in less than a second using paver, and know whether
I messed up (I *recompile* my Javascript before I think about test suite
incantations now). I had a functional product almost all the time. I *loved*
refactoring code then, and I didn't stress writing perfect code. I could
hone it as I went along.

I'm an imperfect being and I know I should write much more tests than I do.
I will continue fighting the urge to just ignore tests and keep telling
myself why they are so crucial. I'm trying to be part of the solution, and
I've committed a few mochitest patches along the way. What Mozilla could do
to help me significantly is to give me great tools, and I highly recommend
they ask some of our web developers about their favorite frameworks.

FWIW, I think Jetpack is on the path to salvation. I wish I could use it for
browser work.

Ben

Matt Brubeck

unread,
Aug 10, 2011, 4:47:17 PM8/10/11
to b...@stechz.com
On 08/10/2011 10:49 AM, Benjamin Stover wrote:
> I find writing frontend tests to be an exercise in frustration. I feel that
> the test infrastructure is 1) unreliable (land something, random orange,
> backout) 2) takes a long time to write and run tests (could be days and days
> of sending to try because there's some leak on Windows) 3) no proper line
> between functional and unit tests and 4) just plain no fun.

This is so true. As a fellow front-end Fennec developer, I find the
test tools I'm using do not really work for test-driven development. To
get the most benefit from tests, I want to be able to run a test I'm
working on in <1 second (instead of >1 minute), so that I will re-run it
after every code modification.

I wonder if we could improve this by writing more of our front-end code
as really separated modules that do not depend on global browser objects
or chrome windows, so they could be tested without launching the full
browser application.

I haven't actually written any tests except for browser-chrome
mochitests. Should we be writing more xpcshell tests instead? Could we
have pure JS tests that can run in Spidermonkey without depending on
Gecko at all? Why is a test like browser_Geometry.js (testing the
Geometry.jsm module) written as a browser-chrome mochitest?

Kyle Huey

unread,
Aug 10, 2011, 4:58:45 PM8/10/11
to Matt Brubeck, dev-pl...@lists.mozilla.org
On Wed, Aug 10, 2011 at 4:47 PM, Matt Brubeck <mbru...@mozilla.com> wrote:

> On 08/10/2011 10:49 AM, Benjamin Stover wrote:
>
>> I find writing frontend tests to be an exercise in frustration. I feel
>> that
>> the test infrastructure is 1) unreliable (land something, random orange,
>> backout) 2) takes a long time to write and run tests (could be days and
>> days
>> of sending to try because there's some leak on Windows) 3) no proper line
>> between functional and unit tests and 4) just plain no fun.
>>
>
> This is so true. As a fellow front-end Fennec developer, I find the test
> tools I'm using do not really work for test-driven development. To get the
> most benefit from tests, I want to be able to run a test I'm working on in
> <1 second (instead of >1 minute), so that I will re-run it after every code
> modification.
>

If you're modifying front end code (read: not C++) why is it taking you over
1 minute to run a test?

I wonder if we could improve this by writing more of our front-end code as
> really separated modules that do not depend on global browser objects or
> chrome windows, so they could be tested without launching the full browser
> application.
>

Maybe. I don't know enough about the front end to answer.

I haven't actually written any tests except for browser-chrome mochitests.


Heh, that's a shame.


> Should we be writing more xpcshell tests instead?


Maybe.


> Could we have pure JS tests that can run in Spidermonkey without depending
> on Gecko at all?


Unlikely.


> Why is a test like browser_Geometry.js (testing the Geometry.jsm module)
> written as a browser-chrome mochitest?


I don't know. Ask stechz? :-P

That could probably be written as an xpcshell test (it doesn't look like
Geometry.jsm does anything other than math) ...

- Kyle

Mike Shaver

unread,
Aug 10, 2011, 5:05:42 PM8/10/11
to Kyle Huey, dev-pl...@lists.mozilla.org, Matt Brubeck
On Wed, Aug 10, 2011 at 4:58 PM, Kyle Huey <m...@kylehuey.com> wrote:
> On Wed, Aug 10, 2011 at 4:47 PM, Matt Brubeck <mbru...@mozilla.com> wrote:
>> Could we have pure JS tests that can run in Spidermonkey without depending
>> on Gecko at all?
>
> Unlikely.

>
>>  Why is a test like browser_Geometry.js (testing the Geometry.jsm module)
>> written as a browser-chrome mochitest?
>
> That could probably be written as an xpcshell test (it doesn't look like
> Geometry.jsm does anything other than math) ...

Seems like it could run in Spidermonkey, since it doesn't depend on
Gecko at all...

Mike

William Lachance

unread,
Aug 10, 2011, 5:10:21 PM8/10/11
to
On 08/10/2011 04:47 PM, Matt Brubeck wrote:
>
> I haven't actually written any tests except for browser-chrome
> mochitests. Should we be writing more xpcshell tests instead? Could we
> have pure JS tests that can run in Spidermonkey without depending on
> Gecko at all? Why is a test like browser_Geometry.js (testing the
> Geometry.jsm module) written as a browser-chrome mochitest?

For your first question at least, the answer is yes. :)

https://developer.mozilla.org/en/Mochitest#Try_to_avoid_Mochitest

Will

Kyle Huey

unread,
Aug 10, 2011, 5:10:06 PM8/10/11
to Mike Shaver, dev-pl...@lists.mozilla.org, Matt Brubeck
On Wed, Aug 10, 2011 at 5:05 PM, Mike Shaver <mike....@gmail.com> wrote:

> On Wed, Aug 10, 2011 at 4:58 PM, Kyle Huey <m...@kylehuey.com> wrote:
> > On Wed, Aug 10, 2011 at 4:47 PM, Matt Brubeck <mbru...@mozilla.com>
> wrote:

> >> Could we have pure JS tests that can run in Spidermonkey without
> depending
> >> on Gecko at all?
> >

> > Unlikely.


> >
> >> Why is a test like browser_Geometry.js (testing the Geometry.jsm
> module)
> >> written as a browser-chrome mochitest?
> >

> > That could probably be written as an xpcshell test (it doesn't look like
> > Geometry.jsm does anything other than math) ...
>
> Seems like it could run in Spidermonkey, since it doesn't depend on
> Gecko at all...
>

Does the shell handle JSMs? I thought all that logic was in XPConnect.

- Kyle

Mike Shaver

unread,
Aug 10, 2011, 5:11:07 PM8/10/11