Jetpack add-on performance tests

[Jorge Villalobos]

Hello all,

As I mentioned a couple of meetings ago, startup performance has been a
concern or mine for Jetpack add-ons since I did some testing early in
the year and got some troubling results. As requested, I ran updated
tests and here are the results. You can test this yourselves using these
directions:
https://developer.mozilla.org/en/Measuring_Add-on_Startup_Performance

There are 3 result sets, all run on the latest Firefox 9 beta on Mac OS
X 10.7.2. The first one (baseline) has no add-ons. The second has
installed an empty Jetpack add-on generated by the builder. And the
third one is a non-trivial Jetpack add-on I just picked from the update
queue (https://addons.mozilla.org/en-US/firefox/addon/infocon-watcher/).

And, good news! The results were very favorable this time, with the
empty add-on adding only 1.8% (11ms) to startup, and Infocon adding only
3.6% (22ms), on average. As a reference point, after our initial tests
we decided that 7% was roughly the line where add-on startup overhead
stopped being "Excellent" and began being just "Good".

Much more testing is necessary in order to make a clear assessment about
Jetpack add-on performance, but these results are very encouraging. It
looks like much progress has been made in the performance area.

- Jorge

PS: here's the raw data from the tests I ran:

Baseline

i,val
0,772
1,604
2,607
3,618
4,623
5,611
6,599
7,591
8,604
9,603
10,608
11,637
12,617
13,599
14,606
15,753
16,615
17,933
18,606
19,616
RETURN: ts: 625.74

Empty add-on

i,val
0,675
1,611
2,616
3,634
4,607
5,670
6,637
7,605
8,613
9,602
10,630
11,624
12,606
13,641
14,637
15,603
16,840
17,638
18,1062
19,609
RETURN: ts: 636.74

Infocon

i,val
0,691
1,658
2,615
3,611
4,620
5,621
6,621
7,623
8,641
9,621
10,610
11,759
12,748
13,647
14,625
15,641
16,700
17,624
18,748
19,647
RETURN: ts: 648.0

[Irakli Gozalishvili]

Awesome, I'm really glad to see this!!

I think Mossop was going / have created a bug to start taking such measures on every push, this will help us to keep on improving these numbers!

Mossop could you please post a bug number here ?

Regards
--
Irakli Gozalishvili
Web: http://www.jeditoolkit.com/

--

You received this message because you are subscribed to the Google Groups "mozilla-labs-jetpack" group.

To post to this group, send email to mozilla-la...@googlegroups.com.

To unsubscribe from this group, send email to mozilla-labs-jet...@googlegroups.com.

For more options, visit this group at http://groups.google.com/group/mozilla-labs-jetpack?hl=en.

[Dave Townsend]

Yeah this is awesome stuff. We have https://bugzilla.mozilla.org/show_bug.cgi?id=702684 filed to investigate ways we can automate this and get the metrics onto the Jetpack tinderbox tree. Hopefully I'll have some time to talk to people about it this week.

[Alexandre poirot]

Hi Jorge,

Thanks for taking time to test jetpack performances again!
It's even more encouraging because you tested on 1.2 jetpack release and I expect to have performances improvement with 1.4 release with the new loader that allowed to drop tons of JS code and simplify a lot the startup runtime for all addons!


++
Alex

2011/11/29 Dave Townsend <dtow...@oxymoronical.com>

Yeah this is awesome stuff. We have https://bugzilla.mozilla.org/show_bug.cgi?id=702684 filed to investigate ways we can automate this and get the metrics onto the Jetpack tinderbox tree. Hopefully I'll have some time to talk to people about it this week.

--
You received this message because you are subscribed to the Google Groups "mozilla-labs-jetpack" group.

To view this discussion on the web visit https://groups.google.com/d/msg/mozilla-labs-jetpack/-/bKlmWVIaBhwJ.

[David Mason]

Yes! +1 on the tests Jorge! I'm going to mention this at the Firefox
planning meeting tomorrow too just so folks who are interested in Perf
work can take a look as well.

On Tue Nov 29 17:25:07 2011, Alexandre poirot wrote:
> Hi Jorge,
>
> Thanks for taking time to test jetpack performances again!
> It's even more encouraging because you tested on 1.2 jetpack release
> and I expect to have performances improvement with 1.4 release with
> the new loader that allowed to drop tons of JS code and simplify a lot
> the startup runtime for all addons!
>
>
> ++
> Alex
>
>
> 2011/11/29 Dave Townsend <dtow...@oxymoronical.com

> <mailto:dtow...@oxymoronical.com>>

>
> Yeah this is awesome stuff. We have
> https://bugzilla.mozilla.org/show_bug.cgi?id=702684 filed to
> investigate ways we can automate this and get the metrics onto the
> Jetpack tinderbox tree. Hopefully I'll have some time to talk to
> people about it this week.
>
> --
> You received this message because you are subscribed to the Google
> Groups "mozilla-labs-jetpack" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/mozilla-labs-jetpack/-/bKlmWVIaBhwJ.
>
> To post to this group, send email to
> mozilla-la...@googlegroups.com

> <mailto:mozilla-la...@googlegroups.com>.

> To unsubscribe from this group, send email to
> mozilla-labs-jet...@googlegroups.com

> <mailto:mozilla-labs-jetpack%2Bunsu...@googlegroups.com>.

> For more options, visit this group at
> http://groups.google.com/group/mozilla-labs-jetpack?hl=en.
>
>

> --
> You received this message because you are subscribed to the Google
> Groups "mozilla-labs-jetpack" group.