Android/ARM Firefox Performance: where we are and where to go
David Mandelin
Firefox performance on mobile devices. This is really just a start, but
I think it points the way for the performance efforts we have to make to
make Firefox just as great on Android/ARM as it is on desktop. I give an
evaluation of performance today, recommendations for top performance
goals, some big changes we might need, and last, a draft plan, really
just a sketch, of the way forward.
Dave
0. Summary
If you have a powerful device, Firefox performance is in many ways
pretty good. But UI responsiveness and memory usage seem to be in pretty
bad shape, although they are hard to measure. So we need to get better
measurements and start improving performance in those areas, today.
If you don't have a powerful device, things are worse. General
improvements--especially in memory, the prime suspect--should help us
there. But I think we also need to get the message out about what kinds
of hardware give a good Firefox experience, so that we don't have people
trying it on devices where it's just not good yet, and deciding that
we're no good.
A. The State of Performance on Firefox for Android
Unless specified otherwise, all numbers for Firefox are for Firefox 6 on
my Atrix.
A1. Footprint: Acceptable.
Installed footprint is 14 MB. Opera Mobile is 12 MB, so it seems we are
not too bad. There are a lot of complaints about this in the App Store,
but I don't think this is going to make or break us.
Taras told me we could shave off a few MB by eliminating dead code and
making some toolchain tweaks. I don't know how much effort would be
involved.
A2. Startup: Not Good.
Startup time is 5s. Opera is about the same. Stock is 1.5s. There are a
lot of complaints about this in the App Store. For my personal use, I
don't care too much, because I only start the browser once per crash.
But it probably matters at least somewhat.
I had assumed that stock was faster because it used special tricks to
get pre-started, and so there was not much we could do (taking Opera's
similar score as evidence we are about as good as we can be). But Taras
says that we should be able to get down to 2s on his Galaxy S. He listed
these items as wasted time in startup:
- 0.5s XUL reflows
- 0.5s sqlite IO (150-1000ms, avg 500)
- ~0.5s linker badness (needs linker improvements)
- unknown needlessly parsing XML, CSS, etc.
I also tested clicking the app icon when the browser is already open
(maybe this is "warm startup"--not sure what the right term is). I found
that stock is the fastest, with almost no delay, Opera Mobile has a
delay, and Firefox is clearly the slowest. So that area needs help too.
A3. Memory Usage: Unknown, probably bad.
Memory statistics on Android are hard to understand. When I wrote the
first draft, I just checked the browser as it happened to be on my
phone, with a few tabs open. 'Task Manager' showed 'RAM: 64 MB', but
about:memory showed 'resident' (which I believe is supposed to give the
amount of memory the process has currently in RAM) of about 80 MB for
the main process and 40 MB for the content process. I also see these
problems with our memory reporting:
- 'resident' tends to be about 2x 'heap-committed'. On desktop the
numbers are about equal. So either the reporter is wrong or these
numbers mean something else on Android.
- 30-40% of explicit allocations are 'unclassified'.
Getting better information here (higher quality data and/or better
understanding of it) should be a high priority.
For now, based on what information we do have, memory usage on Android
seems pretty bad. Some indirect evidence: I get a lot of crashes, which
I think might be OOMs. Also, my phone has 1 GB RAM while Taras's has 512
MB, and I seem to get better perceived performance, which points to
memory limits. (Taras says other apps, including the music player, get
tossed out of RAM when he uses Firefox.) Finally, comments in the App
Store suggest that on wimpy devices, Firefox is much slower than other
browsers, which appears not to be the case on powerful devices.
Looking at the direct measurements, both Taras and I think that ~80 MB
for the chrome part of the browser is unacceptable. Taras says he thinks
20 MB (of which 10 MB is for JVM) would be a good target. He also says
that XUL seems to be responsible for about half the memory we do use.
A4. Pageload: Acceptable to Good, but maybe not for everyone.
I tested Firefox against Opera Mobile and stock by loading 8 pages and
timing the interval between 'click enter' and 'pageload animation stops'
with a stopwatch. Firefox beat both of them on 6 of 8 tests (but not the
same 6). stock was faster on engadget and a bit faster on nytimes, and
Opera was faster on walmart and a hair faster on loading a techcrunch
article.
Taras seems less sanguine about this--I think his device's smaller RAM
may be the difference. We should get more information about how pageload
looks on different devices and device classes.
Taras also said that Alon Zakai told him IPC overhead between main and
content processes is hurting us here.
A5. JavaScript: Acceptable to Good.
Current benchmark results:
SunSpider-0.9.1 V8-v6
Firefox 2031 306
Opera Mobile 2254 318
Stock 2.3 4219 336
We're the best on SunSpider, beating stock by over 2x. We're slower on
V8 but still within 10% of both competitors.
Current projects (IonMonkey, Generational GC) should give us a big boost
on V8 on all platforms, including Android/ARM. Mainly, we need to get
good support for ARM codegen in IonMonkey from the beginning, and see if
we need any Android-specific GC tuning.
A6. Panning/Zooming: Bad/OK.
I get lots of checkerboarding if I pan quickly, moreso than in other
mobile browsers. Zooming is OK for me, but I do see complaints about it
in the app store, and the graphics quality during zooming is lower than
stock, so maybe we are using that to compensate for lower underlying
performance.
A7. UI Responsiveness: Unknown, probably bad.
The UI often seems sluggish, and I hear this a lot from other people.
But I tried a simple eyeball test of various UI clicks (e.g., clicking
the location bar) against Opera Mobile and stock, and it seems to me
that the time from 'click' to 'thing I wanted is visible and ready to
use' is usually about the same, with two exceptions:
- the first time I use a feature (e.g., add-ons manager), it is extra slow
- sometimes there is just a long delay
So there are a couple of concrete problems, but otherwise it seems OK.
But users still seem to perceive Firefox to be unresponsive, and that
may be the bigger problem:
- Firefox is unique among mobile browsers in not having a progress bar.
- I notice that Opera Mobile and stock animate other things: e.g., if I
click the options button (the O button on Opera), the option panel
smoothly slides up from the bottom, and the sliding starts instantly on
the click. But in Firefox, nothing happens for a brief interval, then
the panel appears.
B. Performance Goals
I propose these broad goals and priorities for mobile performance:
Top Goals (start immediately on all of them):
B1. Fully understand memory usage. This means: know what all the
different memory metrics mean, make sure they all fit together, and
understand how different features of Firefox's architecture and design
affect memory usage.
B2. Reduce memory usage, a lot. We don't quite know this is a valid goal
at this point, but if it's as bad as it looks, it can't wait until B1 is
done.
B3. Improve perceived performance. This can start with a progress bar
for pageload and some more animation. Presumably someone else knows a
lot more about this.
B4. Fix panning performance. I've heard it used to be worse, so someone
out there probably knows something about this.
Secondary Goals:
B5. Improve startup time.
B6. Improve pageload time.
B7. Improve JavaScript performance.
B8. Reduce footprint.
C. Crazy Ideas
The first two aren't really crazy--I think they need to be taken
seriously. The third is more of a research idea and a long-term bet, but
I still want to mention it here.
C1. Non-XUL Front End Architecture.
Taras found that XUL about doubles our memory usage. That's pretty bad.
It also likely slows things down. I think we should seriously consider
creating a new, non-XUL front end for Android.
A key question is, "Can we ever have competitive performance with a
XUL-based front-end". If the answer is "no", then we must start now on a
new front end. If the answer is "maybe", then we should probably still
start now on a new front end.
C2. Single-Process Architecture.
We apparently have evidence that IPC is slowing us down. I don't have
any measurements there, so I don't know much about the problem. But
again, we need to think hard about whether the current multiprocess
architecture is going to get us to where we want to be (taking into
account any predicted improvements in Android IPC) and if it won't, it's
time to do something about that.
C3. Mobile Performance Research.
The discussion in this document so far is focused on closing the gap
where we are behind, and incrementally improving performance elsewhere.
But we can think bigger on performance, in particular about closing the
gap between mobile and desktop.
For example, you can use the Netflix queue management page from a tablet
or smartphone, but it's painfully slow--*barely* usable. What can we do
to make it fully usable? I think we want HTML5 to be the thing, not
closed app stacks. We could look into how to make something like the
Netflix app in HTML5, but faster. We could look into changes to the
standards that make it easier to run fast on mobile.
D. Draft Plan
Definitely start now:
D1. Make about:memory make sense. This means fixing the reporters,
understanding what the numbers mean and documenting it, or both.
D2. Make mobile memory usage top priority for MemShrink.
D3. Make the call on whether to start work on a non-XUL front end. My
gut says a new front end is the only way and we should just do it, but
I'm not particularly well-informed. Notice that I'm not saying to make
the call on whether to *use* the non-XUL front end--we do that only when
we have it. We can hedge bets in various ways depending on opportunities
and risks.
D4. Make the call on going to a single-process architecture.
D5. Add a progress bar for pageload.
D6. Add animations for UI actions.
Other stuff that seems important, and could be started now, but I'm less
sure about, or isn't as urgent:
D7. Develop tools to visualize latencies (like the timeline views that
webdevs get now) to help figure out how to make things faster. Taras
says we don't need the general tool, just direct instrumentation as
needed. This is probably true, but the tool might still have value.
D8. Get oprofile working on Android and teach people how to use it.
Taras says that it won't help with latencies. He's probably right. There
might still be some use for this, but I'm not sure.
D9. Fix panning performance.
D10. Dig in on cold startup time.
D11. Dig in on "warm startup" time.
D12. Add tools to directly expose resource usage in the UI: CPU, memory,
network, etc. This way users and devs can see what's happening as the
use the product, and find opportunities to improve. Think tools like
Framerate monitor.
D13. Collect data on pageload times: stats on times in the wild, and
comparisons with other browsers.
D14. Dig in pageload time: measure the latencies and figure out how to
make it faster on pages of interest.
D15. Dig in on long UI pauses.
D16. Spin up research on mobile performance.
--end