Startup Wrangling
Vladimir Vukicevic
Shortly before our office move, we kicked off an effort to take a hard
look at our startup time, to both understand what we all do, and to
figure out how to improve it. zpao (Paul O'Shannessy), ddahl (David
Dahl), and I have been working towards a few goals:
- Document how to reproducibly get a cold and warm startup on
Windows (XP/Vista/7), MacOS X, and Linux
- Create tools to capture both JS execution during startup, as well as
file IO
- Add instrumentation to firefox to identify "big blocks" of startup
for timing
- Create tools to visualize the captured data in a way that's easy to
analyze
One thing that's fairly obvious with playing with startup is that
"warm" startup is significantly faster than "cold" startup; that is,
when you've launched Firefox before, the OS caches a bunch of the data
off the disk, and it doesn't have to hit the disk again. This
directly points to IO being a major component of our startup time,
which is why IO is part of the capture above. This is a pretty big
problem even on desktop systems; on my fairly beefy Windows 7 box, a
cold startup takes upwards of 12 seconds (!); warm startup is also
fairly slow if the system is under load.
We've fixed some bugs in our dtrace javascript provider along the way
(bug 403345),
so dtrace will actually give correct (and sane) data now. Also, I've
been doing a lot of work with Microsoft's xperf (part of the Windows
Performance Toolkit), which can capture much the same data. (In
theory we should be able to create JS providers for xperf as well, but
that's out of scope for this particular project.)
One example of the type of data we're capturing and tools that we're
building is
http://people.mozilla.com/~vladimir/misc/startviz/startviz.html --
this is just a quick io capture with xperf, with the data dumped into
a Timeline widget from the SIMILE project. (The time scales are a bit
off; the raw data is in microseconds, but SIMILE only handles
milliseconds... so all times need to be divided by 1000, which becomes
a problem when you go over 60 seconds -- which is actually just 60 ms!
Something that we'll fix.)
Another example is the result of a startup trace; zpao is still
working on the visualization and data capture, but you can see an
early version at http://playground.zpao.com/dtrace_treemaps2/ -- the
"Exclusive function elapsed times" view will provide the most accurate
data, basically telling you "how long did we spend in a given
function, ignoring all descendants". In this view, the "null"
filename dominates, generally indicating native code. And within
that, calls to "getService" also dominate, which indicates that much
of the time is spent within getService, presumably initializing
whatever the requested service is.
In the future, we hope to have hierarchy correctly represented in the
inclusive view, as well as adding IO operations as part of that
hierarchy. Also, these tools aren't really limited to analyzing startup;
they will hopefully form the basis of a set of javascript performance
analysis tools that we can apply to any browser operation.
Besides IO and JS, Taras Glek found in earlier examinations of startup
that loading CSS/XBL/etc. was taking a significant amount of time.
We're working on instrumenting those parts of the code as well, so that
we can capture it along with the raw js/io/etc. portions.
Is there any other data that we should be capturing? Let us know, and
we'll see if we can figure out how to add it in. I'll keep posting
updated data as we have it, and will probably create a web page to
collect it all -- at that point it'll be open season on any issues that
can be identified.
- Vlad
John J. Barton
>
> - Create tools to capture both JS execution during startup, as well as
> file IO
Since I start up FF about 8 billion times a day I can see all of the JS
execution during start up if you are interested. Profiling it without
impacting is it harder.
One thing I believe really impacts start up time is errors, but that may
because I have four error consoles running.
jjb
John J. Barton
> Vladimir Vukicevic wrote:
>>
>> - Create tools to capture both JS execution during startup, as well as
>> file IO
>
> Since I start up FF about 8 billion times a day I can see all of the JS
> execution during start up if you are interested. Profiling it without
> impacting is it harder.
Sorry I forgot to link to the code that may help you:
http://code.google.com/p/fbug/source/browse/chromebug/branches/chromebug1.5/components/chromebug-startup-observer.js
jjb
Michael Lefevre
>
> Shortly before our office move, we kicked off an effort to take a hard
> look at our startup time, to both understand what we all do, and to
> figure out how to improve it. zpao (Paul O'Shannessy), ddahl (David
> Dahl), and I have been working towards a few goals:
Well, hurrah. I was looking out for some mention of start up time. The
Tsnap stuff is cool, but I'm rather more concerned about Firefox taking
35 seconds to start up (4 year old PC with 6 months of history) than I
am about saving 50ms here and there in use.
> One thing that's fairly obvious with playing with startup is that
> "warm" startup is significantly faster than "cold" startup; that is,
> when you've launched Firefox before, the OS caches a bunch of the data
> off the disk, and it doesn't have to hit the disk again. This
> directly points to IO being a major component of our startup time,
> which is why IO is part of the capture above. This is a pretty big
> problem even on desktop systems
[...]
Indeed. Not sure I'm adding anything here, as it sounds like you
already have this covered. But from a little amateur poking around I
did with CPU and disk monitors, during my 35 second start up, my CPU is
mostly doing nothing except waiting for the hard drive (warm start has
no disk activity, and start time is then around 3 seconds).
A large amount of the disk access seemed to be with the places database
(taking up something like 20 seconds - my system can actually read the
whole places file into memory and write it out again in 10 seconds, so
that seems a little crazy), and also a lot of time getting Windows to
access all 182 font files on my machine (repeatedly, I think, but I
guess that doesn't matter so much if it hits the disk cache after the
first time). As far as I could see, it was Windows doing all the font
file I/O presumably as a result of Firefox asking Windows about fonts,
so hopefully your measurements will pick that up. My measurements may
not have been terribly useful, because I was just looking at when a big
list of file operations, probably happening in parallel to some extent,
without knowing how long each was taking.
Anyway, happy that you're looking at it, and if there's any user-level
testing that can be done...
Michael
Ciprian M.
that are often used, I would create a cache.zip and I will save them
on shutdown of firefox. This cache.zip should save read only files
that firefox uses. This optimization will remove the seek time of file
IO (as all are in one file), also being zip, will be supposedly much
more smaller, so less chance to be in a fragmented file. At the end,
firefox can optimize it's usage profile by using this cache and count
the used files and startup performance.
Sounds pretty silly optimization but I think it may improve as it will
require more CPU (for extracting zip files) but less disk seek.
Great job with FF, I love this new FF 3.5 Preview. I already wait to
release it.