[FBTF] Things you can do to speed to prevent the build from getting slower
Elliot Glaysher (Chromium)
reminder on things that you can do to not slow down the build:
PART 1: Forward declare things instead of including headers
-----------------------------------------------------------------------------------------
1) Don't include headers you don't need. Especially in headers.
I've seen several places where a header will include something, like <deque> or
"googleurl/src/gurl.h" and then not use the std::deque or GURL in the
file. These are probably oversights from previous refactorings or
reorganizations where the files used to be necessary, but they should
be fixed whenever they are seen.
2) Forward declare classes instead of including headers.
You can use forward declarations in all the following cases:
class Forward;
class ObjectUser {
public:
ObjectUser(Forward by_value);
void OrByPointer(Forward* by_pointer);
void OrByReference(const Forward& by_reference);
private:
Forward* pointer_member_;
scoped_ptr<Forward> most_smart_pointer_templates_;
std::vector<Forward> some_stl_containers_;
// But you can't just:
// Forward cant_full_member_;
// std::deque<Forward> cant_some_other_stl_containers_;
};
Whenever you can forward declare, you should forward declare instead of
including a header.
3) Move inner classes into the implementation.
You can also forward declare classes inside a class:
class Whatever {
public:
/* ... */
private:
struct DataStruct;
std::vector<DataStruct> data_;
};
Any headers that DataStruct needed no longer need to be included in the header
file and only need to be included in the implementation. This will
often let you pull
headers includes out of the header. For reference, the syntax in the
implementation file is:
struct Whatever::DataStruct {
};
Note that sometimes you can't do this because a handful of STL data structures
require the full definition at declaration time (most notably, std::deque and
the STL adapters that wrap it).
4) Move static implementation details to the implementation whenever possible.
If you have the class in a header file:
#include "BigImplementationDetail.h"
class PublicInterface {
public:
/* ... */
private:
static BigImplementationDetail detail_;
};
You should try to move that from a class member into the anonymous namespace in
the implementation file:
namespace {
BigImplementationDetail detail_;
} // namespace
That way, people who don't use your interface don't need to know about or care
about BigImplementationDetail.
You can do this for helper functions that have hard to forward declare types,
too.
PART II: Stop inlining code
-------------------------------------------------------------------------------
BACKGROUND: We're doing too much inlining. Remember that definitions in classes
are requested to be inlined. Remember that asking the compiler to inline a
method is a suggestion, one that's often ignored by the compiler. A compiler
that never inlined anything would be perfectly standards compliant. Every file
that has to use an inline method will also emit a function version in the
resulting .o, even if the method was inlined. (This is to support function
pointer behaviour properly.)
class InlinedMethods {
InlinedMethods() {
// This constructor is equivalent to having the inline keyword in front
// of it!
}
void Method() {
// Same here!
}
};
5) Stop inlining constructors and destructors.
Constructors and destructors are often significantly more complex than you
think they are, especially if your class has any non-POD data members. Many STL
classes have inlined constructors/destructors which may be copied into your
function body. Because the bodies of these appear to be empty, they often seem
like trivial functions that can safely be inlined. Don't give in to this
temptation. Define them in the implementation file unless you really _need_
them to be inlined. Even if they do nothing now, someone could later add
something seemingly-trivial to the class and make your hundreds of inlined
destructors much more complex.
For more information, read Item 30 in Effective C++.
Even worse though, inlining constructors/destructors prevents you from using
forward declared variables.
class Forward;
class WontCompile {
public:
// THIS WON'T COMPILE, BUT IT WOULD HAVE IF WE PUT THESE IN THE
// IMPLEMENTATION FILE!
//
// The compiler needs the definition of Forward to call the
// vector/scoped_ptr ctors/dtors.
Example() { }
~Example() { }
private:
std::vector<Forward> examples_;
scoped_ptr<Forward> super_example_;
};
6) Stop inlining complex methods.
class DontDoThis {
public:
int ComputerSomehting() {
int sum =0;
for (int i = 0; i < shurg; ++i) {
sum += OtherMethod(i, ... );
}
return sum;
}
private:
/* ... */
};
Anything more than a few operations on integral data types will probably not be
inlined by the compiler. Remember that inline is a suggestion, and that you're
just adding crud to the .o files which the linker will need to do work to
resolve.
If the method deals with implementation details, there's also a good chance
that you could eliminate some included headers in the header.
7) Stop inlining virtual methods.
You can't inline virtual methods under most circumstances, even if the method
would otherwise be inlined because it's very short. The compiler must do
runtime dispatch on any virtual method where the compiler doesn't know the
object's complete type, which rules out the majority of cases where you have an
object.
Thanks,
-- Elliot
Lei Zhang
> Chromium Developers mailing list: chromi...@chromium.org
> View archives, change email options, or unsubscribe:
> http://groups.google.com/a/chromium.org/group/chromium-dev
>
Elliot, thanks for this write-up of the issues with headers and
includes. On this topic, I have a couple other suggestions to add:
8. When you copy an existing file as the template for your new code,
be sure to remove the headers you're not using. I've seen many cases
of this where most of the files in a directory have the same set of
unneeded includes that all came from an existing file in that
directory.
9. When refactoring code, i.e. moving code from base/win_util.h to
app/win_util.h, remember to remove the base/win_util.h include if
you're no longer using anything from it. Same for when you're removing
code - check and see if the code being removed is the last user of the
headers in that file.
Nico Weber
Mike Pinkerton
guide" page on dev.chromium.org?
--
Mike Pinkerton
Mac Weenie
pink...@google.com
--
Mike Pinkerton
Mac Weenie
pink...@google.com
Lei Zhang
I looked at this briefly too and decided parsing C++ is hard.
James Hawkins
Darin Fisher
7) Stop inlining virtual methods.
You can't inline virtual methods under most circumstances, even if the method
would otherwise be inlined because it's very short. The compiler must do
runtime dispatch on any virtual method where the compiler doesn't know the
object's complete type, which rules out the majority of cases where you have an
object.
Elliot Glaysher (Chromium)
> Does this seem useful? We can analyze the output and answer questions
> like "how many includes do we have to preprocess when building
> Chromium?", and "how many object files includes foo.h?"
My immediate hunch is yes, but I'm not sure how the raw data would be
visualized. Maybe some sort of munging with dot?
-- Elliot
Lei Zhang
In terms of measuring the number of includes, I have a python script
[1] that one can use as a replace $CC/$CXX. It generates fake output
files to keep Make happy, and instead runs g++ with -H to get the
headers used.
You run it as:
CC="deps.py gcc" CXX="deps.py g++" make
and it intercepts and rewrites the g++ command with -H -E. It sends
the preprocessor output to /dev/null, and saves header information to
a file next to the dummy .o file. There's also a few special cases to
deal with like ncdecode_table and mksnapshot.
On my slower workstation, this takes ~8 minutes to run with -j4. For
every .o file, we get a second file with the list of headers used.
i.e. for base/memory_debug.o, we have the output below.
Does this seem useful? We can analyze the output and answer questions
like "how many includes do we have to preprocess when building
Chromium?", and "how many object files includes foo.h?"
[1] http://leiz.org/chromium/deps.py
. ./base/memory_debug.h
.. ./base/basictypes.h
... /usr/lib/gcc/x86_64-linux-gnu/4.2.4/include/limits.h
.... /usr/lib/gcc/x86_64-linux-gnu/4.2.4/include/syslimits.h
..... /usr/lib/gcc/x86_64-linux-gnu/4.2.4/include/limits.h
...... /usr/include/limits.h
....... /usr/include/features.h
........ /usr/include/sys/cdefs.h
......... /usr/include/bits/wordsize.h
........ /usr/include/gnu/stubs.h
......... /usr/include/bits/wordsize.h
......... /usr/include/gnu/stubs-32.h
....... /usr/include/bits/posix1_lim.h
........ /usr/include/bits/local_lim.h
......... /usr/include/linux/limits.h
....... /usr/include/bits/posix2_lim.h
....... /usr/include/bits/xopen_lim.h
........ /usr/include/bits/stdio_lim.h
... /usr/lib/gcc/x86_64-linux-gnu/4.2.4/include/stddef.h
... /usr/include/string.h
.... /usr/lib/gcc/x86_64-linux-gnu/4.2.4/include/stddef.h
.... /usr/include/xlocale.h
... ./base/port.h
.... /usr/lib/gcc/x86_64-linux-gnu/4.2.4/include/stdarg.h
.... ./build/build_config.h
... /usr/include/stdint.h
.... /usr/include/bits/wchar.h
.... /usr/include/bits/wordsize.h
On Mon, Aug 2, 2010 at 3:09 PM, Elliot Glaysher (Chromium)
<e...@chromium.org> wrote:
Evan Martin
V=1 > headersoup.log 2>&1
Does the same thing, if you're willing to rebuild everything
I wrote an analyzer which I've uploaded here:
http://gist.github.com/511981
It does some summary statistics and attempts to tell you what the
worst offenders are.
If the analysis doesn't match what you were thinking of, at least the
parser is likely useful.
Scott Violet
I've come back to this email a number of times since you've written
it. Any chance you could incorporate the feedback on this thread and
post it some where on dev.chromium.org for all?
-Scott
On Mon, Aug 2, 2010 at 3:09 PM, Elliot Glaysher (Chromium)
<e...@chromium.org> wrote:
Elliot Glaysher (Chromium)
Elliot,
I've come back to this email a number of times since you've written
it. Any chance you could incorporate the feedback on this thread and
post it some where on dev.chromium.org for all?
Scott Violet
Thanks,
-Scott