Possibly heretic questions
Roland Bock
last week, Ted Mielczarek and Chris Lord helped me in getting started
with the headless backend, which is cool since it has much less external
dependencies.
In the meantime, spurred on by Chris' remark, that the rendering part
could be removed completely, I started to look through the code and
documentation, which leads to a bunch of questions.
I found several hints in the [XP]COM documentation to Scott Meyers (e.g.
https://developer.mozilla.org/en/Using_nsCOMPtr/Getting_Started_Guide).
Great books, I am using the second edition here:
*) Effective C++, Item 14: Make sure base classes have virtual destructors.
When I compile the code, I get tons of warnings like
nsIRequestObserver.h:32: warning: �class nsIRequestObserver� has virtual
functions but non-virtual destructor
(using /usr/local/bin/gcc -Wall -Wextra)
*) More Effective C++, Item 35: Familiarize yourself with the language
standard
I would not claim to be a C++ guru and I know that ten years back the
C++ standard was not very well supported, but nowadays? Most modern
compilers support the STL very well and even extensions from TR1. Are
there any plans to switch to using such standards instead of the own
classes? E.g.
nsCOMPtr => std::tr1::shared_ptr
nsIInputStream => std::istream
nsCOMArray<T> => std::vector<std::tr1::shared_ptr<T> >
...
I understand that the code evolved to use own classes because the
support for the standard was quite bad a few years ago. But this has
changed a lot.
I consider the mozilla code and the products which are created by them
pretty awesome. I wonder though, if it wouldn't be a good idea to
through a few of the homemade classes overboard and replace them by the
standard classes. At least for people like me it would make it much
easier to concentrate on the essential topics within the software.
Sorry, if the question is not appropriate, but I found practically
nothing regarding the use of STL in mozilla.
If this is not the right list to discuss such a topic, please let me
know which would be the correct one.
Thanks and regards,
Roland
Boris Zbarsky
> *) Effective C++, Item 14: Make sure base classes have virtual destructors.
> When I compile the code, I get tons of warnings like
>
> nsIRequestObserver.h:32: warning: �class nsIRequestObserver� has virtual
> functions but non-virtual destructor
This only matters for classes being deleted through the base class
pointer. Since no one will ever be deleting anything through an
nsIRequestObserver pointer, it's not an issue.
That said, we could change xpidl to generate an empty virtual destructor
here just to shut up the compiler warning. Might be a good idea. Bug
filed?
> *) More Effective C++, Item 35: Familiarize yourself with the language
> standard
> I would not claim to be a C++ guru and I know that ten years back the
> C++ standard was not very well supported, but nowadays? Most modern
> compilers support the STL very well and even extensions from TR1. Are
> there any plans to switch to using such standards instead of the own
> classes? E.g.
>
> nsCOMPtr => std::tr1::shared_ptr
> nsIInputStream => std::istream
> nsCOMArray<T> => std::vector<std::tr1::shared_ptr<T> >
This has been discussed in the past (including in this very newsgroup,
iirc). Please search?
As a brief summary, a minimal requirement for this is enabling
exceptions (because the STL requires them) and rewriting a lot of code
to deal with that. There are other issues, but that's a first hurdle.
-Boris
Neil
> Roland Bock wrote:
>
>> nsIRequestObserver.h:32: warning: �class nsIRequestObserver� has
>> virtual functions but non-virtual destructor
>
> That said, we could change xpidl to generate an empty virtual
> destructor here just to shut up the compiler warning. Might be a good
> idea.
Minuscule performance hit, if you're going to force the destructor to be
called through the vtable. (And removes any chance of inlining it, too.)
--
Warning: May contain traces of nuts.
Boris Zbarsky
> Minuscule performance hit, if you're going to force the destructor to be
> called through the vtable. (And removes any chance of inlining it, too.)
The typical nsISupports-derived concrete class already has a virtual
destructor, though. So I doubt this would change any generated code in
practice.
-Boris
dbradley
We just had this debate in Songbird. And was briefly worried. Then
realized that all XPCOM classes are destroyed via Release,which is
always virtual so the virtual destructor isn't really an issue except
in unusual cases.
Say I have class B which is derived from an interface and provides
some common bits including an AddRef/Release. This provided a common
Release implementation. Then I derived class C from this for a
specific implementation. Now, class B is doing the destruction, but if
for some reason it didn't declare a destructor then C's destructor
would not get called. It is a bit contrived.
That sparked another idea. I wonder how much all those AddRef/Release
implementations take up in code space. I know they're small, but there
are a lot of them. Wonder if much code space would be gained by
sharing the implementation. Unfortunately the logging aspect would
become problematic.
L. David Baron
> This only matters for classes being deleted through the base class
> pointer. Since no one will ever be deleting anything through an
> nsIRequestObserver pointer, it's not an issue.
>
> That said, we could change xpidl to generate an empty virtual destructor
> here just to shut up the compiler warning. Might be a good idea. Bug
> filed?
I think it's not a good idea since it would change the vtable layout
of frozen interfaces -- at least those that are derived from other
interfaces, and perhaps all -- and also bloat some vtables.
-David
--
L. David Baron http://dbaron.org/
Mozilla Corporation http://www.mozilla.com/
Roland Bock
>> nsIInputStream => std::istream
>> nsCOMArray<T> => std::vector<std::tr1::shared_ptr<T> >
>
> This has been discussed in the past (including in this very newsgroup,
> iirc). Please search?
> As a brief summary, a minimal requirement for this is enabling
> exceptions (because the STL requires them) and rewriting a lot of code
> to deal with that. There are other issues, but that's a first hurdle.
OK, I am aware of the obstacles, and I wasn't suggesting to switch to
STL-usage by tomorrow, of course. I was rather thinking in longer terms,
say the major release after the next (maybe even later).
I found additional stuff from the archives in the meantime, thanks for
insisting :-)
For instance, the standard libraries, exceptions, etc are dicussed in
the portability guide
https://developer.mozilla.org/index.php?title=en/C%2b%2b_Portability_Guide
and related threads.
Interestingly enough, until November last year, the portability guide
explicitly stated that namespaces are not to be used (because they are a
relatively new feature of C++).
The same guide states that exceptions and the Standard Libs are not to
be used.
There also is a discussion from April this year (I guess that's the one
you meant). The question whether exceptions or STL could be used, is
answered with
"Why? We turn exceptions and RTTI off, so you cannot use them in our
codebase."
With such a mindset, one wonders why anybody ever bothered to improve
the C++ standard at all? Why not stick forever with what we had 10 years
ago?
I am aware that its not easy to change a big system like mozilla. I know
that it cannot be done within a few weeks.
But I would be willing to contribute. I need to refactor some of the
html parser code anyways in order to use it in my project. The main
problem being the horrific amount of dependencies which sum up to some
27MB of include files and 30MB of library files just to parse html files
into my own nsIContentSink. I need to boil that down a lot. I hope to
end up with considerably less than 10MB in total :-)
I am sure others would be willing to contribute, too. And I am convinced
it would help the mozilla project in the long run to concentrate on the
mozilla-specific stuff, while others concentrate on the standard.
Regards,
Roland
Roland Bock
> here just to shut up the compiler warning. Might be a good idea. Bug
> filed?
OK, thanks for the feedback (David's, too). I'll file a bug to get rid
of these warnings, hoping that the issue of vtable
modifictation/bloating is considered small compaired to the warnings :-)
Regards,
Roland
Roland Bock
> On Wed, Jun 3, 2009 at 4:07 AM, Roland Bock <rb...@eudoxos.de
> <mailto:rb...@eudoxos.de>> wrote:
>
>
> There also is a discussion from April this year (I guess that's the
> one you meant). The question whether exceptions or STL could be
> used, is answered with
>
> "Why? We turn exceptions and RTTI off, so you cannot use them in our
> codebase."
>
>
> With such a mindset, one wonders why anybody ever bothered to
> improve the C++ standard at all? Why not stick forever with what we
> had 10 years ago?
>
>
> inertia. For example, as you noted, we just recently started widely
> using namespaces. However, other things have been investigated, and
> there are obstacles to their use. The last time someone looked into
> enabling C++ exceptions (which is a prerequisite to using the STL) there
> was an unacceptable performance hit. If you or someone else would like
> to repeat this test to see if it's still a factor, that would provide
> some of the necessary data to make it possible to make that sort of switch.
If exceptions still reduce performance in such a way, it would be a very
valid reason indeed.
Do you know if the sources for this test are available for download
somewhere? I could certainly throw in the results for my platform here
(gcc 4.2.4 on Ubuntu8.04, 64bit).
Regards,
Roland
Ted Mielczarek
>
> There also is a discussion from April this year (I guess that's the one you
> meant). The question whether exceptions or STL could be used, is answered
> with
>
> "Why? We turn exceptions and RTTI off, so you cannot use them in our
> codebase."
>
>
> With such a mindset, one wonders why anybody ever bothered to improve the
> C++ standard at all? Why not stick forever with what we had 10 years ago?
>
>
There are certainly some things that haven't changed simply due to inertia.
For example, as you noted, we just recently started widely using namespaces.
However, other things have been investigated, and there are obstacles to
their use. The last time someone looked into enabling C++ exceptions (which
is a prerequisite to using the STL) there was an unacceptable performance
hit. If you or someone else would like to repeat this test to see if it's
still a factor, that would provide some of the necessary data to make it
possible to make that sort of switch.
-Ted
Roland Bock
[...]
> That sparked another idea. I wonder how much all those AddRef/Release
> implementations take up in code space. I know they're small, but there
> are a lot of them. Wonder if much code space would be gained by
> sharing the implementation. Unfortunately the logging aspect would
> become problematic.
Actually, reference counting and finally releasing resources is what the
std::tr1::shared_ptr is doing. Being used to this, the AddRef/Release
mechanism seems counter-intuitive, since every class that is to be used
with XPComPtr etc. needs to implement these methods. With shared_ptr,
you can handle pointers to everything.
Of course, since the shared_ptr is a template, the respective code is
still multiplied even though you don't see it in the source.
On the other hand, I assume that due to inlining, the AddRef/Release
code is multiplied, too.
Boris Zbarsky
> Do you know if the sources for this test are available for download
> somewhere? I could certainly throw in the results for my platform here
> (gcc 4.2.4 on Ubuntu8.04, 64bit).
I believe you just need to add --enable-coo-rtti and
--enable-cpp-exceptins to the ac_add_options in your mozconfig.
-Boris
Boris Zbarsky
Actually, the other way around. David's point about frozen interfaces
(All of them, since xpidl would generate a dtor for nsISupports as well
with my proposed change) changing vtable layout makes this a non-starter
unless we're actually saying that we're breaking our binary compat promises.
-Boris
Roland Bock
I had to use --enable-cpp-rtti instead of --enable-coo-rtti (the latter
should imply the former, I guess, but it doesn't).
I had to edit the configure script because I was not able to get rid of
the -fno-exceptions otherwise.
I run into linker errors, though:
UniversalChardetTest.o:(.data.rel.ro._ZTI22nsUniversalChardetTest[typeinfo
for nsUniversalChardetTest]+0x0): undefined reference to `vtable for
__cxxabiv1::__si_class_type_info'
../src/base/libuniversalchardet_s.a(nsUniversalDetector.o):(.data.rel.ro._ZTI19nsUniversalDetector[typeinfo
for nsUniversalDetector]+0x0): undefined reference to `vtable for
__cxxabiv1::__class_type_info'
(Can't dive into that right now, can someone help me out here? I tried
to build firefox, and it worked fine with no RTTI and no exceptions)
I have done some googling, in the meantime. From what I have read,
current compilers should be able to produce no performance impact at
all, if no exceptions are actually thrown.
If they are thrown, though, which should happen under exceptional
circumstances, there is a non-neglectable overhead. This has to be
compared with the "non-exception" way of handling the same situation.
A nice presentation by Kevin Frei about the internals of exception
handling in MSVC8 is given here:
http://www.nwcpp.org/Meetings/2006/10.html
The Technical Report on C++ Performance has a section about how
exception handling can be done by compilers and what the effects are:
http://www.open-std.org/jtc1/sc22/wg21/docs/TR18015.pdf
It seems to me that the fear of performance degradation due to exception
usage is outdated (assuming the use of modern compilers), if they are
used the way there name implies: for exceptions.
Regards,
Roland
Neil
> On the other hand, I assume that due to inlining, the AddRef/Release
> code is multiplied, too.
My understanding is that AddRef/Release are virtual, so definitely not
inlined. However, due to multiclassing, there is some thunking overhead.
Not knowing anything about shared_ptr I do wonder where it stores the
refcount.
Roland Bock
> My understanding is that AddRef/Release are virtual, so definitely not
Oops, right
> Not knowing anything about shared_ptr I do wonder where it stores the
> refcount.
The shared_ptr is a template class which does the reference counting
itself. No reference counting is required by the objects. Here is some
documentation from boost.org (shared_ptr was originally developed by the
boost community):
http://www.boost.org/doc/libs/1_39_0/libs/smart_ptr/shared_ptr.htm
If you require local reference counting, the following shows how to do
that with shared_ptr:
http://anteru.net/2008/09/01/260/
Regards,
Roland
dbradley
> On Wednesday 2009-06-03 00:08 -0400, Boris Zbarsky wrote:
>
> I think it's not a good idea since it would change the vtable layout
> of frozen interfaces -- at least those that are derived from other
> interfaces, and perhaps all -- and also bloat some vtables.
Good point. Likely to hork XPTCall as well, since it uses indexes into
the vtable.
David Bradley
dbradley
> "Why? We turn exceptions and RTTI off, so you cannot use them in our
> codebase."
>
> With such a mindset, one wonders why anybody ever bothered to improve
> the C++ standard at all? Why not stick forever with what we had 10 years
> ago?
>
> I am aware that its not easy to change a big system like mozilla. I know
> that it cannot be done within a few weeks.
The tough thing is the wide range of platforms supported. It's no
small feat, though a little easier now with the build farm, to see
what impact a language feature has on the system on the various
platforms. Just because it looks ok on your particular compiler and OS
doesn't mean that it will on all the others.
I'd really like to be able to use the standard library collections and
algorithms. I really miss not having access to them. And it's a
impedance to people who casually work with the code who have a lot of
experience outside of Mozilla.
That said. There's been a lot of talk about JavaScript. Which makes me
wonder if such C++ efforts are really worth it if at some point
JavaScript transitions as the primary language. Not been keeping close
track, so not sure if that's still the direction or not.
David Bradley
Boris Zbarsky
>> I believe you just need to add --enable-coo-rtti and
>> --enable-cpp-exceptins to the ac_add_options in your mozconfig.
>
> I had to use --enable-cpp-rtti instead of --enable-coo-rtti
Good, good. Especially because "coo" was just a typo. ;)
> I have done some googling, in the meantime. From what I have read,
> current compilers should be able to produce no performance impact at
> all, if no exceptions are actually thrown.
That would be interesting, especially since the compiler does have to
generate code for handling the exceptions, I would think, and even if it
doesn't run it affects caching behavior....
> If they are thrown, though, which should happen under exceptional
> circumstances, there is a non-neglectable overhead. This has to be
> compared with the "non-exception" way of handling the same situation.
Yep. This has also been discussed to death, generally while discussing
enabling exceptions in Gecko.
> It seems to me that the fear of performance degradation due to exception
> usage is outdated (assuming the use of modern compilers), if they are
> used the way there name implies: for exceptions.
It's not a matter of fear; it's a matter of having data saying there is
degradation. If there is new data that shows there is not, great.
-Boris
Roland Bock
> degradation. If there is new data that shows there is not, great.
Up to now, it is just information gathered from 3rd party. The "real"
test still has to be performed. I am currently stuck due to the linkage
problem...
And even if most modern compilers should have little or no performance
degradation, there still might be supported platforms out there which
would suffer. I could not tell. But I hope that this discussion has
interested a few more people so that maybe new performance data could be
gathered soon.
As someone being used to the full power of the standard lib, its classes
and algorithms, I would certainly love to see Mozilla move in that
direction one fine day in the foreseeable future :-)
Regards,
Roland
Zack Weinberg
> Roland Bock wrote:
> >> I believe you just need to add --enable-coo-rtti and
> >> --enable-cpp-exceptins to the ac_add_options in your mozconfig.
> >
> > I had to use --enable-cpp-rtti instead of --enable-coo-rtti
>
> Good, good. Especially because "coo" was just a typo. ;)
>
> > I have done some googling, in the meantime. From what I have read,
> > current compilers should be able to produce no performance impact
> > at all, if no exceptions are actually thrown.
>
> That would be interesting, especially since the compiler does have to
> generate code for handling the exceptions, I would think, and even if
> it doesn't run it affects caching behavior....
People mean a bunch of different things by the statement quoted.
Having actually worked on this stuff, allow me to attempt to unpack it
a bit. You can set up a four-way classification of C++ functions and
what they do with exceptions:
1) Neither throws nor catches exceptions, and does not need to run
destructors if an exception is thrown through it.
The compilers (such as gcc 4.x) that claim "zero cost for the
no-exception path" really do generate exactly the same code for
functions in this class that they would with exceptions turned off.
2) Throws exceptions, but does not catch them itself, and does not need
to run destructors.
A throw statement compiles to a call to a function in the runtime that
doesn't return. On amd64 Linux, I get six instructions for a throw and
five for assert(false).
3) Needs to run destructors if an exception is thrown while its stack
frame is live; or contains an explicit catch clause which always
rethrows.
These are supposed to generate the same code with and without
exceptions, except that there's an extra block of code (typically
placed immediately after the function) that receives control if an
exception is thrown. In practice it's not so simple; with gcc 4.3 I see
code generation differences within the normal path as well -- it looks
like more registers are being saved, possibly to pass information from
the normal path to the catch block.
Compilers will get better; for my simple test function, gcc 4.4
eliminates almost all the differences in the normal path, the only
remaining one being that the stack frame is bigger for some reason. An
obvious further improvement would be to move the catch blocks to their
own text subsection.
4) Contains an explicit catch clause which may return control to the
normal path.
These are the same as 3, except that there's more code in the catch
block and it might jump back to the main body of the function.
---
The price you pay for (near-)identical code generation in classes 1 and
3 is a large table of data in a special ELF section when exceptions are
enabled. This table contains an entry for every function, no matter
what class it's in, and it makes the executable noticeably bigger.
(Note, however, that the amd64 ABI requires the table to be generated
even with exceptions off, so there's no binary-size impact there.)
Also, in this design a throw operation is quite expensive -- the runtime
has to look up the table entry for every function on the stack until it
finds something that will consume the exception, and then use the
information in the table entries to "unwind" to earlier stack frames;
contrast longjmp() which simply restores a bunch of registers.
The present Gecko codebase, if compiled with exceptions enabled,
consists entirely of functions in classes 1 and 3. If we were to
convert the codebase to make proper use of the feature, the number of
functions in class 1 would shrink, and the number of functions in
classes 2 and 3 would grow. The end point would probably have only a
few functions in class 4, but intermediate stages of conversion could
easily have many of them (anywhere we needed to bridge exceptions back
to nsresult codes).
A complete conversion would also get rid of a lot of
test-result-of-function operations that *are* affecting our cache usage
right now, but it would be very hard to measure which way was better
without doing all the work of the conversion first :-(
> > It seems to me that the fear of performance degradation due to
> > exception usage is outdated (assuming the use of modern compilers),
> > if they are used the way there name implies: for exceptions.
>
> It's not a matter of fear; it's a matter of having data saying there
> is degradation. If there is new data that shows there is not, great.
It's also worth mentioning that the last time *I* brought this up I was
told that a major part of the previously-measured performance impact
was not due to exceptions themselves, but to the extra runtime
relocation operations (thousands of them, apparently) that RTTI needs.
zw
Zack Weinberg
>
> 1) Neither throws nor catches exceptions, and does not need to run
> destructors if an exception is thrown through it.
>
> The compilers (such as gcc 4.x) that claim "zero cost for the
> no-exception path" really do generate exactly the same code for
> functions in this class that they would with exceptions turned off.
There's something I forgot to mention: exceptions limit the compiler's
ability to interchange other instructions with function calls, because
if a function throws an exception, side effects that were coded after
the call to that function must not have happened yet. That can
potentially cause code generation differences just from turning
exceptions on, for any function.
I doubt this is a problem for our code base, because in practice,
function calls are scheduling barriers anyway, and even if they
weren't, these kinds of optimizations tend to be lost in the noise on a
program of Mozilla's size. If we turned exceptions on and saw a >1%
performance degradation on a *specific function* that was a hot spot,
that would be worth staring at assembly dumps for, I think, but not
otherwise.
zw
Robert O'Callahan
> But I would be willing to contribute. I need to refactor some of the
> html parser code anyways in order to use it in my project. The main
> problem being the horrific amount of dependencies which sum up to some
> 27MB of include files and 30MB of library files just to parse html files
> into my own nsIContentSink. I need to boil that down a lot. I hope to
> end up with considerably less than 10MB in total :-)
You should know that we are planning to switch to a new parser which is
actually based on Henri Sivonen's standalone parser.
Rob
Roland Bock
Thanks for the info!
Is there some kind of project page, wiki, discussion board where this is
being discussed?
Regards,
Roland
Igor Bukanov
> As a brief summary, a minimal requirement for this is enabling exceptions
> (because the STL requires them) and rewriting a lot of code to deal with
> that. There are other issues, but that's a first hurdle.
For example, the image cache freely using STL facilities, see
http://mxr.mozilla.org/mozilla-central/source/modules/libpr0n/src/imgLoader.h#203
without any worries about exceptions. So what makes the image cache so
special that other parts of the code cannot follow the same path?
Igor
dbradley
XPConnect would most likely have to be adjusted to handle exceptions,
or we'd have follow a rule that no exceptions escape past XPCOM
methods. The latter might be difficult to enforce.
David
Sylvain Pasche
I'm curious about this too. Apparently the GNU and Visual C++ STL (so
mostly all what we care about if I'm correct) can be used without
exceptions. However your program will crash when an exception is hit, so
you have to take care.
For a real world example, Google Chrome is using the STL on the three
platforms without exceptions [1] [2].
Sylvain
[1]
http://www.google.com/codesearch/p?hl=en#h0RrPvyPu-c/build/common.gypi&q=fno-exceptions%20package:chromium.org&l=342
[2]
http://www.google.com/codesearch/p?hl=en#h0RrPvyPu-c/site_scons/site_tools/target_platform_windows.py&q=/EHsc%20package:chromium.org&l=156
Boris Zbarsky
> For example, the image cache freely using STL facilities, see
> without any worries about exceptions. So what makes the image cache so
> special that other parts of the code cannot follow the same path?
Good question. ;) The firm belief that the rules don't apply to it?
I _think_ that was an experiment to see whether it would work ok, but
you'd have to ask the people who reviewed that part of the code to make
sure.
-Boris
Igor Bukanov
> On 6/5/09 1:42 PM, Igor Bukanov wrote:
>> For example, the image cache freely using STL facilities
> I'm curious about this too. Apparently the GNU and Visual C++ STL (so mostly
> all what we care about if I'm correct) can be used without exceptions.
> However your program will crash when an exception is hit, so you have to
> take care.
AFAIK STL data structures and implementations of algorithms do not
throw itself. So you can use them with exceptions disabled. The caveat
is that without exceptions the new allocation returns null on
out-of-memory and STL code is not prepared to deal with that. So at
best the program will just crash, but thinks like arbitrary code
execution are also possible.
Igor
Jonas Sicking
The only part I wasn't able to follow was this:
> It's also worth mentioning that the last time *I* brought this up I was
> told that a major part of the previously-measured performance impact
> was not due to exceptions themselves, but to the extra runtime
> relocation operations (thousands of them, apparently) that RTTI needs.
What's a "relocation operation"? And when are they run?
Personally though, I think the hard part of using exceptions is ensuring
that the *massive* codebase that we have is exception safe. We're
currently pretty far from being exception safe.
/ Jonas
Zack Weinberg
> > It's also worth mentioning that the last time *I* brought this up I
> > was told that a major part of the previously-measured performance
> > impact was not due to exceptions themselves, but to the extra
> > runtime relocation operations (thousands of them, apparently) that
> > RTTI needs.
>
> What's a "relocation operation"? And when are they run?
Runtime relocations are done by the dynamic linker; each one is
essentially correcting a pointer whose ultimate target address was not
known at static link time. Some are done at program startup, some can
be delayed until the value is actually needed. Also, there are a
number of tricks that the compiler can use to reduce the number of
pointers that need correction in the first place.
Unfortunately, an it-made-sense-at-the-time decision by the committee
that specified the cross-platform C++ ABI that GCC uses, plus C++'s
lack of any real notion of modules, mean that typeinfo objects are
emitted many times by the compiler but are required to be unique in the
entire runtime image. This in turn means that none of the available
optimizations apply - the dynamic linker must relocate *every* pointer
to a typeinfo object at program startup time. And there can be
thousands of them; exceptions and dynamic_cast use them implicitly.
I heard some years ago that the openoffice.org team had some ideas for
how to improve this but I never heard any results and I'm pretty sure
nothing has changed.
> Personally though, I think the hard part of using exceptions is
> ensuring that the *massive* codebase that we have is exception safe.
> We're currently pretty far from being exception safe.
100% agreed here.
zw
Graydon Hoare
exns. In a big way. But I think it might be Hard.
On 09-06-04 10:21 AM, Zack Weinberg wrote:
> A complete conversion would also get rid of a lot of
> test-result-of-function operations that *are* affecting our cache usage
> right now, but it would be very hard to measure which way was better
> without doing all the work of the conversion first :-(
Yes. Personally I've never bought the line about our code suddenly
exploding in size due to the cases you enumerated. We'd gain some and
lose some. But we'd also gain better error handling facilities and
access to the STL. So ... from my perspective, huge win. Unfortunately
your enumeration is only valid on GCC or places that have grown out of
SJLJ-style exns. See below...
> It's also worth mentioning that the last time *I* brought this up I was
> told that a major part of the previously-measured performance impact
> was not due to exceptions themselves, but to the extra runtime
> relocation operations (thousands of them, apparently) that RTTI needs.
This is something that I'd worry about, yes. Along with the next point.
I did a little googling and it appears binutils at least can do a
forced-internal binding on typeinfos (--dynamic-list-cpp-typeinfo),
which lets you then prelink the whole thing. So ... possibly it can be
made fast. If you're building with binutils at least. No idea what MSVC
does here.
The next point though is serious.
GCC has moved away from SJLJ exns to these nice pretty 0-overhead (on
the normal execution path) DWARF2 exns that use a pile of static
metadata to take apart frames and work out which dtors to run when
unwinding. That's great. But we don't only compile with GCC. We also use
MSVC. MSVC and windows-ABI libraries of the sort we are linked against
all over the place have this old SEH thing for C code that looks a lot
like SJLJ, and they implement their C++ exns on top of it. I.e. there is
mandatory, dynamic prologue and epologue overhead in the *normal*
execution path or *all* frames with nonempty unwind semantics (say, with
local objects that have destructors).
Now ... I *think* you can do something a bit gnarly and build with mingw
not MSVC, and tell it to force wildly platform-inappropriate DWARF2
unwind info into the result. It just sticks it in a sectioin that the
windows loader doesn't care about, and gives you a runtime unwinder that
can cope. I think this gets you 0-overhead C++ exns in your *own* code,
but you still get the problem that SEH from some linkee might jump over
you without mentioning it. Not cool. So then you need to call
_set_se_translator at startup and tell it to map all SEH stuff into some
wrapper type you'll use the DWARF2 tables to handle. YES!
And if you do all that, and the stars align, and you're willing to give
up on using MSVC for windows builds ... I *think* you can get 0-overhead
win32 exns. Maybe. But since we're so hung up on MSVC PGO and generally
spiffy codegen (not to mention, er, perhaps /GS?) that we're not into
using GCC for windows builds. And also, to my knowledge very few people
have proven this sort of scheme to even work.
I would love to be wrong about all this. If someone knows how I'm wrong,
please speak up. It breaks my heart to be saying so.
-Graydon
Robert O'Callahan
> For example, the image cache freely using STL facilities, see
> http://mxr.mozilla.org/mozilla-central/source/modules/libpr0n/src/imgLoader.h#203
> without any worries about exceptions. So what makes the image cache so
> special that other parts of the code cannot follow the same path?
That code is just wrong.
Rob
Robert O'Callahan
> AFAIK STL data structures and implementations of algorithms do not
> throw itself. So you can use them with exceptions disabled. The caveat
> is that without exceptions the new allocation returns null on
> out-of-memory and STL code is not prepared to deal with that. So at
> best the program will just crash, but thinks like arbitrary code
> execution are also possible.
That's still OK if we make our memory allocators abort on OOM.
So, we should do that, like we've been talking about, then we can
reexamine the question of whether it's safe to use STL.
Rob
bent
> For a real world example, Google Chrome is using the STL on the three
> platforms without exceptions [1] [2].
You're correct for gcc but I believe that Chrome still builds with
exceptions enabled on Windows. That link you posted is to the scons
for windows but Chrome uses auto-gen'd visual studio solutions to
build on windows.
I toyed around with disabling exceptions on windows and did not have
very much luck, see http://ccollomb.free.fr/blog/?p=34 for some of the
issues that you'll likely run into.
-bent
bent
> then we can reexamine the question of whether it's safe to
> use STL.
Also we've discovered this great tidbit: libstdc++ is compiled in gcc
with a 4 byte wchar_t whereas mozilla is currently compiled with a 2
byte wchar_t so that wide strings (literals, in particular) are binary
compatible with our PRUnichar strings. This means that any STL classes/
functions that use wcslen or the like will corrupt memory. Things like
std::wstring simply cannot be used in mozilla until we fix that.
-bent
L. David Baron
There are also other system functions that we can't use because of
our use of -fshort-wchar.
When we use a gcc that supports u"" [1] (which seems to be gcc 4.4
[2]) we can stop using -fshort-wchar, since the only thing we need
it for is making NS_LITERAL_STRING (and variants) avoid runtime
string conversion overhead. (At least, that was our original reason
for needing it; hopefully we haven't come to depend on it in other
ways, and if so, it ought to be fixable.)
Then, once we stop supporting gcc prior to 4.4, we could start
depending on having a wchar_t that matches the system's.
-David
[1] http://en.wikipedia.org/wiki/C%2B%2B0x#New_string_literals
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2442.htm
[2] http://gcc.gnu.org/gcc-4.4/cxx0x_status.html
http://gcc.gnu.org/projects/cxx0x.html
--
L. David Baron http://dbaron.org/
Mozilla Corporation http://www.mozilla.com/
Zack Weinberg
> Also we've discovered this great tidbit: libstdc++ is compiled in gcc
> with a 4 byte wchar_t
In case anyone is wondering why 4-byte wchar_t is the default for
gcc/glibc/libstdc++, it's because one of the very few requirements C99
places on wchar_t is that it use a fixed width encoding. wchar_t is
*not* guaranteed to be Unicode, or even to be the same encoding across
locales, but both of those are user expectations -- well, in the
presence of surrogate pairs UTF-16 is not a fixed width encoding. So
a libc implementor can either refuse to support characters outside the
BMP, which is a nonstarter nowadays, or go to UTF-32 for wchar_t.
> whereas mozilla is currently compiled with a 2
> byte wchar_t so that wide strings (literals, in particular) are binary
> compatible with our PRUnichar strings. This means that any STL
> classes/ functions that use wcslen or the like will corrupt memory.
> Things like std::wstring simply cannot be used in mozilla until we
> fix that.
I talked to cjones about this a bit - basically, what you want to be
using is std::basic_string<char16_t>. See
<http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2249.html>.
zw
(footnote: Yes, there *are* systems that break the user expectation
that wchar_t is Unicode; if I remember correctly, Solaris 2.(<6) was
like that, and even nowadays there are EBCDIC-based mainframe OSes
that have a multitude of wide code pages to match their multitude of
one-byte code pages... fortunately nobody tries to run Mozilla anything
on them ;-)
dbradley
You wouldn't use std::wstring. You'd create your own typedef using
std::basic_string using PRUnichar. Probably safer all around anyway.
David
Karl Tomlinson
> The caveat is that without exceptions the new allocation returns
> null on out-of-memory
With gcc -fno-exceptions new does not return NULL, but throws an
exception (which gets handled by the default handler).
Some discussion on that is here:
http://gcc.gnu.org/ml/gcc/2007-09/msg00603.html
Are there some compilers where disabling extensions causes nothrow
new to be called?
> and STL code is not prepared to deal with that.
If the STL were compiled with exceptions, then I'd expect it to
call the throwing new.
Or does the exception-disabling also affect linking with some
compilers, linking in a different implementation of throwing new
that returns NULL?
If so, then I'd expect a different STL (also) to be linked?
Benjamin Smedberg
> With gcc -fno-exceptions new does not return NULL, but throws an
> exception (which gets handled by the default handler).
>
> Some discussion on that is here:
>
> http://gcc.gnu.org/ml/gcc/2007-09/msg00603.html
>
> Are there some compilers where disabling extensions causes nothrow
> new to be called?
Yes. MSVC calls a version of ::operator new which returns NULL on OOM when
you compile with exceptions disabled.
We rarely hit this on Linux anyway because the OS overcommits and we get
killed by the OOM killer, instead of hitting C++ exceptions, but there are
various bugs filed on us either overriding ::operator new or rewriting our
code to use the std::nothrow version.
>> and STL code is not prepared to deal with that.
>
> If the STL were compiled with exceptions, then I'd expect it to
> call the throwing new.
>
> Or does the exception-disabling also affect linking with some
> compilers, linking in a different implementation of throwing new
> that returns NULL?
That depends on which portions of the STL are dynamically linked and which
parts are inline templates that are compiled as part of our code. libstdc++
is actually a pretty small portion of the STL code.
libstdc++ is only compiled with exceptions enabled.
--BDS
Karl Tomlinson
> UniversalChardetTest.o:(.data.rel.ro._ZTI22nsUniversalChardetTest[typeinfo
> for nsUniversalChardetTest]+0x0): undefined reference to `vtable
> for __cxxabiv1::__si_class_type_info'
> ../src/base/libuniversalchardet_s.a(nsUniversalDetector.o):(.data.rel.ro._ZTI19nsUniversalDetector[typeinfo
> for nsUniversalDetector]+0x0): undefined reference to `vtable for
> __cxxabiv1::__class_type_info'
>
>
> (Can't dive into that right now, can someone help me out here? I
> tried to build firefox, and it worked fine with no RTTI and no
> exceptions)
Sounds like
http://www.daniweb.com/forums/post562963-2.html
though I haven't looked at the code and don't know why
--enable-cpp-rtti --enable-cpp-exceptions would change things here.
Karl Tomlinson
>> If the STL were compiled with exceptions, then I'd expect it to
>> call the throwing new.
>>
>> Or does the exception-disabling also affect linking with some
>> compilers, linking in a different implementation of throwing new
>> that returns NULL?
>
> That depends on which portions of the STL are dynamically linked and which
> parts are inline templates that are compiled as part of our code. libstdc++
> is actually a pretty small portion of the STL code.
Good point, thanks.
Compiling foreign code with compiler options for non-standard
behavior is going to cause problems if such code was never
expecting to be compiled with such behavior.
Roland Bock
OK, I think I found the solution:
http://www.fifi.org/doc/gcc-3.0-base/libstdc++/html_user/namespace____cxxabiv1.html
This indicates that the problem is RTTI related.
http://gcc.gnu.org/ml/gcc-help/2003-07/msg00097.html
The poster claims that it is sufficient to add -lsupc++
I tried that manually with the linker command that failed: Adding
-lsupc++ works fine. No problems in linking.
Now, what remains is: How do I tell .mozconfig to add -lsupc++?
Regards,
Roland
Karl Tomlinson
> The poster claims that it is sufficient to add -lsupc++
> Now, what remains is: How do I tell .mozconfig to add -lsupc++?
Perhaps a make command somewhere is using $(CC) instead of $(CXX).
What was being built when the link failed?
And what linker program was used in the command?
But a workaround, if all else fails, is
export LDFLAGS="-lsupc++"
ISTR having to force configure to run when adding environment
variables like this.
Roland Bock
The first was something in tests which I then compiled by hand. Then,
linking fails with mozilla-xremote-client.
After make distclean in the object folder, I tried
export LDFLAGS="-lsupc++"
make -f client.mk build
Now there is a new problem:
/usr/bin/ld: ../../staticlib/components/libhtmlpars.a(nsParser.o):
relocation R_X86_64_PC32 against `nsExpatDriver::nsExpatDriver()' can
not be used when making a shared object; recompile with -fPIC
So I tried again with export CXXFLAGS="-fPIC": It compiles and links! :-)
Well, here are the first results:
dist/bin is 0.2MB larger
I can't see any differences in firefox's speed
Are there specific pages or other tests for performance measurement?
Regards,
Roland
Karl Tomlinson
> export LDFLAGS="-lsupc++"
This can be done in the .mozconfig file.
> Are there specific pages or other tests for performance measurement?
This is the "standard" test framework
https://wiki.mozilla.org/StandaloneTalos
That includes a small pageset.
There is a larger pageset floating around, but I don't know
whether its available.
The small pageset is probably good enough to get some initial
figures.