The export keyword
Kevin S. Van Horn
not
yet implemented in *any* compiler. I have been told that it is a
difficult feature to implement. Can someone explain why, or point me
to a discussion on the difficulties of implementing export?
Kevin S. Van Horn
kevin....@ndsu.nodak.edu
[ Send an empty e-mail to c++-...@netlab.cs.rpi.edu for info ]
[ about comp.lang.c++.moderated. First time posters: do this! ]
Sebastian Kapfer
Van Horn writes...
| As far as I can tell, the export keyword is the only major C++ feature
| not
| yet implemented in *any* compiler.
Locate a post by Greg Comeau and read his signature ;)
| I have been told that it is a
| difficult feature to implement. Can someone explain why, or point me
| to a discussion on the difficulties of implementing export?
It is not thaaat difficult. It's just incompatible with the traditional
C compilation process. In C, each source file could be compiled
separately, generating an object file. It was the linker's task to put
these object files together and form binaries.
With the advent of C++, the situation is nearly unchanged: Any
information the compiler needs to generate object code from a source
file is found in the file itself or in header files which are included.
Except for exported templates.
When an exported template is used, the compiler can't see its definition
- and can generate no code. OTOH when the template definition is
compiled, the compiler can't predict what instantiations of the template
will be required to successfully link the program. That means, again, it
can't generate any code.
To solve this dilemma is the only difficulty with exported templates...
you have to pass information around between the different invocations of
the compiler.
Matthew Austern
> As far as I can tell, the export keyword is the only major C++ feature
> not
> yet implemented in *any* compiler. I have been told that it is a
> difficult feature to implement. Can someone explain why, or point me
> to a discussion on the difficulties of implementing export?
As far as I know, EDG is the only compiler vendor that has attempted
to implement export. They found it difficult, and they're very good.
I don't think John Spicer has published anything on techniques for
implementing export, so you're probably best off looking for his
Usenet articles and his postings to the Standardization Committee
reflector.
But a very brief and oversimplified summary: it's complicated because
it requires name lookup in a merged context. Nothing else in the
language is analogous.
Hubert Matthews
Currently both templates and inline functions have to be in header files
so that the compilation system can see the definition at the point of
use. A question: Could the techniques for implementing export be used for
inline too?
--
Hubert Matthews http://www.oxyware.com/
Software Consultant hub...@oxyware.com
Francis Glassborow
Kapfer <skapf...@yahoo.de> writes
>To solve this dilemma is the only difficulty with exported templates...
>you have to pass information around between the different invocations of
>the compiler.
That is rather like saying going from A to B isn't that difficult once
we have built a bridge over the deep, fast flowing river that separates
them.
Actually what makes this harder is ensuring that the solution will work
for template template parameters.
--
Francis Glassborow
Check out the ACCU Spring Conference 2002
4 Days, 4 tracks, 4+ languages, World class speakers
For details see: http://www.accu.org/events/public/accu0204.htm
Attila Feher
[SNIP]
> But a very brief and oversimplified summary: it's complicated because
> it requires name lookup in a merged context. Nothing else in the
> language is analogous.
This sounds impressive and very professional. What does it mean? :-)))
(Attila in SYI - Show Your Ignorance - mode again).
Attila
Ulrich Gerhardt
news:42aba677.02022...@posting.google.com...
> As far as I can tell, the export keyword is the only major C++ feature
> not
> yet implemented in *any* compiler. I have been told that it is a
> difficult feature to implement. Can someone explain why, or point me
> to a discussion on the difficulties of implementing export?
There has been a loooooooong thread about "Implementability of export"
in comp.std.c++.
I found it at
http://groups.google.de/groups?hl=de&threadm=52f2f9cd.0201100806.2999690c%40
posting.google.com&rnum=1&prev=/groups%3Fq%3Dimplementability%2Bof%2Bexport%
26hl%3Dde%26selm%3D52f2f9cd.0201100806.2999690c%2540posting.google.com%26rnu
m%3D1
(the thread is even longer than that url!)
Ciao, Uli.
Daveed Vandevoorde
> As far as I can tell, the export keyword is the only major C++ feature
> not yet implemented in *any* compiler.
In any supported end-user compiler, that is. Comeau Computing
has put up a beta version of their compiler (based on EDG
technology) on their web-site. However, the nature of the
on-line demo prevents you from trying "interesting" export
cases ;-)
> I have been told that it is a
> difficult feature to implement.
Certainly not an understatement.
> Can someone explain why, or point me
> to a discussion on the difficulties of implementing export?
I don't think there are public in-depth discussions anywhere, but
I pointed to some issues in the thread "Implementability of export"
in comp.std.c++ earlier this year (you can use groups.google.com to
find it).
I cannot really go into all the gory details, and even if I did I
doubt I could faithfully convey the horror of it all ;-) However,
here are some issues:
(a) Export relies on argument-dependent lookup (ADL; also sometimes
called "extended Koenig lookup", or even "Koenig lookup").
I think our (EDG's) compiler is still the only one that correctly
implements that _within_ a translation unit. However, with export
you need to be able to perform ADL _across_ translation units.
(b) Instantiating an exported template can trigger additional
cascaded instantiations in other translation units. Each of
those instantiations must be able to refer to entities that
were existing (or "sort of existing") when the template definition
was parsed. Hence, instead of having to deal with one symbol
table (which in C++ is already challenging enough), you need
(at least conceptually) to _simultaneously_ deal with an arbitrary
number of such tables.
(c) The ODR specifies a number of cases that result in undefined
behavior because it involves mismatches across translation
units that a traditional C/C++ linker cannot catch. However,
with export you do need to catch those mismatches in practice.
E.g.:
// File 1:
struct A {
typedef int I;
};
...
// File 2:
struct A {
typedef unsigned I; // ODR violation
};
...
There are also a lot of minor little annoyances that show up when
you do this (e.g., entities with internal linkage must be promoted to
having external linkage with some uses of export; a similar issue
occurs with unnamed namespaces; you'll probably need an upgrade to
the diagnostic reporting system so you can refer to multiple locations
across multiple files; and so forth).
When you first work on a design for "export", it may not seem entirely
crazy. However, once you work out the details, you're likely to come
to the conclusion that "export" really fits poorly with the other
language mechanisms.
Daveed
Daveed Vandevoorde
[...]
> | I have been told that it is a
> | difficult feature to implement. Can someone explain why, or point me
> | to a discussion on the difficulties of implementing export?
>
> It is not thaaat difficult.
I think there should be a law that prohibits someone from issuing such
a statement unless they have actually implemented it themselves ;-P
(I'm kidding.)
[...]
> To solve this dilemma is the only difficulty with exported templates...
> you have to pass information around between the different invocations of
> the compiler.
That's like saying "To solve hunger in the world, the only difficulty
is to make sure everyone gets enough adequate food." (Fortunately,
"export" is not as important a problem.)
My experience with implementing export is that it is thaaat difficult.
Daveed
John Levon
> (a) Export relies on argument-dependent lookup (ADL; also sometimes
> called "extended Koenig lookup", or even "Koenig lookup").
> I think our (EDG's) compiler is still the only one that correctly
> implements that _within_ a translation unit. However, with export
> you need to be able to perform ADL _across_ translation units.
I'm interested, what cases do other compilers get wrong ?
And what is the supposed difference between your definition of "Koenig
lookup" and "extended Koenig lookup" ?
thanks
john
Michiel Salters
> Matthew Austern wrote:
> [SNIP]
> > But a very brief and oversimplified summary: it's complicated because
> > it requires name lookup in a merged context. Nothing else in the
> > language is analogous.
>
> This sounds impressive and very professional. What does it mean? :-)))
> (Attila in SYI - Show Your Ignorance - mode again).
I guess it refers to the fact that the name lookup for a template
instantiation E<T> of exported template E can be done in both the
context of E and of the instantiation E<T>, depending on the exact
use of the name. 14.6.2 [Dependant Names] gives a situation in which
a name needs to be looked up in both scopes at the same time; I
guess this is the "merged scope"; it's not a standards term.
Regards,
--
Michiel Salters
Daveed Vandevoorde
> Currently both templates and inline functions have to be in header files
> so that the compilation system can see the definition at the point of
> use. A question: Could the techniques for implementing export be used for
> inline too?
Probably, but you need much more "front end" information for the
export model. Cross-TU inliners have been around for a while; the
ones I've had experience with used a lower level representation
that would not be able to represent templates. (On the other
hand, such inliners aren't _that_ common either, which is perhaps
an indication of the difficulties involved.)
Daveed
James Kanze
> [SNIP]
> > But a very brief and oversimplified summary: it's complicated
> > because it requires name lookup in a merged context. Nothing else
> > in the language is analogous.
> This sounds impressive and very professional. What does it mean?
> :-))) (Attila in SYI - Show Your Ignorance - mode again).
When instantiating a template, the compiler has to look up names both
at the point of instantiation and where the template was defined.
With non-exported templates, both points are in fact in the same
compilation unit, and the compiler has everything it needs handy.
With exported templates, the point of instantiation and the template
definition are in different compilation units, compiled at different
times; the compiler has to "see" the results of the compilation of a
different translation.
In theory, there is a simple solution: when compiling the template
instantiation, take a snapshot of the context, and when the compiler
encounters the point of instantiation, take a snapshot of the context,
then invoke the compiler at link time, passing it the two snapshots to
initialize the context. Like many theories, it tends to be a lot
harder in practice:
- The most obvious problem is what has to be recompiled when a
source file somewhere has been modified. This also has a simple
solution: always recompile all of the instantiations. Which is a
sure way of getting a reputation for unbearably long build times.
Short of that, it is incredibly difficult to be sure that
everything that needs recompiling actually is recompiled.
CFront did something along these lines for a much simpler model of
template instantiation. I rather liked the compiler, but it did
have a reputation for excessively long build times. And it was
well known that any time you got a strange error on execution, the
first step was to clear out all of the repositories of the
template instantiations, and rebuild (thus, recompiling all
instantiations again).
- Unless some sort of trimming is done, the snapshots are likely to
be quite big, say on the order of a megabyte. In the simplest
solution, there will be one per instantiation, per translation
unit that does the instantiation. That is, thousands, or tens of
thousands of snapshots. Requiring several tens of Gigabytes extra
disk space per application is going to cause problems even with
today's disks, and even supposing the space isn't a problem, it
takes a certain amount of time to write that much data, which
isn't going to help compile times either.
Of course, these are only the problems I see off hand, never having
tried to implement something like this. From experience, I can say
that the problems actually encountered are likely to be even more
difficult. The result is that this theoretically simple solution may
end up too complex to be reasonably implemented, and that some other
(theoretically more complex) solution has to be adopted.
--
James Kanze mailto:ka...@gabi-soft.de
Beratung in objektorientierer Datenverarbeitung --
-- Conseils en informatique orientée objet
Ziegelhüttenweg 17a, 60598 Frankfurt, Germany, Tél.: +49 (0)69 19 86 27
Sergey P. Derevyago
> > | I have been told that it is a
> > | difficult feature to implement. Can someone explain why, or point me
> > | to a discussion on the difficulties of implementing export?
> > It is not thaaat difficult.
> I think there should be a law that prohibits someone from issuing such
> a statement unless they have actually implemented it themselves ;-P
> (I'm kidding.)
if the pilot implementation doesn't exits (I'm _not_ kidding ;)
In particular, the C++ standard really suffers due to the absence of such law
:(( It's just like to release a program without any testing...
--
With all respect, Sergey. http://cpp3.virtualave.net/
mailto : ders at skeptik.net
Sebastian Kapfer
Vandevoorde writes...
| > To solve this dilemma is the only difficulty with exported templates...
| > you have to pass information around between the different invocations of
| > the compiler.
|
| That's like saying "To solve hunger in the world, the only difficulty
| is to make sure everyone gets enough adequate food." (Fortunately,
| "export" is not as important a problem.)
|
| My experience with implementing export is that it is thaaat difficult.
Implementing a compliant C++ compiler is not an easy task general. I am
aware of that and I'm sorry if my post hurt anybody's feelings :) I just
wanted to point out what makes "export" different from most other
constructs in the C++ spec, at least from my (limited) perspective.
Francis Glassborow
<non-ex...@iobox.com> writes
> IMHO there should be a law that prohibits someone from issuing a PL standard
>if the pilot implementation doesn't exits (I'm _not_ kidding ;)
> In particular, the C++ standard really suffers due to the absence of such law
>:(( It's just like to release a program without any testing...
I am not going to rehash history here but without 'export' there was a
high chance that there would have been no C++ standard. The introduction
of that keyword gave implementors a definite target to focus on with
regard to 'separate compilation of templates'.
You may be among those who wish that templates had been one of those
things left out of the C++ standard, however I do not think that many on
the Committees would have found that acceptable (and they were part of
the original remit)
--
Francis Glassborow
Check out the ACCU Spring Conference 2002
4 Days, 4 tracks, 4+ languages, World class speakers
For details see: http://www.accu.org/events/public/accu0204.htm
[ Send an empty e-mail to c++-...@netlab.cs.rpi.edu for info ]
P.J. Plauger
> | My experience with implementing export is that it is thaaat difficult.
>
> Implementing a compliant C++ compiler is not an easy task general. I am
> aware of that and I'm sorry if my post hurt anybody's feelings :) I just
> wanted to point out what makes "export" different from most other
> constructs in the C++ spec, at least from my (limited) perspective.
We've had a running joke with the EDG folks for the past year or so
about export. All you gotta do is add another keyword to the parser,
and a semantic function to call when you find the keyword in an
appropriate context. It's writing the latter that seems to have taken
some time...
Of course, now the joke is on us. All we had to do was restructure our
library headers into six or seven hundred separate files, then get the
flags right for the EDG front end...
P.J. Plauger
Dinkumware, Ltd.
http://www.dinkumware.com
Daveed Vandevoorde
> > (a) Export relies on argument-dependent lookup (ADL; also sometimes
> > called "extended Koenig lookup", or even "Koenig lookup").
> > I think our (EDG's) compiler is still the only one that correctly
> > implements that _within_ a translation unit. However, with export
> > you need to be able to perform ADL _across_ translation units.
>
> I'm interested, what cases do other compilers get wrong ?
Actually, I should have been more accurate: those other compilers
get the two-phase lookup wrong wrt. to ADL (i.e., I don't think
that they get ADL "per se" wrong). Here is the classic example:
template<class T> void f(T x) {
g(x); // Ordinary lookup here finds nothing
}
void g(int) {
}
int main() {
f(1); // Error: ADL here finds nothing for g either
}
The call of g(x) is cannot be resolved in standard C++ because ADL
doesn't associate a namespace with type "int". However, most
compilers accept this example without a diagnostic (ours does too
in its default not-strictly-standard-conforming mode).
> And what is the supposed difference between your definition of "Koenig
> lookup" and "extended Koenig lookup" ?
Koenig lookup is a term that was introduced to describe ADL for
operators; this was not particularly related to templates but
mostly motivated by the fact that explicit namespace qualification
is ugly for operators. When the SGI folks worked on resurrecting
the separation model at the Stockholm WG21/J16 meeting they proposed
"extended Koenig lookup" as an alternative to "intermediate points of
instantiations": this extended Koenig lookup to non-operators.
Daveed
Daveed Vandevoorde
[...]
> Actually what makes this harder is ensuring that the solution will work
> for template template parameters.
Well, that strictly speaking true, but template template parameters
didn't cause us any particular grief that I know of. I'm assuming
that you mention this because this causes templates (in addition to
template instantiations) to cross translation unit boundaries?
However, that's also the case with ordinary type parameters:
T::X can be a dependent (member) template.
Daveed
Nikolai Pretzell
> Koenig lookup is a term that was introduced to describe ADL for
> operators; this was not particularly related to templates but
> mostly motivated by the fact that explicit namespace qualification
> is ugly for operators.
Daveed, as you seem to know about the history and reasoning of Koenig
lookup,
could you (or anybody who knows) explain to me, why it was introduced?
Background of this question:
Of course, you need Koenig lookup, if you declare overloaded operators
within
a namespace. But why ever should an operator be declared within a
namespace?
Every overloaded operator is already completely defined by its (one or
two)
arguments. For instance
bool operator<( const std::string &, const std::string&);
would have had no need (at least no one, that comes appearant to me yet)
to be declared in namespace std itself. So why putting them there?
And for named functions preceding them with a namespace is readable well
Sergey P. Derevyago
> > IMHO there should be a law that prohibits someone from issuing a PL standard
> >if the pilot implementation doesn't exits (I'm _not_ kidding ;)
> > In particular, the C++ standard really suffers due to the absence of such law
> >:(( It's just like to release a program without any testing...
> I am not going to rehash history here but without 'export' there was a
> high chance that there would have been no C++ standard.
choose this standard.
But if you ask me "can a PL standard be issued without the pilot
implementation?", I'll say "no, it should not!".
> You may be among those who wish that templates had been one of those
> things left out of the C++ standard,
No, I'm not an idiot ;))
I'd like to see the C++ standard with circumspect definitions of templates,
namespaces, exception handling and the interoperation between them. BTW the
STL is quite good but it lacks the "move() nothrow" function (kind of widely
used "swap() nothrow") which is very useful for exception-safe containers.
But unfortunately there was no pilot implementations so the standard has too
many core language issues :((
--
With all respect, Sergey. http://cpp3.virtualave.net/
mailto : ders at skeptik.net
[ Send an empty e-mail to c++-...@netlab.cs.rpi.edu for info ]
Attila Feher
[SNIP]
> IMHO there should be a law that prohibits someone from issuing a PL standard
> if the pilot implementation doesn't exits (I'm _not_ kidding ;)
> In particular, the C++ standard really suffers due to the absence of such law
> :(( It's just like to release a program without any testing...
Actually I think you are not at all far away from the ways Mr.
Stroustrup (whose voice is rarely heard here - which makes me wonder
what does he do nowadays) was handling proposed new features. If you
read his publications, esp. Design and Evolution of C++ you will read
this as well. I mean that he has always tried to implement the feature,
test it and _only_ suggest the feature if it was "reasonably simple".
And he did this on a "rule bases". At least that is what I remember
from his book. And several examples of cases when he did so and how did
he manage to implement the thing.
Attila
James Kanze
news:<3c7e7eda$0$7870$724e...@reader2.ash.ops.us.uu.net>...
> > | My experience with implementing export is that it is thaaat difficult.
> > Implementing a compliant C++ compiler is not an easy task
> > general. I am aware of that and I'm sorry if my post hurt
> > anybody's feelings :) I just wanted to point out what makes
> > "export" different from most other constructs in the C++ spec, at
> > least from my (limited) perspective.
> We've had a running joke with the EDG folks for the past year or so
> about export. All you gotta do is add another keyword to the parser,
> and a semantic function to call when you find the keyword in an
> appropriate context. It's writing the latter that seems to have
> taken some time...
It's the famous "N'y a que" in French. Totally untranslatable, but
anytime you tell a boss or a customer it will take so much time or
money, they respond, "Why, n'y a que ..." -- roughly, all ya gotta do
is...
In this case, it is even more impressive that the EDG people got it
first, since they only make the front end. I would imagine that it
would be easier for someone with control over the entire process. (I
think Visual Age has something similar to export. Based on the
compiler and the linker sharing a common data base, which also
contains the entire sources, so a maximum of information is available
at each step.)
--
James Kanze mailto:ka...@gabi-soft.de
Beratung in objektorientierer Datenverarbeitung --
-- Conseils en informatique orientée objet
Ziegelhüttenweg 17a, 60598 Frankfurt, Germany, Tél.: +49 (0)69 19 86 27
[ Send an empty e-mail to c++-...@netlab.cs.rpi.edu for info ]
James Kanze
news:<DESlGcD7...@robinton.ntlworld.com>...
> In article <3C7E1C55...@iobox.com>, Sergey P. Derevyago
> <non-ex...@iobox.com> writes
> > IMHO there should be a law that prohibits someone from
> >issuing a PL standard if the pilot implementation doesn't exits
> >(I'm _not_ kidding ;) In particular, the C++ standard really
> >suffers due to the absence of such law :(( It's just like to
> >release a program without any testing...
> I am not going to rehash history here but without 'export' there was
> a high chance that there would have been no C++ standard. The
> introduction of that keyword gave implementors a definite target to
> focus on with regard to 'separate compilation of templates'.
You know as well as I that the issue is more complicated than that.
There were existing implementations of C++ which supported separate
compilation of templates. If it is impossible, or at least so
difficult, to implement some sort of separate compilation of templates
with the rules in the standard, we could have very well lived with the
name binding rules of CFront, or something similar.
> You may be among those who wish that templates had been one of those
> things left out of the C++ standard, however I do not think that
> many on the Committees would have found that acceptable (and they
> were part of the original remit)
So was garbage collection:-). (But I agree that C++ without templates
would be a lot poorer language. I don't think anyone would have
wanted to standardize <generic.h>:-).)
--
James Kanze mailto:ka...@gabi-soft.de
Beratung in objektorientierer Datenverarbeitung --
-- Conseils en informatique orientée objet
Ziegelhüttenweg 17a, 60598 Frankfurt, Germany, Tél.: +49 (0)69 19 86 27
[ Send an empty e-mail to c++-...@netlab.cs.rpi.edu for info ]
Francis Glassborow
<Attila...@lmf.ericsson.se> writes
>Actually I think you are not at all far away from the ways Mr.
>Stroustrup (whose voice is rarely heard here - which makes me wonder
>what does he do nowadays) was handling proposed new features.
If you must use an honorific it is Dr. These days he does what he has
always done, works for his employer. I also bet that he reads this, just
because he chooses not to post very much does not mean he is not around.
I am pretty damn sure he never tested a combination of exceptions,
namespaces, and templates with look-up in merged scope, though he was
responsible in some way for each of those.
--
Francis Glassborow
Check out the ACCU Spring Conference 2002
4 Days, 4 tracks, 4+ languages, World class speakers
For details see: http://www.accu.org/events/public/accu0204.htm
[ Send an empty e-mail to c++-...@netlab.cs.rpi.edu for info ]
Francis Glassborow
Kanze <ka...@gabi-soft.de> writes
>You know as well as I that the issue is more complicated than that.
>There were existing implementations of C++ which supported separate
>compilation of templates. If it is impossible, or at least so
>difficult, to implement some sort of separate compilation of templates
>with the rules in the standard, we could have very well lived with the
>name binding rules of CFront, or something similar.
I am sorry James, but I was there and I think I understand the issues
involved. The CFront solution was not acceptable for templates as they
had developed. And just about every implementor shared that opinion. If
it was easy to do, I think at least one would have wanted the status quo
as it was at that time because it would have given that company a
commercial advantage, but I am not aware of any implementor (on the
committee) that would accept separate compilation with the CFront model.
--
Francis Glassborow
Check out the ACCU Spring Conference 2002
4 Days, 4 tracks, 4+ languages, World class speakers
For details see: http://www.accu.org/events/public/accu0204.htm
[ Send an empty e-mail to c++-...@netlab.cs.rpi.edu for info ]
Francis Glassborow
<non-ex...@iobox.com> writes
> Well, if you ask me "this standard or no standard at all?", I will certainly
>choose this standard.
> But if you ask me "can a PL standard be issued without the pilot
>implementation?", I'll say "no, it should not!".
In that case you would have very few language standards. Who do you
think should write this pilot? Who should validate it? export was
introduced to the language because there was no compiler implementor
willing to even consider trying to implement the non-inclusion model for
template compilation. No, I tell a lie, I heard one mutter that he could
do it but the result would be so slow that no-one other than a compiler
validation tester would even consider using it and he considered it
dishonest to produce a supposedly conforming compiler that was unusable
in practice.
Would you have been happy had 'separate compilation of templates' been
removed from the standard? There were (are) a group for whom that would
be completely unacceptable.
--
Francis Glassborow
Check out the ACCU Spring Conference 2002
4 Days, 4 tracks, 4+ languages, World class speakers
For details see: http://www.accu.org/events/public/accu0204.htm
[ Send an empty e-mail to c++-...@netlab.cs.rpi.edu for info ]
Michiel Salters
news:<3c7e7eda$0$7870$724e...@reader2.ash.ops.us.uu.net>...
>
> > | My experience with implementing export is that it is thaaat difficult.
> >
> > Implementing a compliant C++ compiler is not an easy task general. I am
> > aware of that and I'm sorry if my post hurt anybody's feelings :) I just
> > wanted to point out what makes "export" different from most other
> > constructs in the C++ spec, at least from my (limited) perspective.
>
> We've had a running joke with the EDG folks for the past year or so
> about export. All you gotta do is add another keyword to the parser,
> and a semantic function to call when you find the keyword in an
> appropriate context. It's writing the latter that seems to have taken
> some time...
>
> Of course, now the joke is on us. All we had to do was restructure our
> library headers into six or seven hundred separate files, then get the
> flags right for the EDG front end...
>
> P.J. Plauger
> Dinkumware, Ltd.
> http://www.dinkumware.com
I can't see a reason why you need to split your headers into 600+
files with export, but apparently you do. Could you explain the
benefits ?
I can imagine it has something to do with eliminating unneeded
template instantitions, but I expect an export-capable linker
to eliminate unused template instantiations just like it
eliminates other unused entities
Regards,
--
Michiel Salters
P.J. Plauger
> > Of course, now the joke is on us. All we had to do was restructure our
> > library headers into six or seven hundred separate files, then get the
> > flags right for the EDG front end...
>
> I can't see a reason why you need to split your headers into 600+
> files with export, but apparently you do. Could you explain the
> benefits ?
>
> I can imagine it has something to do with eliminating unneeded
> template instantitions, but I expect an export-capable linker
> to eliminate unused template instantiations just like it
> eliminates other unused entities
I am far from an expert on these matters. Mostly, I've just volunteered
some Dinkumware effort for EDG to ensure that their product is useful
with a real live Standard C++ library. My understanding (and it could
be flawed) was that everything that goes into an expored file will
contribute to the executable image if any part of the file is needed.
(This is just like a conventional binary file in a library archive.
The only difference is that compiling and linking are deferred until
the template parameters, and the need, are known at prelink time.)
We began by identifying all possible candidates for export. Basically,
these are all the template functions, member functions of a template
class, and template member functions of a class that are poor
candidates for inlining. In our implementation, we ended up with
nearly 700 such candidates. Then we had to strip the definitions
out of the header and decorate each definition so that it can exist
as a separately compiled entity. In fact, we doctored our baseline
headers so they can be processed either into self-sufficient headers
(as usual) or into the stripped headers plus exported files.
Once we get some confidence that this extremely fragmented version
works solidly, the next task is to test the effect of clumping
expored files that are likely to be loaded together anyway. I expect
judicious clumping to help, but it's way too early to say by how
much. I've already been disappointed with the small reduction in
header size from stripping out exported functions. The total size
of our C++ header files drops by maybe 15 per cent, and the total
size of our C/C++ header files combined (a more meaningful measure)
drops by about 10 per cent. Counting the exported files, the total
library size increases by over 20 per cent. This does not bode well
for improvements in overall compile/link time, but we still have
little actual timing data from which to project.
P.J. Plauger
Dinkumware, Ltd.
http://www.dinkumware.com
Herb Sutter
<francis.g...@ntlworld.com> wrote:
>Would you have been happy had 'separate compilation of templates' been
>removed from the standard? There were (are) a group for whom that would
>be completely unacceptable.
I know you're only reporting others' viewpoints, but since you mention it:
But just now "unacceptable" can it be when here we are, four years after
C++98 was ratified, and nobody has yet shipped an export-capable compiler
(although one is very close)? Obviously we've lived without it for this
long, and C++ is still the dominant commercial programming language, and
people are still writing and shipping untold gazillions of lines of
production code written in C++, and we don't seem to be that much the worse
for wear.
I can accept that some may consider it "unacceptable" to not have the
facility in the standard, but that's less meaningful to me when that's still
the only place the facility exists, and it's not available in the real
world. I realize that's changing, finally, but the point is valid
nonetheless. I'm perfectly happy to accept that a long-absent feature is
desirable, but I won't easily accept that a long-absent feature should be
branded 'essential' and its absence "unacceptable" when: a) it's not
well-understood by most (I think I know many of the ramifications but I know
I don't know them all); b) it adds further complication in areas that are
already complex (if that weren't true it wouldn't have been such a challenge
for the world's smartest compiler writers); and c) I haven't yet heard
real-world programmers clamoring for it widely (the few that I have heard
did not correctly understand what "export" as specified in C++98 really
does).
On the other hand, those last three things are mostly not true of typedef
templates and strict typeof, for example, and so I'm much more willing to
consider those two things 'essential' and their absence "unacceptable." :-)
Herb
---
Herb Sutter (www.gotw.ca)
Secretary, ISO WG21/ANSI J16 (C++) standards committee (www.gotw.ca/iso)
Contributing Editor, C/C++ Users Journal (www.cuj.com)
Check out "THE C++ Seminar" - Boston, March 2002 (www.gotw.ca/cpp_seminar)
Daveed Vandevoorde
[...]
> I can't see a reason why you need to split your headers into 600+
> files with export, but apparently you do. Could you explain the
> benefits ?
There may be reasons beyond what I'm going to say here.
However, remember that the context at the point of the
template definition has to be carried to the point of
instantiation. The more declarations you have in a
translation unit (TU), the more context combinations
that amounts to. So from a build-resources perspective,
it is indeed probably advantageous to keep exported
templates in their own minimal TUs.
Daveed
Daveed Vandevoorde
> Daveed Vandevoorde wrote:
> > > | I have been told that it is a
> > > | difficult feature to implement. Can someone explain why, or point me
> > > | to a discussion on the difficulties of implementing export?
> > > It is not thaaat difficult.
> > I think there should be a law that prohibits someone from issuing such
> > a statement unless they have actually implemented it themselves ;-P
> > (I'm kidding.)
> IMHO there should be a law that prohibits someone from issuing a PL
standard
> if the pilot implementation doesn't exits (I'm _not_ kidding ;)
> In particular, the C++ standard really suffers due to the absence of such
law
> :(( It's just like to release a program without any testing...
I have a lot of sympathy for that point of view. From a realistic
point of view, I think the committee was right to innovate in a
number of matters though; if it hadn't, C++ templates would not
be nearly as powerful as they are today. However, in the case of
"export" I believe the committee made a mistake (I must confess
that at the time I was in favor of that mistake): export does not
add expressive power to the language and was introduced as a
fundamentally new concept very late in the standardization process
(July 96).
Daveed
Daveed Vandevoorde
> high chance that there would have been no C++ standard.
I'm assuming you referring to political arguments here?
The language standard is certainly technically sound without
"export".
> The introduction
> of that keyword gave implementors a definite target to focus on with
> regard to 'separate compilation of templates'.
If that was a goal (I don't think it was though), I think it failed
quite miserably. As far as I know, no major C++ front end (besides
EDG) had even _started_ working on "export" in April 2001 (Copenhagen
WG21/J16 meeting, when I talked to a number of implementors about
that). That's almost five years after "export" was added, which
hardly speaks for "focusing implementors" ;-)
> You may be among those who wish that templates had been one of those
> things left out of the C++ standard, however I do not think that many on
> the Committees would have found that acceptable (and they were part of
> the original remit)
I love the concept of C++ templates, but I'm not sure "export"
added anything technically useful to that concept (the future
will tell, I suppose).
Daveed
Daveed Vandevoorde
> think should write this pilot? Who should validate it? export was
> introduced to the language because there was no compiler implementor
> willing to even consider trying to implement the non-inclusion model for
> template compilation.
At the time we couldn't really talk about "the" non-inclusion model.
There was no description in the standard of what that meant.
Several compiler implementors explicitly expressed willingness to
work on a separation model, but most of them also said it would
be better to ship the first C++ standard without it since it would
take some time (at least) to come with up something reasonable.
(Although I did not agree with them at the time, in hindsight
they've been proven right IMO.)
> Would you have been happy had 'separate compilation of templates' been
> removed from the standard? There were (are) a group for whom that would
> be completely unacceptable.
That's true of many elements of C++, of course :-) We've lived
quite happily without "export" for a while, though.
Daveed
Jerry Coffin
hsu...@acm.org says...
[ ... ]
> But just now "unacceptable" can it be when here we are, four years after
> C++98 was ratified, and nobody has yet shipped an export-capable compiler
> (although one is very close)? Obviously we've lived without it for this
> long, and C++ is still the dominant commercial programming language, and
> people are still writing and shipping untold gazillions of lines of
> production code written in C++, and we don't seem to be that much the worse
> for wear.
While I generally agree, I'm forced to take the devil's advocate
position for a moment: how many library vendors are there who might
be happy to ship templated versions of their libraries, but ONLY if
they can do so without basically giving away their source code?
Offhand, I don't know the number, but I suspect a library vendor has
to at least consider this, and might easily forego using templates as
well as possible in favor of using them in a way that still allowed
them to ship at least the majority of the code in pre-compiled format
for the sake of protection.
Apparently some library vendors (e.g. Dinkumware) aren't particularly
concerned over this, but I wonder whether some other libraries
(especially ones that embody more or less proprietary information)
might make better use of templates when/if export becomes widely
available.
In fairness, I suspect the lag between definition and implementation
of export may (have) to some degree rendered this a moot point:
people who wouldn't normally have released source code to their
libraries had no choice, and after 5+ years of distributing the
source code, they're going to have a hard time unringing the bell, so
to speak.
--
Later,
Jerry.
The Universe is a figment of its own imagination.
Pete Becker
>
> I can't see a reason why you need to split your headers into 600+
> files with export, but apparently you do. Could you explain the
> benefits ?
>
In addition to the more technical reasons in the other answers, the
basic rule of library packaging is one function per source file. That
translates to one function per object file, so you don't have to
postulate magic linkers to minimize application size.
--
Pete Becker
Dinkumware, Ltd. (http://www.dinkumware.com)
Daveed Vandevoorde
> Hi,
>
> > Koenig lookup is a term that was introduced to describe ADL for
> > operators; this was not particularly related to templates but
> > mostly motivated by the fact that explicit namespace qualification
> > is ugly for operators.
>
> Daveed, as you seem to know about the history and reasoning of Koenig
> lookup,
> could you (or anybody who knows) explain to me, why it was introduced?
Note that I wasn't yet attending WG21/J16 meetings when the
original Koenig lookup was introduced. I remember an Overload
article by Sean Corfield (secretary of the committee at the
time, I believe) which explained exactly what I said above:
without something like Koenig lookup you'd either have to
litter you code with "using" or you have to write things
like "N::operator"(a, b)" instead of "a+b".
> Background of this question:
>
> Of course, you need Koenig lookup, if you declare overloaded
> operators within a namespace. But why ever should an operator
> be declared within a namespace?
> Every overloaded operator is already completely defined by its
> (one or two) arguments. For instance
>
> bool operator<( const std::string &, const std::string&);
>
> would have had no need (at least no one, that comes appearant to
> me yet) to be declared in namespace std itself. So why putting
> them there?
I'm not sure that argument was considered at the time. However,
I think your reasoning is somewhat brittle. Two issues that
come up off hand:
(a) With Koenig lookup the set of operators is limited to those
in namespace that are directly associated with the argument
types. If all the operators are in global scope, however,
all of them will have to participate in overload resolution
and seamingly unrelated operators (more likely: operator
templates) can trigger ambiguity errors. It also can have
terrible consequences for overload resolution performance.
(A C++ compiler I worked on in the past was having a hard
time compiling a bunch of debug code of the form "cout << x"
in a timely manner. After some investigation it turned
out the overload sets for "operator <<" contained about
5000 operator functions generated by a CORBA related tool!)
(b) At the time Koenig lookup was introduced, visible friend name
injection was still part of the language. Consider the following
example:
namespace N {
template<typename T> class X {
friend bool operator==(X<T> const&, X<T> const&) {
// This is N::operator==
// ...
}
};
}
This is sometimes referred to as the "Barton-Nackman trick"
(after the authors who pointed out the advantages of that
technique in a C++ without function template overloading).
Note that operator== above describes a family of ordinary
(nontemplate) operator functions. Nowadays you'd probably
just add an overloaded function template:
template<typename T> bool operator==(N::X<T> const&,
N::X<T> const&);
However, when Koenig lookup was introduced you couldn't
overload function templates. (Also, the nontemplate version
allows implicit conversions that the template version
doesn't, if you like that sort of thing ;-). This technique
still mostly works in the modern language, but argument-
dependent lookup had to be modified to support it.
(Don't construe this as evidence that I particularly like ADL. :^/)
I'll re-plug the book I'm writing with Nico Josuttis: it explains
all the stuff above in detail ("C++ Templates", later this year
from Addison-Wesley).
> And for named functions preceding them with a namespace is readable well
Yes, but that's not always possible in the context of templates.
That's why Koenig lookup was extended when "export" was
introduced. (See the thread "Implementability of export"
in comp.std.c++ earlier this year.)
Daveed
Daveed Vandevoorde
[...]
> In this case, it is even more impressive that the EDG people got it
> first, since they only make the front end. I would imagine that it
> would be easier for someone with control over the entire process.
I'm not sure it would give a significant advantage, actually.
(It's hard to know without actually trying to implement it in such
an environment. However, at no point during our implementation
did I think "gee, I wish I could rely on some integrated
environment here").
However, the great irony here is that EDG engineers were among
the strongest opponents of "export" and now they are behind the
first implementation of the beast. I am now of the opinion that
at the time "export" was voted in, John Spicer (of EDG) had by
far the deepest insight into the implications of "export".
However, I'm afraid reality was even worse than those insights.
Daveed
Pete Becker
>
> I am not going to rehash history here but without 'export' there was a
> high chance that there would have been no C++ standard.
The same is true of auto_ptr, and that, too, turned out to be a mess.
That's the problem with political demands that aren't accompanied by
technical solutions.
--
Pete Becker
Dinkumware, Ltd. (http://www.dinkumware.com)
[ Send an empty e-mail to c++-...@netlab.cs.rpi.edu for info ]
Pete Becker
>
> While I generally agree, I'm forced to take the devil's advocate
> position for a moment: how many library vendors are there who might
> be happy to ship templated versions of their libraries, but ONLY if
> they can do so without basically giving away their source code?
>
That is one of the arguments made in support of export. But there
doesn't seem to be a big demand for export from library vendors, so it
doesn't seem like it matters much to them.
--
Pete Becker
Dinkumware, Ltd. (http://www.dinkumware.com)
[ Send an empty e-mail to c++-...@netlab.cs.rpi.edu for info ]
P.J. Plauger
news:MPG.16eadfe91...@news.direcpc.com...
> While I generally agree, I'm forced to take the devil's advocate
> position for a moment: how many library vendors are there who might
> be happy to ship templated versions of their libraries, but ONLY if
> they can do so without basically giving away their source code?
Implicit in this rhetorical question are two assumptions:
1) Exported templates give you a way to protect template function
definitions from prying eyes.
2) Headers in their usual form do not.
Both assumptions are false, or at least not necessarily true.
> Offhand, I don't know the number, but I suspect a library vendor has
> to at least consider this, and might easily forego using templates as
> well as possible in favor of using them in a way that still allowed
> them to ship at least the majority of the code in pre-compiled format
> for the sake of protection.
>
> Apparently some library vendors (e.g. Dinkumware) aren't particularly
> concerned over this, but I wonder whether some other libraries
> (especially ones that embody more or less proprietary information)
> might make better use of templates when/if export becomes widely
> available.
I've personally had good success relying on copyright protection --
and the growing honesty of my customer base. Dinkumware has of course
also done so for its 6+ years of existence. I've heard rumors that
our erstwhile competitors years ago tried to demand code shrouding,
encryption, or protection of their headers by their potential OEMs.
But once that requirement threatened to cost serious sales, they
backed off.
> In fairness, I suspect the lag between definition and implementation
> of export may (have) to some degree rendered this a moot point:
> people who wouldn't normally have released source code to their
> libraries had no choice, and after 5+ years of distributing the
> source code, they're going to have a hard time unringing the bell, so
> to speak.
Perhaps. We'll soon begin to see if libraries start to appear that
take advantage of the putative protection offered by exported templates.
My bet is that market forces will keep such critters to a minimum.
P.J. Plauger
Dinkumware, Ltd.
http://www.dinkumware.com
Jerry Coffin
p...@dinkumware.com says...
[ ... ]
> Implicit in this rhetorical question are two assumptions:
>
> 1) Exported templates give you a way to protect template function
> definitions from prying eyes.
>
> 2) Headers in their usual form do not.
>
> Both assumptions are false, or at least not necessarily true.
In reality, I made one implicit assumption: that some people
_believe_ that exported templates would be likely to give them
_better_ protection than current distribution methods.
People base their decision on beliefs, whether those beliefs are
right or wrong. It's also mostly relative -- a library can be
reverse engineered if necessary, but at some point independent
development is easier, and that's generally sufficient.
> I've personally had good success relying on copyright protection --
> and the growing honesty of my customer base. Dinkumware has of course
> also done so for its 6+ years of existence. I've heard rumors that
> our erstwhile competitors years ago tried to demand code shrouding,
> encryption, or protection of their headers by their potential OEMs.
> But once that requirement threatened to cost serious sales, they
> backed off.
That's not surprising in markets where there's a reasonable amount of
competition, but the smaller the niche, the less likely one is to
have competitors that force the issue.
I also suspect that shrouding headers and such is perceived more
negatively by many people than shipping libraries (without source
code) simply because the latter has been common practice for a long
time.
From a more practical viewpoint: error messages about problems in
templates are often difficult to read at best, and shrouded headers
would seem likely to exacerbate the situation. Unless something was
done about this, I'd probably be MUCH happier with exported templates
than with a shrouded header.
> Perhaps. We'll soon begin to see if libraries start to appear that
> take advantage of the putative protection offered by exported templates.
> My bet is that market forces will keep such critters to a minimum.
I don't know how soon we could expect that to happen in any case: EDG
is close to delivering a compiler that supports export, but that's
still only a relatively small part of the market. The real test
doesn't arrive until Microsoft, Borland, Sun, HP, etc., support it.
I _hope_ you're right, and the primary thrust of export is to reduce
build times, but greed being as common as it is, I suspect we'll see
at least some people try to turn it more toward their own profit than
the benefit of their customers. In the long run, that's rarely a
winning strategy, but human nature being what it is, that seems
unlikely to stop people from trying.
--
Later,
Jerry.
The Universe is a figment of its own imagination.
[ Send an empty e-mail to c++-...@netlab.cs.rpi.edu for info ]
Francis Glassborow
<jco...@taeus.com> writes
>In fairness, I suspect the lag between definition and implementation
>of export may (have) to some degree rendered this a moot point:
>people who wouldn't normally have released source code to their
>libraries had no choice, and after 5+ years of distributing the
>source code, they're going to have a hard time unringing the bell, so
>to speak.
However there is no requirement that an implmentation of export will
work with semi-compiled templates, whatever they might be.
--
Francis Glassborow
Check out the ACCU Spring Conference 2002
4 Days, 4 tracks, 4+ languages, World class speakers
For details see: http://www.accu.org/events/public/accu0204.htm
[ Send an empty e-mail to c++-...@netlab.cs.rpi.edu for info ]
Francis Glassborow
writes
>Francis Glassborow wrote:
>>
>> I am not going to rehash history here but without 'export' there was a
>> high chance that there would have been no C++ standard.
>
>The same is true of auto_ptr, and that, too, turned out to be a mess.
>That's the problem with political demands that aren't accompanied by
>technical solutions.
That is simply not true (that it was required to ensure a standard) Its
removal, or its amendment along the lines that happened was required for
the UK to change its vote from 'No' to 'Yes' but the standards process
does not require every NB to vote 'yes'. The situation with regards to
'separate compilation of templates' was quite different with no
consensus, or even substantial majority possible. I cannot remember if
you were at the meeting but I do know that WG21 met (in separate
session) more often that week than at any other meeting.
--
Francis Glassborow
Check out the ACCU Spring Conference 2002
4 Days, 4 tracks, 4+ languages, World class speakers
For details see: http://www.accu.org/events/public/accu0204.htm
[ Send an empty e-mail to c++-...@netlab.cs.rpi.edu for info ]
Pete Becker
>
> In article <3C81410C...@acm.org>, Pete Becker <peteb...@acm.org>
> writes
> >Francis Glassborow wrote:
> >>
> >> I am not going to rehash history here but without 'export' there was a
> >> high chance that there would have been no C++ standard.
> >
> >The same is true of auto_ptr, and that, too, turned out to be a mess.
> >That's the problem with political demands that aren't accompanied by
> >technical solutions.
>
> That is simply not true (that it was required to ensure a standard) Its
> removal, or its amendment along the lines that happened was required for
> the UK to change its vote from 'No' to 'Yes' but the standards process
> does not require every NB to vote 'yes'.
>
The goal of achieving concensus means that standards committees must
take seriously any threat to vote no on the standard, and that's what we
did. The threat of a vote against the standard is what got auto_ptr in.
--
Pete Becker
Dinkumware, Ltd. (http://www.dinkumware.com)
[ Send an empty e-mail to c++-...@netlab.cs.rpi.edu for info ]
Ed L Cashin
> Michiel...@cmg.nl (Michiel Salters) wrote:
> [...]
> > I can't see a reason why you need to split your headers into 600+
> > files with export, but apparently you do. Could you explain the
> > benefits ?
>
> There may be reasons beyond what I'm going to say here.
> However, remember that the context at the point of the
> template definition has to be carried to the point of
> instantiation. The more declarations you have in a
> translation unit (TU), the more context combinations
> that amounts to. So from a build-resources perspective,
> it is indeed probably advantageous to keep exported
> templates in their own minimal TUs.
Without export, with regard to the template functions that are
included into each client's translation unit, you must give up the
advantages of separate compilation and also the advantages of shared
libraries.
But is it true that *with* export the library builder would have to
create huge libraries that contain hundreds of definitions, one for
each possible sequence of template parameters?
In that case, the library creator cannot anticipate the user-defined
types in the client's translation unit, so you lose one of the nicest
things about C++: that user-defined types can be used in most of the
ways that primitives can be used.
--
--Ed L Cashin | PGP public key:
eca...@uga.edu | http://noserose.net/e/pgp/
Nikolai Pretzell
thank you for your answer I learned a lot.
Nevertheless ...
> > But why ever should an operator
> > be declared within a namespace?
> > Every overloaded operator is already completely defined by its
> > (one or two) arguments.
> I'm not sure that argument was considered at the time. However,
> I think your reasoning is somewhat brittle. Two issues that
> come up off hand:
> (a) With Koenig lookup the set of operators is limited to those
> in namespace that are directly associated with the argument
> types. If all the operators are in global scope, however,
> all of them will have to participate in overload resolution
> and seamingly unrelated operators (more likely: operator
> templates) can trigger ambiguity errors.
May be .... most ambiguity errors would appear within a class hierarchy,
and those are declared within one namespace most times. But that might be
a mistaken assumptions of mine.
> It also can have
> terrible consequences for overload resolution performance.
> (A C++ compiler I worked on in the past was having a hard
> time compiling a bunch of debug code of the form "cout << x"
> in a timely manner. After some investigation it turned
> out the overload sets for "operator <<" contained about
> 5000 operator functions generated by a CORBA related tool!)
Though this doesn't convince me theoretical, it convinces me practical.
;-)
> (b) At the time Koenig lookup was introduced, visible friend name
> injection was still part of the language. Consider the following
> example:
> namespace N {
> template<typename T> class X {
> friend bool operator==(X<T> const&, X<T> const&) {
> // This is N::operator==
> // ...
> }
> };
> }
May be, I'm missing the point, but why not as well:
[ ...]
friend bool ::operator==(X<T> const&, X<T> const&) {}
[ ...]
?
> (Don't construe this as evidence that I particularly like ADL. :^/)
I won't. :-)
> I'll re-plug the book I'm writing with Nico Josuttis: it explains
> all the stuff above in detail ("C++ Templates", later this year
> from Addison-Wesley).
I'll look out for it.
> > And for named functions preceding them with a namespace is readable well
> Yes, but that's not always possible in the context of templates.
Mhm, yes, there may be quite some cases I didn't think about, yet. On the
other hand,
it is difficult enough to understand exactly what happens in more
complicated
template code. It seems to me, Koenig Lookup increases the
unfathomability of such
code. But probably that's a question of getting accustomed to it.
( I'm just enjoying, but not yet finished with, A. Alexandrescus Modern
C++ Design ;-) )
> That's why Koenig lookup was extended when "export" was
> introduced. (See the thread "Implementability of export"
> in comp.std.c++ earlier this year.)
Ok but, if 'export' reveals itself to be not implementable in practice,
(see the thread, you mentioned), that reasoning would become weaker. ;-)
Regards,
Nikolai
--
Nikolai Pretzell
Software Engineer Development Tools
Star Office Gmbh, Hamburg
Michiel Salters
news:<86k7stx...@cs.uga.edu>...
> Without export, with regard to the template functions that are
> included into each client's translation unit, you must give up the
> advantages of separate compilation and also the advantages of shared
> libraries.
>
> But is it true that *with* export the library builder would have to
> create huge libraries that contain hundreds of definitions, one for
> each possible sequence of template parameters?
No. The library builder will create libraries which contain the
parsed template, with its context. It isn't fully compiled.
Compilation still happens when a template is instantiated.
Regards,
--
Michiel Salters
Francis Glassborow
writes
>But is it true that *with* export the library builder would have to
>create huge libraries that contain hundreds of definitions, one for
>each possible sequence of template parameters?
NO. As there are an infinite number of types 'derived from built-in
types (arrays - each size being its own type - multi dimension arrays,
pointers, mixtures of pointer to pointer... with const and volatile) it
is impossible. That is not what 'export' is about.
--
Francis Glassborow
Check out the ACCU Spring Conference 2002
4 Days, 4 tracks, 4+ languages, World class speakers
For details see: http://www.accu.org/events/public/accu0204.htm
[ Send an empty e-mail to c++-...@netlab.cs.rpi.edu for info ]
Francis Glassborow
<nik...@sun.com> writes
>May be, I'm missing the point, but why not as well:
>
>[ ...]
> friend bool ::operator==(X<T> const&, X<T> const&) {}
>[ ...]
>?
Because that throws everything into the global namespace resulting in
potentially enormous overload sets. However note that operators called
via operator symbols have always had extended lookup rules (class scope
+ global scope) Koenig Lookup largely generalises this rule (in the
context of namespaces) to apply to all functions.
Do not forget that within a class scope member function calling syntax
is indistinguishable from that for free functions.
--
Francis Glassborow
Check out the ACCU Spring Conference 2002
4 Days, 4 tracks, 4+ languages, World class speakers
For details see: http://www.accu.org/events/public/accu0204.htm
[ Send an empty e-mail to c++-...@netlab.cs.rpi.edu for info ]
James Kanze
> ka...@gabi-soft.de (James Kanze) wrote:
> [...]
> > In this case, it is even more impressive that the EDG people got
> > it first, since they only make the front end. I would imagine
> > that it would be easier for someone with control over the entire
> > process.
> I'm not sure it would give a significant advantage, actually. (It's
> hard to know without actually trying to implement it in such an
> environment. However, at no point during our implementation did I
> think "gee, I wish I could rely on some integrated environment
> here").
A mapmaker trying to choose an appropriate projection doesn't think,
"Gee, I wish the world were flat", but his job would still be a lot
easier if it was.
The one implementation today which comes closest to export, Visual
Age, uses a global data base. Of course, I don't know if this makes
that particular aspect of their compiler easier or not, but a priori,
the more information you can maintain, and the more freedom you have
in choosing when to do something, the more chances you have to find an
optimal solution.
> However, the great irony here is that EDG engineers were among the
> strongest opponents of "export" and now they are behind the first
> implementation of the beast. I am now of the opinion that at the
> time "export" was voted in, John Spicer (of EDG) had by far the
> deepest insight into the implications of "export". However, I'm
> afraid reality was even worse than those insights.
I think that part of the problem is that no one (or very few) at the
time even understood all of the implications of the new template name
lookup. I know I didn't -- if I had realized that almost everything
about templates was new and untested, I suspect that there would have
been a no vote from one of the national bodies. As it is, I was not
following templates too closely at the time (for lack of time), and I
thought that the argument was between Borland model and CFront model,
and not something radically new and untested.
--
James Kanze mailto:ka...@gabi-soft.de
Beratung in objektorientierer Datenverarbeitung --
-- Conseils en informatique oriente objet
Ziegelhttenweg 17a, 60598 Frankfurt, Germany, Tl.: +49 (0)69 19 86 27
James Kanze
news:<6fq08ukrf16kc0fb5...@4ax.com>...
> On 1 Mar 2002 12:58:29 -0500, Francis Glassborow
> <francis.g...@ntlworld.com> wrote:
> >Would you have been happy had 'separate compilation of templates'
> >been removed from the standard? There were (are) a group for whom
> >that would be completely unacceptable.
> I know you're only reporting others' viewpoints, but since you
> mention it:
> But just now "unacceptable" can it be when here we are, four years
> after C++98 was ratified, and nobody has yet shipped an
> export-capable compiler (although one is very close)? Obviously
> we've lived without it for this long, and C++ is still the dominant
> commercial programming language, and people are still writing and
> shipping untold gazillions of lines of production code written in
> C++, and we don't seem to be that much the worse for wear.
Of course, by that argument, we don't need C++ at all. We lived for
years with C, and shipped gazillions of lines of production code
written in C, and we didn't seem that much worse for wear. For that
matter, I'm old enough to have delivered quite a few lines in
assembler.
We can live with anything we don't have. In a certain sense, that's
what being an engineer (as opposed to a scientist) is about. On the
other hand, the lack of export represents a potential bottle neck in
programmer productivity on large projects. Potential, because today,
very few if any people actually use templates for anything more than
the basic containers. But as the work of people like Andrei
propagates down to the masses, we can expect this to change.
> I can accept that some may consider it "unacceptable" to not have
> the facility in the standard, but that's less meaningful to me when
> that's still the only place the facility exists, and it's not
> available in the real world.
How true.
The problem, of course, isn't that we need export. It is that if we
are going to use templates in a large scale way, we definitly need to
be able to separate the implementation and the definition. If not,
C++ ends up like Java with regards to the compilation model, and
having worked on a medium size Java project, I can assure you that we
don't want that.
> I realize that's changing, finally, but the point is valid
> nonetheless. I'm perfectly happy to accept that a long-absent
> feature is desirable, but I won't easily accept that a long-absent
> feature should be branded 'essential' and its absence "unacceptable"
> when: a) it's not well-understood by most (I think I know many of
> the ramifications but I know I don't know them all);
This is really an argument that the feature is misspecified. If it is
that hard to understand, it won't be used, and templates will remain
something exotic for the experimentors and the basic library.
> b) it adds further complication in areas that are already complex
> (if that weren't true it wouldn't have been such a challenge for the
> world's smartest compiler writers); and c) I haven't yet heard
> real-world programmers clamoring for it widely (the few that I have
> heard did not correctly understand what "export" as specified in
> C++98 really does).
Well, I'm one real world programmer who is clamoring for something
that will allow me to use templates without introducing all of the
dependancies. Of course, I work mainly on Unix, which means that
except for g++, most compilers do have various hacks which allow
this. It isn't export, but it very often is just as good.
> On the other hand, those last three things are mostly not true of
> typedef templates and strict typeof, for example, and so I'm much
> more willing to consider those two things 'essential' and their
> absence "unacceptable." :-)
Well, I'm producing gazillions of code without them..
The frustrating thing about templates is that we had something that
sort of worked, modulo errors in the compiler and long compile times.
And it was taken away from us, without providing any replacement.
--
James Kanze mailto:ka...@gabi-soft.de
Beratung in objektorientierer Datenverarbeitung --
-- Conseils en informatique oriente objet
Ziegelhttenweg 17a, 60598 Frankfurt, Germany, Tl.: +49 (0)69 19 86 27
[ Send an empty e-mail to c++-...@netlab.cs.rpi.edu for info ]
Ed L Cashin
> Ed L Cashin <eca...@uga.edu> wrote in message
> news:<86k7stx...@cs.uga.edu>...
>
> > Without export, with regard to the template functions that are
> > included into each client's translation unit, you must give up the
> > advantages of separate compilation and also the advantages of shared
> > libraries.
> >
> > But is it true that *with* export the library builder would have to
> > create huge libraries that contain hundreds of definitions, one for
> > each possible sequence of template parameters?
>
> No. The library builder will create libraries which contain the
> parsed template, with its context. It isn't fully compiled.
> Compilation still happens when a template is instantiated.
That makes sense, but it doesn't seem possible to avoid losing the
advantages of shared libraries without a central location for the
compiled object code associated with each specialization.
Does anyone know how implmentors plan to do this? If they put all
generated code for specializations in a central, shared location, then
none of the advantages of shared libraries are lost. But doing that
is probably a bit tricky to get right on multi-user or
multi-security-level platforms.
--
--Ed L Cashin | PGP public key:
eca...@uga.edu | http://noserose.net/e/pgp/
[ Send an empty e-mail to c++-...@netlab.cs.rpi.edu for info ]
Daveed Vandevoorde
> goo...@vandevoorde.com (Daveed Vandevoorde) wrote in message news:<52f2f9cd.02030...@posting.google.com>...
> > ka...@gabi-soft.de (James Kanze) wrote:
> > [...]
> > > In this case, it is even more impressive that the EDG people got
> > > it first, since they only make the front end. I would imagine
> > > that it would be easier for someone with control over the entire
> > > process.
>
> > I'm not sure it would give a significant advantage, actually. (It's
> > hard to know without actually trying to implement it in such an
> > environment. However, at no point during our implementation did I
> > think "gee, I wish I could rely on some integrated environment
> > here").
>
> A mapmaker trying to choose an appropriate projection doesn't think,
> "Gee, I wish the world were flat", but his job would still be a lot
> easier if it was.
Really? If I were the map maker, I'm quite sure I would have had
that thought on a very regular basis. I think you're underestimating
people's ability to "think outside of the box."
> The one implementation today which comes closest to export, Visual
> Age, uses a global data base.
What makes you think Visual Age's scheme is any closer to the
standard "export" than, say, HP's version of implicit inclusion?
(IMO, your statement is without objective base.)
Note also that I was responding to your thoughts about "controlling
the entire process". Controlling "only the front end" does not
prevent EDG from having a "global database" (and in fact, we do
have one).
> Of course, I don't know if this makes
> that particular aspect of their compiler easier or not, but a priori,
> the more information you can maintain, and the more freedom you have
> in choosing when to do something, the more chances you have to find an
> optimal solution.
Again, based on my experience (which includes some serious
_detailed_ study of possible approaches to the problem, but
not an alternative implementation), controlling more than
the front end wouldn't be a significant help in implementing
export semantics.
> > However, the great irony here is that EDG engineers were among the
> > strongest opponents of "export" and now they are behind the first
> > implementation of the beast. I am now of the opinion that at the
> > time "export" was voted in, John Spicer (of EDG) had by far the
> > deepest insight into the implications of "export". However, I'm
> > afraid reality was even worse than those insights.
>
> I think that part of the problem is that no one (or very few) at the
> time even understood all of the implications of the new template name
> lookup. I know I didn't -- if I had realized that almost everything
> about templates was new and untested, I suspect that there would have
> been a no vote from one of the national bodies. As it is, I was not
> following templates too closely at the time (for lack of time), and I
> thought that the argument was between Borland model and CFront model,
> and not something radically new and untested.
The national bodies had access to substantial documentation to the
contrary. In addition, more than one year elapsed between the
inclusion of "export" in the working paper and the final approval
by J16 of the would-be standard. Key consequences of the new concepts
were pointed out during that time (e.g., by Erwin Unruh).
I.e., there was really no excuse.
Daveed
Francis Glassborow
Vandevoorde <goo...@vandevoorde.com> writes
>The national bodies had access to substantial documentation to the
>contrary. In addition, more than one year elapsed between the
>inclusion of "export" in the working paper and the final approval
>by J16 of the would-be standard. Key consequences of the new concepts
>were pointed out during that time (e.g., by Erwin Unruh).
>I.e., there was really no excuse.
IIRC, we had implementors more or less en bloc saying that the inclusion
model was the only one that they wanted to do, while the users were
saying that as 'separate template compilation' had always been promised
a standard without the potential for that was unacceptable. The NBs were
more concerned with breaking the deadlock.
It seems to me that any method of providing STC would allow export to be
implemented so I think the difficulty with implementing 'export'
highlights the difficulty of providing STC with templates as they have
become.
Knowing that there are problems is not always a reason to try to tackle
the can of worms. Perhaps we need to rethink the concept of the
'definition context' of a template, some way of insulating a template
definition from all other definitions that are not explicitly desired by
the template designer. If we could do that, that is what would need to
be imported into the instantiation context.
--
Francis Glassborow
Check out the ACCU Spring Conference 2002
4 Days, 4 tracks, 4+ languages, World class speakers
For details see: http://www.accu.org/events/public/accu0204.htm
[ Send an empty e-mail to c++-...@netlab.cs.rpi.edu for info ]
Bob Bell
> Michiel...@cmg.nl (Michiel Salters) writes:
>
> > Ed L Cashin <eca...@uga.edu> wrote in message
> > news:<86k7stx...@cs.uga.edu>...
> >
> > > Without export, with regard to the template functions that are
> > > included into each client's translation unit, you must give up the
> > > advantages of separate compilation and also the advantages of shared
> > > libraries.
> > >
> > > But is it true that *with* export the library builder would have to
> > > create huge libraries that contain hundreds of definitions, one for
> > > each possible sequence of template parameters?
> >
> > No. The library builder will create libraries which contain the
> > parsed template, with its context. It isn't fully compiled.
> > Compilation still happens when a template is instantiated.
>
> That makes sense, but it doesn't seem possible to avoid losing the
> advantages of shared libraries without a central location for the
> compiled object code associated with each specialization.
>
> Does anyone know how implmentors plan to do this? If they put all
> generated code for specializations in a central, shared location, then
> none of the advantages of shared libraries are lost. But doing that
> is probably a bit tricky to get right on multi-user or
> multi-security-level platforms.
How can an implementor (or anyone else) do that? How can they place
all specializations into a shared library, when the set of all
specializations is unbounded?
Templates and shared libraries don't really interact at all; export
will not change that in any way.
Bob Bell
Sungbom Kim
>
> There has been a loooooooong thread about "Implementability of export"
> in comp.std.c++.
> I found it at
> http://groups.google.de/groups?hl=de&threadm=52f2f9cd.0201100806.2999690c%40posting.google.com&rnum=1&prev=/groups%3Fq%3Dimpleme
ntability%2Bof%2Bexport%26hl%3Dde%26selm%3D52f2f9cd.0201100806.2999690c%2540posting.google.com%26rnum%3D1
>
> (the thread is even longer than that url!)
When you post a URL of Google Groups thread-view page, only the
'threadm' parameter will be sufficient. Others don't help other
readers much. (Disclaimer: I found it only from experience.)
So the looo URL can be re-written as:
http://groups.google.com/groups?threadm=52f2f9cd.0201100806.2999690c%40posting.google.com
(I also changed groups.google.de to groups.google.com)
HTH
--
Sungbom Kim <musi...@bawi.org>
Daveed Vandevoorde
> In article <52f2f9cd.02030...@posting.google.com>, Daveed
> Vandevoorde <goo...@vandevoorde.com> writes
> >The national bodies had access to substantial documentation to the
> >contrary. In addition, more than one year elapsed between the
> >inclusion of "export" in the working paper and the final approval
> >by J16 of the would-be standard. Key consequences of the new concepts
> >were pointed out during that time (e.g., by Erwin Unruh).
> >I.e., there was really no excuse.
>
> IIRC, we had implementors more or less en bloc saying that the inclusion
> model was the only one that they wanted to do, while the users were
> saying that as 'separate template compilation' had always been promised
> a standard without the potential for that was unacceptable. The NBs were
> more concerned with breaking the deadlock.
I remember it differently (see below).
However, independent of that, I think your statement contains the incorrect
(and baneful) notion that there is a "user" vs. "implementor" partition within
the committee. Clearly, many like to claim they "simply represent/are users".
There are many kinds of users and all members of the committee (including
compiler and library implementors) speak for some subset of those users.
Anyway, I went back to the 1996 committee docs (Santa Cruz, Stockholm and
Kona), and found that at the time, many implementors indeed sponsored a
proposal to remove the incompletely specified separation model that was
hinted at in the working paper (Santa Cruz). Their argument was not
"the inclusion model is the only we want to do", but "the separation
model is incompletely specified and specifying it for the first standard
is too much work, considering we are still cleaning up other mess that
was introduced several years ago." Some of the papers opposing the
separation model of that time almost feel like supernatural prophecy
when read in the context of just having implemented "export".
> It seems to me that any method of providing STC would allow export to be
> implemented
That's a mistaken perception.
> so I think the difficulty with implementing 'export'
> highlights the difficulty of providing STC with templates as they have
> become.
I'm assuming that you mean "as they have become [even if export had been
left out]"? If so, I disagree: there are alternative separate template
compilation (your "STC"?) models that can be thought of and which I
believe would both be simpler to implement and provide a better _actual_
"separation" (they would prohibit some usage patterns valid with "export").
> Knowing that there are problems is not always a reason to try to tackle
> the can of worms. Perhaps we need to rethink the concept of the
> 'definition context' of a template, some way of insulating a template
> definition from all other definitions that are not explicitly desired by
> the template designer. If we could do that, that is what would need to
> be imported into the instantiation context.
Whether or not this makes sense (I think it's the instantiation context
that needs the most change in a separation model; not the definition),
it seems pretty clear to me that by prematurely having standardized on
the current "export" semantics, we have made it much more difficult to
add a sound separation model.
Daveed
Francis Glassborow
Vandevoorde <goo...@vandevoorde.com> writes
> > model was the only one that they wanted to do, while the users were
> > saying that as 'separate template compilation' had always been promised
> > a standard without the potential for that was unacceptable. The NBs were
> > more concerned with breaking the deadlock.
>
>I remember it differently (see below).
The trouble is that the technical documents only document the technical
material and not what happened. I am trying not to name names because I
was one of those trying to broker peace between two opposing camps who
were throwing around more heat than light. J16 was adamant that the
proposal to remove 'separate compilation' went to vote, WG21 considered
any such vote as counter-productive. The political solution is only
remarkable in that all participants stuck to the script.
FWIW some form of separate template compilation (STC) was the only
supported mechanism in early drafts of the Standard. A group (led, I
believe by Borland) had the inclusion model added. In the perception of
some that same group latter came back with a proposal to remove STC from
the draft (that was at the 1996 Santa Cruz meeting). That resulted in a
group of 13 attending voting members to declare that such removal would
be 'over their dead bodies.' Such a block would certainly mean that not
form of consensus could be achieved.
export was a consensus forming solution four months latter.
>
>However, independent of that, I think your statement contains the incorrect
>(and baneful) notion that there is a "user" vs. "implementor" partition
within
>the committee. Clearly, many like to claim they "simply represent/are
users".
>There are many kinds of users and all members of the committee (including
>compiler and library implementors) speak for some subset of those users.
I do not disagree with that, however there was at that time, on that
issue a group of 13 who stated that they were defending promises that
had been made repeatedly to the user community, their term, not mine.
>
>Anyway, I went back to the 1996 committee docs (Santa Cruz, Stockholm and
>Kona), and found that at the time, many implementors indeed sponsored a
>proposal to remove the incompletely specified separation model that was
>hinted at in the working paper (Santa Cruz). Their argument was not
>"the inclusion model is the only we want to do", but "the separation
>model is incompletely specified and specifying it for the first standard
>is too much work, considering we are still cleaning up other mess that
>was introduced several years ago." Some of the papers opposing the
>separation model of that time almost feel like supernatural prophecy
>when read in the context of just having implemented "export".
Yes but see below.
>
> > It seems to me that any method of providing STC would allow export to be
> > implemented
>
>That's a mistaken perception.
OK, I won't argue because I am not a compiler expert.
>
> > so I think the difficulty with implementing 'export'
> > highlights the difficulty of providing STC with templates as they have
> > become.
>
>I'm assuming that you mean "as they have become [even if export had been
>left out]"? If so, I disagree: there are alternative separate template
>compilation (your "STC"?) models that can be thought of and which I
>believe would both be simpler to implement and provide a better _actual_
>"separation" (they would prohibit some usage patterns valid with "export").
see below.
>
> > Knowing that there are problems is not always a reason to try to tackle
> > the can of worms. Perhaps we need to rethink the concept of the
> > 'definition context' of a template, some way of insulating a template
> > definition from all other definitions that are not explicitly desired by
> > the template designer. If we could do that, that is what would need to
> > be imported into the instantiation context.
>
>Whether or not this makes sense (I think it's the instantiation context
>that needs the most change in a separation model; not the definition),
>it seems pretty clear to me that by prematurely having standardized on
>the current "export" semantics, we have made it much more difficult to
>add a sound separation model.
As no one has implemented export there can be no code that relies on its
implementation. That means that the Standards Committees could revisit
the issues. Indeed if those with expertise in the area came back and
said 'We have made a serious effort to do implement 'export' as
currently specified and cannot do so, at least in practical terms. We
need a different and workable separate compilation model." I for one
would want to listen. I know that at least two groups of implementors
have made serious attempts at the task and that problems continually
arise to block success. I begin to wonder if it is one of those problems
where success always seems to be just over the horizon. A problem that
no one manages to solve and no one manages to prove to be insoluble.
Perhaps 'export' will prove to be exactly what some wanted, a mechanism
to give time to find an achievable STC model.
--
Francis Glassborow
Check out the ACCU Spring Conference 2002
4 Days, 4 tracks, 4+ languages, World class speakers
For details see: http://www.accu.org/events/public/accu0204.htm
[ Send an empty e-mail to c++-...@netlab.cs.rpi.edu for info ]
P.J. Plauger
> In article <52f2f9cd.02030...@posting.google.com>, Daveed
> Vandevoorde <goo...@vandevoorde.com> writes
> > .....
> >Whether or not this makes sense (I think it's the instantiation context
> >that needs the most change in a separation model; not the definition),
> >it seems pretty clear to me that by prematurely having standardized on
> >the current "export" semantics, we have made it much more difficult to
> >add a sound separation model.
>
> As no one has implemented export there can be no code that relies on its
> implementation. That means that the Standards Committees could revisit
> the issues. Indeed if those with expertise in the area came back and
> said 'We have made a serious effort to do implement 'export' as
> currently specified and cannot do so, at least in practical terms. We
> need a different and workable separate compilation model." I for one
> would want to listen. I know that at least two groups of implementors
> have made serious attempts at the task and that problems continually
> arise to block success. I begin to wonder if it is one of those problems
> where success always seems to be just over the horizon. A problem that
> no one manages to solve and no one manages to prove to be insoluble.
But it's far worse than that. One company HAS implemented export template.
They supply front ends for upwards of 100 of their customers, many of whom
supply compilers in turn to the wider community. And we at Dinkumware will
follow EDG's lead and supply a Standard C++ library that makes use of
export templates for our common customers who request it. There's now no
question that, over the next couple of years, we will see widespread
availability of the very mechanism promised in the C++ Standard that froze
technically in late 1997 and was approved as an International Standard in
1998. We don't have to like the design, or what we have to do to deliver
the goods. We're in the business of supplying some of the critical pieces
needed to make a standard-conforming C++ implementation.
I don't want to hear that this is just another experiment in progress.
Nor will a broad community of C++ programmers want to hear about the
neat new things that will come with V2.0 of export templates, now that
we finally have some field experience under our belts, at least not if
V2.0 is anything other than 100 per cent backward compatible with the
current International Standard. That's the problem with standardizing
paper tigers -- they're hard to change once you know what's wrong with
them in practice.
Had export template been unimplementable, the C++ committee would have
broader latitude in redesigning it later. But apparently it's not.
Now we have some real prior art, codified in an existing International
Standard, that we'll have to live with for years to come.
> Perhaps 'export' will prove to be exactly what some wanted, a mechanism
> to give time to find an achievable STC model.
Uh huh, and then what?
P.J. Plauger
Dinkumware, Ltd.
http://www.dinkumware.com
Ed L Cashin
> Ed L Cashin <eca...@uga.edu> wrote in message
...
> > That makes sense, but it doesn't seem possible to avoid losing the
> > advantages of shared libraries without a central location for the
> > compiled object code associated with each specialization.
> >
> > Does anyone know how implmentors plan to do this? If they put all
> > generated code for specializations in a central, shared location, then
> > none of the advantages of shared libraries are lost. But doing that
> > is probably a bit tricky to get right on multi-user or
> > multi-security-level platforms.
>
> How can an implementor (or anyone else) do that? How can they place
> all specializations into a shared library, when the set of all
> specializations is unbounded?
It would be like the shared place for formatted man pages generated
from sources, or like the shared place for tex-generated bitmap fonts,
or in some kind of database ...
The specializations would be compiled once and cached in a system-wide
shared resource location. That location would be analogous to shared
libraries in terms of its advantages: clients benefit from updates;
less code bloat in clients; etc.
That way the infinity of possible specializations doesn't matter,
since only a finite number of *used* specializations would be kept in
the shared location.
> Templates and shared libraries don't really interact at all; export
> will not change that in any way.
I've never been much of a shared library advocate, but in hindsight it
seems like the shared library idea is well-accepted enough to have
become the norm.
If you look at sharing compiled code as progress from static linking,
then C++ templates look like a step backwards. Shared libraries are
the way we implement system-wide repositories of shared object code
today, but different approaches are possible.
I'm trying to find out whether the folks who develop C++ programming
platforms believe that system-wide repositories of shared object code
are "a good idea" and are working to make it possible with compiled
template specializations.
--
--Ed L Cashin | PGP public key:
eca...@uga.edu | http://noserose.net/e/pgp/
[ Send an empty e-mail to c++-...@netlab.cs.rpi.edu for info ]
Francis Glassborow
Plauger <p...@dinkumware.com> writes
>Had export template been unimplementable, the C++ committee would have
>broader latitude in redesigning it later.
I agree.
> But apparently it's not.
>Now we have some real prior art, codified in an existing International
>Standard, that we'll have to live with for years to come.
Fine if the EDG version works. However, on the other side of the fence I
do not want to hear moans from library implementors (and I am not trying
to get at anyone) that it would have been so much nicer had we done x.
--
Francis Glassborow
Check out the ACCU Spring Conference 2002
4 Days, 4 tracks, 4+ languages, World class speakers
For details see: http://www.accu.org/events/public/accu0204.htm
[ Send an empty e-mail to c++-...@netlab.cs.rpi.edu for info ]
Pete Becker
>
> A group (led, I
> believe by Borland) had the inclusion model added.
We had the most widely used compiler at the time, and that may have
influenced people's views. But we took no position on the template
compilation model, despite the informal term "Borland model" for
inclusion.
--
Pete Becker
Dinkumware, Ltd. (http://www.dinkumware.com)
[ Send an empty e-mail to c++-...@netlab.cs.rpi.edu for info ]
P.J. Plauger
> >Had export template been unimplementable, the C++ committee would have
> >broader latitude in redesigning it later.
>
> I agree.
>
> > But apparently it's not.
> >Now we have some real prior art, codified in an existing International
> >Standard, that we'll have to live with for years to come.
>
> Fine if the EDG version works.
Uh, do you have any reason to believe otherwise? THEY say it does. They've
got a sterling reputation. They have a tester whose perversity and
thoroughness is legendary. Comeau and Dinkumware have had access to beta
versions for several weeks now. In short, I wouldn't count on a complete
lack of implementations of export template for another two years as an
excuse to design yet another paper tiger, to satisfy the putative demands
of a constituency who can't live without separate compilation of templates
(but have done so for years now).
> However, on the other side of the fence I
> do not want to hear moans from library implementors (and I am not trying
> to get at anyone) that it would have been so much nicer had we done x.
I'm not sure where that came from, but that wasn't the point of my earlier
posting. I was supporting Daveed in his observation that ramming an
untried mechanism for separate compilation of templates into the C++
Standard may well set back the development of a decent mechanism for doing
so. The C++ committee has created a backward compatibility problem out of
whole cloth, and now must live with it.
I am not moaning about the current form of export template. I supported the
move in Santa Cruz to eliminate separate compilation altogether, and I
still think that would have been best for everyone. But now that it's there
and Dinkumware has access to a compiler that supports it, we're doing our
best to ensure that our customers will be able to make use of it.
I reserve my moans for issues that might still be salvageable.
P.J. Plauger
Dinkumware, Ltd.
http://www.dinkumware.com
Bob Bell
> bel...@pacbell.net (Bob Bell) writes:
>
> > How can an implementor (or anyone else) do that? How can they place
> > all specializations into a shared library, when the set of all
> > specializations is unbounded?
>
> It would be like the shared place for formatted man pages generated
> from sources, or like the shared place for tex-generated bitmap fonts,
> or in some kind of database ...
>
> The specializations would be compiled once and cached in a system-wide
> shared resource location. That location would be analogous to shared
> libraries in terms of its advantages: clients benefit from updates;
> less code bloat in clients; etc.
>
> That way the infinity of possible specializations doesn't matter,
> since only a finite number of *used* specializations would be kept in
> the shared location.
And what happens when someone wants to use a specialization that isn't
there? It seems to me that someone is just out of luck.
> > Templates and shared libraries don't really interact at all; export
> > will not change that in any way.
>
> I've never been much of a shared library advocate, but in hindsight it
> seems like the shared library idea is well-accepted enough to have
> become the norm.
>
> If you look at sharing compiled code as progress from static linking,
> then C++ templates look like a step backwards.
I don't look at DLLs as progress from static linking. Static libraries
and DLLs both have their advantages and disadvantages; one is not a
"better" version of the other.
> Shared libraries are
> the way we implement system-wide repositories of shared object code
> today, but different approaches are possible.
The only way to fix the "out of luck" scenario above is to integrate
the C++ export feature into the OS's DLL system, something I don't see
happening any time soon. And with good reason; integrating export into
the OS would be a big mistake, IMHO.
> I'm trying to find out whether the folks who develop C++ programming
> platforms believe that system-wide repositories of shared object code
> are "a good idea" and are working to make it possible with compiled
> template specializations.
There is no reason why a DLL can't contain a number of compiled
template specializations. However, this does not have anything to do
with template export, nor do I think it should.
Bob Bell
James Kanze
news:<3c863e3f$0$9414$4c41...@reader1.ash.ops.us.uu.net>...
[Concerning export...]
> I don't want to hear that this is just another experiment in
> progress.
But it is, isn't it? The same as many other things in the standard.
There was no existing practice to base the decisions on.
Of course, since it is also part of the standard, we will have to live
with it, even if the experiment shows it was an error.
> Nor will a broad community of C++ programmers want to hear about the
> neat new things that will come with V2.0 of export templates, now
> that we finally have some field experience under our belts, at least
> not if V2.0 is anything other than 100 per cent backward compatible
> with the current International Standard.
No body can speak for all the users. I can only say that amongst the
users I know, and with whom I work, what we need is 100 per cent
backward compatibility with existing compilers, not with any
theoretical document. Normally, of course, the document in question
is what guarantees this compatibility, even between different
suppliers. But it just isn't working out that way with C++. At least
not yet.
> That's the problem with standardizing paper tigers -- they're hard
> to change once you know what's wrong with them in practice.
> Had export template been unimplementable, the C++ committee would
> have broader latitude in redesigning it later. But apparently it's
> not. Now we have some real prior art, codified in an existing
> International Standard, that we'll have to live with for years to
> come.
Well, the real prior art will come when uses actually start using
export. But basically, I agree with you; we've got it, we have to
live with it.
--
James Kanze mailto:ka...@gabi-soft.de
Beratung in objektorientierer Datenverarbeitung --
-- Conseils en informatique orientée objet
Ziegelhüttenweg 17a, 60598 Frankfurt, Germany, Tél.: +49 (0)69 19 86 27
James Kanze
news:<52f2f9cd.02030...@posting.google.com>...
> ka...@gabi-soft.de (James Kanze) wrote:
> > goo...@vandevoorde.com (Daveed Vandevoorde) wrote in message
> > news:<52f2f9cd.02030...@posting.google.com>...
> > > ka...@gabi-soft.de (James Kanze) wrote:
> > > [...]
> > > > In this case, it is even more impressive that the EDG people
> > > > got it first, since they only make the front end. I would
> > > > imagine that it would be easier for someone with control over
> > > > the entire process.
> > > I'm not sure it would give a significant advantage, actually.
> > > (It's hard to know without actually trying to implement it in
> > > such an environment. However, at no point during our
> > > implementation did I think "gee, I wish I could rely on some
> > > integrated environment here").
> > A mapmaker trying to choose an appropriate projection doesn't
> > think, "Gee, I wish the world were flat", but his job would still
> > be a lot easier if it was.
> Really? If I were the map maker, I'm quite sure I would have had
> that thought on a very regular basis. I think you're
> underestimating people's ability to "think outside of the box."
Maybe. I know that I generally don't spend a lot of time thinging
about alternatives that aren't available.
> > The one implementation today which comes closest to export, Visual
> > Age, uses a global data base.
> What makes you think Visual Age's scheme is any closer to the
> standard "export" than, say, HP's version of implicit inclusion?
> (IMO, your statement is without objective base.)
Let's say that from what I have heard, Visual Age's scheme is a lot
closer to the separate compilation that I (and I suspect others)
desired.
> Note also that I was responding to your thoughts about "controlling
> the entire process". Controlling "only the front end" does not
> prevent EDG from having a "global database" (and in fact, we do have
> one).
Certainly. I'm quite confident that the people at EDG can find a
solution. And even the most primitive forms of C++ (before templates)
require something more than just a standard back-end and linker, if
only for static initializers and destructors.
Still, the more of the total process you control, the more options you
have at your disposal.
> > Of course, I don't know if this makes that particular aspect of
> > their compiler easier or not, but a priori, the more information
> > you can maintain, and the more freedom you have in choosing when
> > to do something, the more chances you have to find an optimal
> > solution.
> Again, based on my experience (which includes some serious
> _detailed_ study of possible approaches to the problem, but not an
> alternative implementation), controlling more than the front end
> wouldn't be a significant help in implementing export semantics.
Even without export, you need some extra control at link time to
handle static initialization, and possibly to remove duplicate
instantiations. With control of the linker, it would also be possible
to signal many violations of the one definition rule which currently
go undetected.
In practice, of course, EDG isn't the only one working with such
constraints. The first time I wanted to add something to a linker was
back in 1988, to support BCD arithmetic for another language I was
working on. For various reasons, it wasn't possible, even though the
compiler and linker were from the same company.
> > > However, the great irony here is that EDG engineers were among
> > > the strongest opponents of "export" and now they are behind the
> > > first implementation of the beast. I am now of the opinion
> > > that at the time "export" was voted in, John Spicer (of EDG)
> > > had by far the deepest insight into the implications of
> > > "export". However, I'm afraid reality was even worse than
> > > those insights.
> > I think that part of the problem is that no one (or very few) at
> > the time even understood all of the implications of the new
> > template name lookup. I know I didn't -- if I had realized that
> > almost everything about templates was new and untested, I suspect
> > that there would have been a no vote from one of the national
> > bodies. As it is, I was not following templates too closely at the
> > time (for lack of time), and I thought that the argument was
> > between Borland model and CFront model, and not something
> > radically new and untested.
> The national bodies had access to substantial documentation to the
> contrary. In addition, more than one year elapsed between the
> inclusion of "export" in the working paper and the final approval by
> J16 of the would-be standard. Key consequences of the new concepts
> were pointed out during that time (e.g., by Erwin Unruh). I.e.,
> there was really no excuse.
Like ANSI, the other national bodies depend on volonteers to do the
work. Except that many countries are smaller than the US, and have a
correspondingly smaller number of potential volonteers. However, I
was thinking of one particular national body, where I had at the time
considerable influence. As it happens, when the final draft appeared,
I did not have considerable time; had I realized that it basically
broke templates as we knew them, I would have fought for a no vote
from that national body, but in practice, I only had time to scan over
a few chapters of the library section.
--
James Kanze mailto:ka...@gabi-soft.de
Beratung in objektorientierer Datenverarbeitung --
-- Conseils en informatique orientée objet
Ziegelhüttenweg 17a, 60598 Frankfurt, Germany, Tél.: +49 (0)69 19 86 27
Ed L Cashin
> Ed L Cashin <eca...@uga.edu> wrote in message news:<866649w...@cs.uga.edu>...
> > bel...@pacbell.net (Bob Bell) writes:
> >
> > > How can an implementor (or anyone else) do that? How can they place
> > > all specializations into a shared library, when the set of all
> > > specializations is unbounded?
> >
> > It would be like the shared place for formatted man pages generated
> > from sources, or like the shared place for tex-generated bitmap fonts,
> > or in some kind of database ...
> >
> > The specializations would be compiled once and cached in a system-wide
> > shared resource location. That location would be analogous to shared
> > libraries in terms of its advantages: clients benefit from updates;
> > less code bloat in clients; etc.
> >
> > That way the infinity of possible specializations doesn't matter,
> > since only a finite number of *used* specializations would be kept in
> > the shared location.
>
> And what happens when someone wants to use a specialization that isn't
> there? It seems to me that someone is just out of luck.
No, the examples I cited -- manpages and tex fonts -- are both systems
where any user can trigger the "compilation" of the resource, which is
put in a system-wide cache.
...
> > Shared libraries are
> > the way we implement system-wide repositories of shared object code
> > today, but different approaches are possible.
>
> The only way to fix the "out of luck" scenario above is to integrate
> the C++ export feature into the OS's DLL system, something I don't see
> happening any time soon. And with good reason; integrating export into
> the OS would be a big mistake, IMHO.
I don't know -- given what little work I've done with D.S.O.'s I'd
guess that it would be relatively easy to add the facility to the C++
runtime itself. The first time you used a program that required a new
specialization for a template, the program would pause a bit while the
compilation and linking occurred.
> > I'm trying to find out whether the folks who develop C++ programming
> > platforms believe that system-wide repositories of shared object code
> > are "a good idea" and are working to make it possible with compiled
> > template specializations.
>
> There is no reason why a DLL can't contain a number of compiled
> template specializations. However, this does not have anything to do
> with template export, nor do I think it should.
It seems a little far out to me, too, but the more I think about it
the more practical it seems. At first I thought that making it secure
would be a problem, but now it seems like that wouldn't be too hard.
--
--Ed L Cashin | PGP public key:
eca...@uga.edu | http://noserose.net/e/pgp/
[ Send an empty e-mail to c++-...@netlab.cs.rpi.edu for info ]
Bob Bell
> bel...@pacbell.net (Bob Bell) writes:
>
> > And what happens when someone wants to use a specialization that isn't
> > there? It seems to me that someone is just out of luck.
>
> No, the examples I cited -- manpages and tex fonts -- are both systems
> where any user can trigger the "compilation" of the resource, which is
> put in a system-wide cache.
Yes, I get that. I'm saying I don't want the same kind of
functionality for templates in the OS.
> > > Shared libraries are
> > > the way we implement system-wide repositories of shared object code
> > > today, but different approaches are possible.
> >
> > The only way to fix the "out of luck" scenario above is to integrate
> > the C++ export feature into the OS's DLL system, something I don't see
> > happening any time soon. And with good reason; integrating export into
> > the OS would be a big mistake, IMHO.
>
> I don't know -- given what little work I've done with D.S.O.'s I'd
> guess that it would be relatively easy to add the facility to the C++
> runtime itself. The first time you used a program that required a new
> specialization for a template, the program would pause a bit while the
> compilation and linking occurred.
So the C++ runtime will include a compiler/linker? Given that is must
include everything that templates can do, including managing explicit
specializations and partial specializations, dependent name lookup,
etc, that's going to beef up the runtime quite a bit. What happens if
there is an error generating one of these specializations (say because
the type you're instantiating the template for doesn't support a
required interface)? Does the program crash? (Crashing is a typical
response when a required DLL is missing, so I guess crashing would be
appropriate in this case as well. :-)
And what happens when the user upgrades his compiler, and the new
version invalidates the old instantiation repository? New builds must
rebuild the repository. Old builds, which still may be around, must
continue to work as well, so the repository functionality must
continue to work with old builds as well.
> > > I'm trying to find out whether the folks who develop C++ programming
> > > platforms believe that system-wide repositories of shared object code
> > > are "a good idea" and are working to make it possible with compiled
> > > template specializations.
> >
> > There is no reason why a DLL can't contain a number of compiled
> > template specializations. However, this does not have anything to do
> > with template export, nor do I think it should.
>
> It seems a little far out to me, too, but the more I think about it
> the more practical it seems. At first I thought that making it secure
> would be a problem, but now it seems like that wouldn't be too hard.
Not only does it seem hard, but you seem to be reasoning from a
position that DLLs are good, therefore the more we can do with them
the better. DLLs open a whole can of worms that, IMHO, make them at
best a mixed blessing. In the real world, I don't see DLLs solving all
the problems they are supposed to solve, and in the bargain they throw
additional problems into the mix.
Couple that with the fact that your proposal doesn't solve any
problems that aren't already solved in other ways, and you can see
where I'm coming from. I just don't see the advantages that a dynamic
template repository would bring, but I see lots of disadvantages.
Bob Bell
Daveed Vandevoorde
> goo...@vandevoorde.com (Daveed Vandevoorde) wrote in message
> news:<52f2f9cd.02030...@posting.google.com>...
> > ka...@gabi-soft.de (James Kanze) wrote:
> > > goo...@vandevoorde.com (Daveed Vandevoorde) wrote in message
> > > news:<52f2f9cd.02030...@posting.google.com>...
> > > The one implementation today which comes closest to export, Visual
> > > Age, uses a global data base.
>
> > What makes you think Visual Age's scheme is any closer to the
> > standard "export" than, say, HP's version of implicit inclusion?
> > (IMO, your statement is without objective base.)
>
> Let's say that from what I have heard, Visual Age's scheme is a lot
> closer to the separate compilation that I (and I suspect others)
> desired.
I'm sure you'll admit yourself that's a pretty thin argument?
I'm actually somewhat familiar with the Visual Age for C++ product.
It comes with two C++ compilers (and a C compiler): a classic C++
compiler (xlC) which implements your average inclusion model, and
an incremental compiler (vacbld). I am assuming you based your
comment on something you heard about the latter.
The incremental compiler implements a language that is quite a bit
different from standard C++ (though interesting in its own right).
One (conceptual) way to think about it is that all source files are
preprocessed and the result of that is #included together into a "sea
of code" (with some special care for entities with internal linkage,
and a mechanism to extend the scope of some macros to multiple files).
This sea is then compiled incrementally in two phases (I don't know
if there are actually two phases) such that namespace scope declarations
are seen no matter where they appear in the sources (i.e., no forward
declarations are needed; IBM calls this "orderless compilation").
One of the advantages of this approach is that you can essentially
dispense with header files (and to gain competitive build times,
you should do so): there is no need to process <iostream> a thousand
times ;-). Then again, precompiled headers can buy you close to the
same advantage at a smaller cost (when used right).
This could be called a "super-inclusion model", but it's even less
of a separation model than the standard inclusion model is: the
concept of separate translation units is mostly gone. This is nice
in principle, but porting existing (large) programs to that platform
is no piece of cake. I therefore suspect that your statement that
this scheme is a lot closer to what you desired from "export" needs
some significant qualification (and perhaps was based on an incomplete
understanding?).
I think it's a neat idea, but it doesn't seem to have caught on.
(I've gotten an lot of feedback from IBM users using the classic
compilation system, but few seem to use the incremental compiler
for large projects. My guess is that it's just too vendor-specific,
but who knows.)
Daveed
Daveed Vandevoorde
[...]
> > The only way to fix the "out of luck" scenario above is to integrate
> > the C++ export feature into the OS's DLL system, something I don't see
> > happening any time soon. And with good reason; integrating export into
> > the OS would be a big mistake, IMHO.
>
> I don't know -- given what little work I've done with D.S.O.'s I'd
> guess that it would be relatively easy to add the facility to the C++
> runtime itself.
I don't believe "relatively easy" and "export" are compatible.
> The first time you used a program that required a new
> specialization for a template, the program would pause a bit while the
> compilation and linking occurred.
[...]
Note that both the library ("DSO") and the executable would need
a "quasi-source" representation to make this work. You might also
end up being surprised at the length of that "pause a bit" part if
instantiations start to cascade (and at what level of optimization
is this compilation going to happen?). Also, imagine the kind of
diagnostics...
OTOH, a little-known annecdote about "export" is that some of the
people behind its original proposal were said to be thinking of
exactly idea: "run-time instantiation". As far as I know, that
project didn't take off.
Daveed
Ed L Cashin
> Ed L Cashin <eca...@uga.edu> wrote in message news:<86sn7a4...@cs.uga.edu>...
> > bel...@pacbell.net (Bob Bell) writes:
> >
> > > And what happens when someone wants to use a specialization that isn't
> > > there? It seems to me that someone is just out of luck.
> >
> > No, the examples I cited -- manpages and tex fonts -- are both systems
> > where any user can trigger the "compilation" of the resource, which is
> > put in a system-wide cache.
>
> Yes, I get that. I'm saying I don't want the same kind of
> functionality for templates in the OS.
OK. I see.
> > > > Shared libraries are
> > > > the way we implement system-wide repositories of shared object code
> > > > today, but different approaches are possible.
> > >
> > > The only way to fix the "out of luck" scenario above is to integrate
> > > the C++ export feature into the OS's DLL system, something I don't see
> > > happening any time soon. And with good reason; integrating export into
> > > the OS would be a big mistake, IMHO.
> >
> > I don't know -- given what little work I've done with D.S.O.'s I'd
> > guess that it would be relatively easy to add the facility to the C++
> > runtime itself. The first time you used a program that required a new
> > specialization for a template, the program would pause a bit while the
> > compilation and linking occurred.
>
> So the C++ runtime will include a compiler/linker?
I was over-simplifying. I don't think that would be the way to go.
The runtime just has to trigger the compiler/linker to create any
missing specializations when they're needed. (Only the first time a
program that uses them is run.)
...
> What happens if there is an error generating one of these
> specializations (say because the type you're instantiating the
> template for doesn't support a required interface)? Does the program
> crash? (Crashing is a typical response when a required DLL is
> missing, so I guess crashing would be appropriate in this case as
> well. :-)
I think so too. The user is trying to perform an operation that isn't
defined, so there's nothing helpful to do besides (in some contexts)
printing an error message.
> And what happens when the user upgrades his compiler, and the new
> version invalidates the old instantiation repository? New builds
> must rebuild the repository. Old builds, which still may be around,
> must continue to work as well, so the repository functionality must
> continue to work with old builds as well.
I'm not sure that any of these issues don't already exist with shared
libraries.
...
> Not only does it seem hard, but you seem to be reasoning from a
> position that DLLs are good, therefore the more we can do with them
> the better. DLLs open a whole can of worms that, IMHO, make them at
> best a mixed blessing. In the real world, I don't see DLLs solving
> all the problems they are supposed to solve, and in the bargain they
> throw additional problems into the mix.
Yes, I agree that there are some hairy issues. Like I said, I'm not
much of an advocate.
> Couple that with the fact that your proposal doesn't solve any
> problems that aren't already solved in other ways, and you can see
> where I'm coming from.
I should ask, then, about those other ways, since inlining code from
header files seems inflexible. For instance I just tried to use the
boehm garbage collection library for memory leak detection, and
definitions for "new" and friends in /usr/include/g++/std/bastring.cc
caused multiple definition errors.
With a library I can specify my own function definitions first in the
list of objects to be linked, and they'll take precedence, making it
easy to override only one function definition from the standard
collection of function definitions. Would the other ways you're
talking about provide that kind of flexibility?
> I just don't see the advantages that a dynamic template repository
> would bring, but I see lots of disadvantages.
Would you agree that the main disadvantages are the same as those of
shared libraries, and the main advantages are too? Since you aren't a
fan of shared libraries, it seems pretty consistant that you wouldn't
like putting template specializations into shared libraries.
But I've heard many people, and some experts, disparaging static
libraries (not to say inlining), so I know there are other ways to
look at it.
--
--Ed L Cashin | PGP public key:
eca...@uga.edu | http://noserose.net/e/pgp/
[ Send an empty e-mail to c++-...@netlab.cs.rpi.edu for info ]
Ed L Cashin
...
> I don't believe "relatively easy" and "export" are compatible.
Fair enough. :)
> > The first time you used a program that required a new
> > specialization for a template, the program would pause a bit while the
> > compilation and linking occurred.
> [...]
>
> Note that both the library ("DSO") and the executable would need
> a "quasi-source" representation to make this work.
How could you get away with quasi? You'd need something as fleshed
out as the STL for the DSO, and something more like a traditional C
header file (declarations without definitions) for the program's
sources, right?
> You might also end up being surprised at the length of that "pause a
> bit" part if instantiations start to cascade (and at what level of
> optimization is this compilation going to happen?).
Yes, it would be part of the installation process for a new program.
> Also, imagine the kind of diagnostics...
>
> OTOH, a little-known annecdote about "export" is that some of the
> people behind its original proposal were said to be thinking of
> exactly idea: "run-time instantiation". As far as I know, that
> project didn't take off.
That makes sense, although run-time instantiation sounds worse than
install-time instantiation. I'd be interested in hearing more about
that effort and why it stopped.
--
--Ed L Cashin | PGP public key:
eca...@uga.edu | http://noserose.net/e/pgp/
[ Send an empty e-mail to c++-...@netlab.cs.rpi.edu for info ]
Bob Bell
> > So the C++ runtime will include a compiler/linker?
>
> I was over-simplifying. I don't think that would be the way to go.
> The runtime just has to trigger the compiler/linker to create any
> missing specializations when they're needed. (Only the first time a
> program that uses them is run.)
So the compiler/linker must be installed wherever this program is
going to run? That's possibly even worse than having a compiler/linker
in the runtime, because some systems integrate the compiler/linker.
> > What happens if there is an error generating one of these
> > specializations (say because the type you're instantiating the
> > template for doesn't support a required interface)? Does the program
> > crash? (Crashing is a typical response when a required DLL is
> > missing, so I guess crashing would be appropriate in this case as
> > well. :-)
>
> I think so too. The user is trying to perform an operation that isn't
> defined, so there's nothing helpful to do besides (in some contexts)
> printing an error message.
I was being facetious. A crash in this context is the programmer's
fault, not the user's. Crashing would be a pretty bad way to deal with
that.
> > And what happens when the user upgrades his compiler, and the new
> > version invalidates the old instantiation repository? New builds
> > must rebuild the repository. Old builds, which still may be around,
> > must continue to work as well, so the repository functionality must
> > continue to work with old builds as well.
>
> I'm not sure that any of these issues don't already exist with shared
> libraries.
Actually, these problems are _in addition to_ the multitude of shared
library problems. The problems I'm talking about are related to binary
compatibility, name-mangling of template instantiations and so on.
Those are not typical shared library problems.
The typical shared library versioning problems exist as well.
> > Couple that with the fact that your proposal doesn't solve any
> > problems that aren't already solved in other ways, and you can see
> > where I'm coming from.
>
> I should ask, then, about those other ways, since inlining code from
> header files seems inflexible. For instance I just tried to use the
> boehm garbage collection library for memory leak detection, and
> definitions for "new" and friends in /usr/include/g++/std/bastring.cc
> caused multiple definition errors.
I've never used the boehm garbage collector, and I don't use gcc, so I
don't know anything about bastring.cc. My point here is derived from
the fact that your proposal adds no functionality to the language --
you still write templates and rely on the compiler/linker to
instantiate them at the "right time." But there are no new code
constructs or even any changes to the way the language is used (as far
as I can see). Export already provides separate compilation of
templates.
> With a library I can specify my own function definitions first in the
> list of objects to be linked, and they'll take precedence, making it
> easy to override only one function definition from the standard
> collection of function definitions. Would the other ways you're
> talking about provide that kind of flexibility?
You are mixing platform-specific extensions with standard C++. You are
describing a situation where a program has two definitions of the same
function. This is a violation of the one definition rule, which
according to C++ is undefined behavior. Your linker allows this
behavior, specifically so you can do the "overriding" you describe.
The "other ways" I'm talking about don't address "that kind of
flexibility" because that flexibility is a platform-specific
extension, and as much as possible I'm talking about the implications
of your proposal on standard C++.
> > I just don't see the advantages that a dynamic template repository
> > would bring, but I see lots of disadvantages.
>
> Would you agree that the main disadvantages are the same as those of
> shared libraries, and the main advantages are too?
No, I would not agree with that. The main disadvantages I can think of
are:
-- added complexity to the runtime, in the sense that somehow code
must be generated, meaning that compiler/linker functionality must be
available at runtime (or at least at install time, as is pointed out
in another post).
-- no sensible error handling if a specialization cannot be generated
-- ambiguity (at least at this stage) about what kind of code should
be generated; what optimization settings apply? What preprocessor
symbols are defined, and to what value? Debug code or non-debug code?
-- the shared repository is read/write, not read-only like with
ordinary shared libraries. Thus, one application can change the shared
repository in ways that affect another application.
The main advantages are pretty much the same as the advantages of
ordinary shared libraries. However, given the extra disavantages, it
seems better to just create the specializations you need at build
time, make an ordinary shared library with them, and use that.
> Since you aren't a
> fan of shared libraries, it seems pretty consistant that you wouldn't
> like putting template specializations into shared libraries.
>
> But I've heard many people, and some experts, disparaging static
> libraries (not to say inlining), so I know there are other ways to
> look at it.
It's not so much that I am not a fan of shared libraries. I believe
they are commonly overrated and overused, but they do have their uses.
It makes little sense to ship an entire application as a set of shared
libraries, in the same way that it makes little sense to have
everything in your program be a class. You pick the right tool for the
right job.
Bob Bell
Matthew Austern
> goo...@vandevoorde.com (Daveed Vandevoorde) writes:
> > OTOH, a little-known annecdote about "export" is that some of the
> > people behind its original proposal were said to be thinking of
> > exactly idea: "run-time instantiation". As far as I know, that
> > project didn't take off.
>
> That makes sense, although run-time instantiation sounds worse than
> install-time instantiation. I'd be interested in hearing more about
> that effort and why it stopped.
Reread the thread that Daveed was responding to: he mentioned the idea
of compiling an exported template and putting it in a DSO. A DSO
blurs the distinction between link time and run time, since a program
can load a DSO at runtime by hand.
Why did that effort stop? For the most part, because it never began.
It was a hard problem, and we knew it from the beginning. We thought
about the idea of compiling templates into DSOs, and we considered it
a desirable long-term goal. We thought a little bit about what it
would involve, and very quickly realized what Daveed pointed out: that
it would mean building most of a C++ front end into the dynamic
loader. We did a few design sketches of what that might look like,
and never got beyond that.
Ed L Cashin
...
> The main advantages are pretty much the same as the advantages of
> ordinary shared libraries. However, given the extra disavantages, it
> seems better to just create the specializations you need at build
> time, make an ordinary shared library with them, and use that.
You're right -- that's a better idea. Would installing a binary-only
distribution of a program mean inserting the program's required
template specializations into existing system libraries?
--
--Ed L Cashin | PGP public key:
eca...@uga.edu | http://noserose.net/e/pgp/
[ Send an empty e-mail to c++-...@netlab.cs.rpi.edu for info ]
James Kanze
[...]
> FWIW some form of separate template compilation (STC) was the only
> supported mechanism in early drafts of the Standard. A group (led, I
> believe by Borland) had the inclusion model added. In the perception
> of some that same group latter came back with a proposal to remove
> STC from the draft (that was at the 1996 Santa Cruz meeting). That
> resulted in a group of 13 attending voting members to declare that
> such removal would be 'over their dead bodies.' Such a block would
> certainly mean that not form of consensus could be achieved.
It is probably worth pointing out that 1) between the earliest drafts
of the standard (based on the ARM) and the discussions of the separate
compilation model, name lookup in templates had radically changed, and
2) my impression, following the recent discussions here, is that those
of us insisting on separate compilation didn't necessarily mean the
same thing by it as those opposing it.
The results, of course, is that we have a name lookup that no one
really understands, coupled with a separate compilation model that no
one implements.
> export was a consensus forming solution four months latter.
> >However, independent of that, I think your statement contains the
> >incorrect (and baneful) notion that there is a "user"
> >vs. "implementor" partition within the committee. Clearly, many
> >like to claim they "simply represent/are users". There are many
> >kinds of users and all members of the committee (including compiler
> >and library implementors) speak for some subset of those users.
> I do not disagree with that, however there was at that time, on that
> issue a group of 13 who stated that they were defending promises
> that had been made repeatedly to the user community, their term, not
> mine.
More to the point, perhaps, is that there are different types of
users. In C++, most of the projects I've worked on have been very
big, and controlling compile time dependancies was a critical issue.
For such users, the current status of templates practically means that
we cannot define templates in our application level code. It's as
simple as that. And while frankly, I suspect that there isn't much
use for templates at that level anyway, it is frustrating to have a
major language feature specified in a way to make it unusable for us.
(We can, of course, use the templates defined in the standard library,
or other very stable libraries. And such use is a definite advantage;
the situation *is* better than when all we had was <generic.h>.)
--
James Kanze mailto:ka...@gabi-soft.de
Beratung in objektorientierer Datenverarbeitung --
-- Conseils en informatique oriente objet
Ziegelhttenweg 17a, 60598 Frankfurt, Germany, Tl.: +49 (0)69 19 86 27
Robert Monroe
news:<3c863e3f$0$9414$4c41...@reader1.ash.ops.us.uu.net>...
>
> broader latitude in redesigning it later. But apparently it's not.
> Now we have some real prior art, codified in an existing International
> Standard, that we'll have to live with for years to come.
>
> > Perhaps 'export' will prove to be exactly what some wanted, a mechanism
> > to give time to find an achievable STC model.
>
> Uh huh, and then what?
>
This thread is interesting. I am a person who is completely unfamiliar
with the technical issues in implementing 'export'. Nevertheless, I am
interested in the direction of the C++ standards committee and the
evolution of the language. This thread has given me some insight into
a different but related issue, that being the preprocessor and why it
was decided that C++ could not do without it. Please excuse a slight
divergence from the 'export' topic.
A couple of months ago there was a thread concerning reasonable use of
the preprocessor in C++. I made a comment expressing my belief that
the preprocessor was carried into C++ out of need for compatibility
with C. A well-known standards person slapped back saying that
compatibility with C was not the reason for including the
preprocessor. The rational was:
1. C++ has no substitute for #include.
2. There are so many variants of the language that for practical
purposes #ifdef and the like are required for compatibility with all
the various compilers and platforms.
I found this logic a little difficult to accept. Why doesn't C++ have
an alternative to #include? For C compatibility, I still believe. It
seems that the one 'legitimate' use of include is to make external
names available. Other uses tend to corrupt a program, making it a
bunch of pasted together source files. So why didn't the committee go
with a mechanism like "withing" a class or namespace declaration? For
C compatibility, I believe, along with other issues that would
complicate C++ standardisation. Other languages handle the problem and
the existence of precompiled header files indicates to me that C++
implementers have the technology. Of course making it part of the
language would require adding keywords and more importantly, making
namespaces, class declarations and class definitions separate
translation units. All of which would be highly C incompatible and a
lot more to argue over.
This thread tells me that there are other reasons for implementers to
want to hang on to the preprocessor. It defers difficult work and
decisions like those faced by Ada implementers with code sharing
generics. That makes good practical sense but the stated rational does
not seem to always reflect the underlying reason. It is interesting to
note that persons so adamant about the need for the preprocessor might
have needs for #include that go beyond C++'s lack of an alternative.
As always, just my thoughts. I've said before, I'm really quite fond
of the preprocessor.
Bob.
Daveed Vandevoorde
> Francis Glassborow <francis.g...@ntlworld.com> wrote in message
> news:<A8tl43D$vfh8...@robinton.ntlworld.com>...
>
> [...]
> > FWIW some form of separate template compilation (STC) was the only
> > supported mechanism in early drafts of the Standard.
Well the "mechanism" wasn't actually described. All the standard
said was "templates can have external linkage". It was this lack
of description that caused a number of people to ask for the
removal of the statement.
[...]
> The results, of course, is that we have a name lookup that no one
> really understands, coupled with a separate compilation model that no
> one implements.
The EDG front end implements the standard export model. (And I like
to think that I "really understand" C++ name lookup, and I know
several others who do too. Without that, it would have been even
harder to implement. ;-)
> > export was a consensus forming solution four months latter.
I don't think the final vote could be called a consensus, BTW.
It was a "majority against opposition" vote at an oversees meeting
(which typically has fewer of the regular attendees).
[...]
> More to the point, perhaps, is that there are different types of
> users. In C++, most of the projects I've worked on have been very
> big, and controlling compile time dependancies was a critical issue.
> For such users, the current status of templates practically means that
> we cannot define templates in our application level code. It's as
> simple as that.
I don't think you can generalize that observation. I worked with
several projects involving tens of millions of line of C++ code
_and_ heavy use of templates (including huge header files, multi-
vendor compiler setups, weird-and-wonderful build processes, etc.).
Build-times were an issue, but not to the point that they didn't
want application-level templates. And in fact, the build-time
issues could be resolved in all the cases I was involved in.
[...]
Daveed
Bob Bell
> bel...@pacbell.net (Bob Bell) writes:
>
> ...
> > The main advantages are pretty much the same as the advantages of
> > ordinary shared libraries. However, given the extra disavantages, it
> > seems better to just create the specializations you need at build
> > time, make an ordinary shared library with them, and use that.
>
> You're right -- that's a better idea. Would installing a binary-only
> distribution of a program mean inserting the program's required
> template specializations into existing system libraries?
If what you mean is to integrate the template specializations into the
system libraries themselves, that sounds scary (not to mention very
platform-specific, so I really couldn't say what the implications
are).
If what you mean is to place the template specializations in a central
location that is available to all applications, without affecting the
system libraries, well, you can do that if you want, but doing that
you court the worst problems of shared libraries, so be wary.
All I meant was to create an ordinary shared library with the
specializations you need, then work with that shared library the way
you'd work with any other shared library. If you place them in the
system, fine. If you place them with your app(s), fine. But I have to
say that, to me, this sounds like a solution in search of a problem. I
still don't see what special issues there are with templates that
require all this machinery.
Bob Bell
Ed L Cashin
> Ed L Cashin <eca...@uga.edu> wrote in message
> news:<86d6ya3...@cs.uga.edu>...
> > bel...@pacbell.net (Bob Bell) writes:
> >
> > ...
> > > The main advantages are pretty much the same as the advantages of
> > > ordinary shared libraries. However, given the extra disavantages, it
> > > seems better to just create the specializations you need at build
> > > time, make an ordinary shared library with them, and use that.
> >
> > You're right -- that's a better idea. Would installing a binary-only
> > distribution of a program mean inserting the program's required
> > template specializations into existing system libraries?
>
> If what you mean is to integrate the template specializations into the
> system libraries themselves, that sounds scary (not to mention very
> platform-specific, so I really couldn't say what the implications
> are).
That's what I meant, yes.
I'm sure it's platform specific, but the interesting thing about this
thread is the question, "Is separate compilation of templates a
feature of C++?" It is in the standard, but without implementations,
it's not really a feature of the language (practically speaking). The
question wouldn't be so interesting if the C++ standard was new.
I'm really just trying to understand how separate compilation of
templates can be implemented, and the templates I'm thinking of are
the STL ones. One way would be to squirrel away any compiled
specializations in a place where everyone can get to them, but as you
say, that's got its complications.
From the discussion here, it sounds like nobody is doing this, though,
so I suppose the complications won.
> If what you mean is to place the template specializations in a central
> location that is available to all applications, without affecting the
> system libraries, well, you can do that if you want, but doing that
> you court the worst problems of shared libraries, so be wary.
If the template specializations are not in the system libraries,
that's functionally equivalent from the user's perspective, so that
too is vaguely what I meant.
> All I meant was to create an ordinary shared library with the
> specializations you need, then work with that shared library the way
> you'd work with any other shared library. If you place them in the
> system, fine. If you place them with your app(s), fine. But I have to
> say that, to me, this sounds like a solution in search of a problem. I
> still don't see what special issues there are with templates that
> require all this machinery.
So rather than putting the compiled specializations in some
system-wide accessable location, each app would just keep it's own
collection of compiled template specializations. That would help with
build times and with flexibility.
As to the special issues that require all this machinery, it's not so
much "specialness" as living up to the status quo. We've gotten used
to the fact that bugfixes for functions in our shared libraries
immediately fix the bug in all the applications that use those shared
libraries. We've also gotten used to saving disk space (I know disk
is cheap) by putting object code that many apps use into shared
libraries.
Like I said, I'm not sure I accept the assumption that those benefits
are "a good thing", but I take interest in the issue because some
smart people advocate shared libraries.
Apps that use STL specializations don't enjoy those two benefits, and
your last suggestion doesn't address that because there's no central
location for the compiled object code. That's OK, but maybe I have
not made it clear enough that the machinery we've been discussing was
supposed to address the special issue: can we enjoy *all* the benefits
of separate compilation (including shared libraries) when using
templates?
--
--Ed L Cashin | PGP public key:
eca...@uga.edu | http://noserose.net/e/pgp/
[ Send an empty e-mail to c++-...@netlab.cs.rpi.edu for info ]
Bob Bell
> I'm sure it's platform specific, but the interesting thing about this
> thread is the question, "Is separate compilation of templates a
> feature of C++?" It is in the standard, but without implementations,
> it's not really a feature of the language (practically speaking). The
> question wouldn't be so interesting if the C++ standard was new.
>
> I'm really just trying to understand how separate compilation of
> templates can be implemented, and the templates I'm thinking of are
> the STL ones. One way would be to squirrel away any compiled
> specializations in a place where everyone can get to them, but as you
> say, that's got its complications.
Correct me if I'm wrong, but I think that when you say specializations
you mean something different than what I mean. When I say
specializations, I mean a specific instantiation like
std::vector<int>. I think you mean something like an intermediate
representation from which a specific instantiation can be generated.
"Squirrel away any compiled specializations" to me means "squirrel
away std::vector<int>, std::vector<float>, and std::vector<double>,
already compiled and ready to go, but nothing else."
You seem to be arguing that because the template export feature is not
implemented (which is actually incorrect -- at least one compiler
already implements it), we should investigate an alternative approach,
like doing the same thing with shared libraries. Well, considering
that
-- the standard has nothing to say about shared libraries, and
changing it to address this would be a non-trivial undertaking,
something I would possibly expect for C++0x but definately not sooner.
-- the dynamic template repository you are describing has all the
complexity of export and more, thus implementing it will take even
longer than implementing export
I don't see your proposal as a way to fill in the gap left by the
absence of export. (Of course, if you're not after filling in that
gap, but just extending the power of export, then forget the previous
two paragraphs.)
> As to the special issues that require all this machinery, it's not so
> much "specialness" as living up to the status quo. We've gotten used
> to the fact that bugfixes for functions in our shared libraries
> immediately fix the bug in all the applications that use those shared
> libraries. We've also gotten used to saving disk space (I know disk
> is cheap) by putting object code that many apps use into shared
> libraries.
That status quo you're talking about ain't so great. We've also gotten
used to fixing a bug in a shared library for one app breaking another
app. We've gotten used to having multiple versions of the same shared
libraries installed but not knowing which one is being actually used.
We've gotten used to different apps requiring different versions of
the same shared library.
The disk-space argument is moot, because, as you say, disk space is
cheap. Another argument for shared libraries, that the code must be
loaded exactly once, thus saving on RAM usage and VM thrashing, is not
compelling in the general case because the amount of code that is
identical between apps is relatively small compared to the amount of
code that differs; thus avoiding the thrashing resulting from
repeating the common code will not have a huge impact. Yes, you can
put common code into libraries, and if those libraries are stable
maybe it's even a good idea. But that's a far cry from "ship your app
as a collection of shared libraries."
> Like I said, I'm not sure I accept the assumption that those benefits
> are "a good thing", but I take interest in the issue because some
> smart people advocate shared libraries.
>
> Apps that use STL specializations don't enjoy those two benefits, and
> your last suggestion doesn't address that because there's no central
> location for the compiled object code.
My suggestion addresses it the same way ordinary shared libraries
addresses it. I can place std::vector<int>, std::list<int>, and
std::deque<int> into a shared library called IntContainers.so. I can
place std::vector<float>, etc into another called FloatContainers.so.
How is this different than having a debug C runtime shared library, a
non-debug C runtime shared library, a debug C++ runtime shared
library, etc.?
How is an app using C++Runtine.so different than an app using
IntContainer.so? What benefit does the first app gain that the second
one misses out on?
> That's OK, but maybe I have
> not made it clear enough that the machinery we've been discussing was
> supposed to address the special issue: can we enjoy *all* the benefits
> of separate compilation (including shared libraries) when using
> templates?
Separate compilation has nothing to do with shared libraries. If I
have two source files, A.cpp and B.cpp, each is compiled separately;
the compiler sees the code for A.cpp without having to see B.cpp and
vice versa. That is independent of whether A.cpp and B.cpp are in a
shared library or an application. Separate compilation does not mean
that A.cpp and B.cpp are placed in A.so and B.so.
Separate compilation of templates is the same. Just because a template
is separately compiled doesn't mean that instantiation happens during
dynamic-link time.
I ask you once more: what problem does instantiation at dynamic-link
time solve?
Bob Bell
Francis Glassborow
<bel...@pacbell.net> writes
>Correct me if I'm wrong, but I think that when you say specializations
>you mean something different than what I mean. When I say
>specializations, I mean a specific instantiation like
>std::vector<int>.
I think you 'jargon' has slipped behind that in current use.
Specialisation used to refer to what you describe but these days usually
means providing specific and different code for some combination of
arguments to a template. So the infamous vector<bool> is a
specialisation of vector for the bool type where different code is
provided from what would otherwise have been provide by instantiating
the vector template with bool.
--
Francis Glassborow
Check out the ACCU Spring Conference 2002
4 Days, 4 tracks, 4+ languages, World class speakers
For details see: http://www.accu.org/events/public/accu0204.htm
[ Send an empty e-mail to c++-...@netlab.cs.rpi.edu for info ]
Bob Bell
> In article <c87c1cfb.02031...@posting.google.com>, Bob Bell
> <bel...@pacbell.net> writes
> >Correct me if I'm wrong, but I think that when you say specializations
> >you mean something different than what I mean. When I say
> >specializations, I mean a specific instantiation like
> >std::vector<int>.
>
>
> I think you 'jargon' has slipped behind that in current use.
> Specialisation used to refer to what you describe but these days usually
> means providing specific and different code for some combination of
> arguments to a template. So the infamous vector<bool> is a
> specialisation of vector for the bool type where different code is
> provided from what would otherwise have been provide by instantiating
> the vector template with bool.
Sometimes the jargon gets in the way of what you want to say, which is
why I wanted to try to clarify what I meant. The terminology I've
heard goes like this:
A template generates an unbounded family of classes or functions. Each
member of such a family is a "specialization" of the template. Some
specializations are written by hand, like vector<bool>, and are called
"explicit specializations." Other specializations are generated from
the template by the compiler, like vector<int>, and are called
"implicit specializations." When you use a specialization, the
compiler "instantiates" it, generating the right code and including it
in the compiler's output.
Perhaps this terminology isn't used much any more. Perhaps my
definitions are just plain wrong. At the least, I think it's common
usage these days to use "specialization" as an abbreviation for
"explicit specialization." I just wanted to make sure Ed and I were
understanding each other when we said "specialization," the key
difference being that I mean it as a single member of a template's
family, whereas he seemed to be using the term to refer to something
more like the partially compiled representation of an exported
template.
Bob Bell
Steve Clamage
>
> A couple of months ago there was a thread concerning reasonable use of
> the preprocessor in C++. I made a comment expressing my belief that
> the preprocessor was carried into C++ out of need for compatibility
> with C. A well-known standards person slapped back saying that
> compatibility with C was not the reason for including the
> preprocessor. The rational was:
>
> 1. C++ has no substitute for #include.
> 2. There are so many variants of the language that for practical
> purposes #ifdef and the like are required for compatibility with all
> the various compilers and platforms.
>
> I found this logic a little difficult to accept.
Me, too. As a member of the C++ Committee since it was founded,
I always thought the primary reason was C compatibility. The
preprocessor definition and use is lifted directly from the
C standard.
> Why doesn't C++ have
> an alternative to #include? For C compatibility, I still believe.
Me, too. Since we have #include, we don't really need yet
another mechanism. We have had informal proposals for an alternative
keyword (e.g. "import") that does not involve the preprocessor as
such. One motivation for such is to remove the necessity of having
a preprocessor phase. (Other motivations are to fix problems
with #include.)
>
> This thread tells me that there are other reasons for implementers to
> want to hang on to the preprocessor.
It is not so much that implementers want to hang onto the
preprocessor. The major reason is that eliminating it would
invalidate every C++ program in existance. Program developers
would consider such a thing pretty undesirable.
--
Steve Clamage, stephen...@sun.com
Herb Sutter
>Ed L Cashin <eca...@uga.edu> wrote in message news:<86sn732...@cs.uga.edu>...
> > templates can be implemented, and the templates I'm thinking of are
> > the STL ones. One way would be to squirrel away any compiled
> > specializations in a place where everyone can get to them, but as you
> > say, that's got its complications.
>
>Correct me if I'm wrong, but I think that when you say specializations
>you mean something different than what I mean. When I say
>specializations, I mean a specific instantiation like
>std::vector<int>. I think you mean something like an intermediate
>representation from which a specific instantiation can be generated.
I believe he means that too, but the meaning of "specialization" is not the
same as "instantiation." To specialize a template means to provide a custom
version for some combination of template parameter values (possibly without
specifying all of them, in which case it's a "partial specialization"). For
example, in the standard, the only specialization of vector is for
vector<bool> which (for better or worse) behaves differently from other
vectors. The other instantiations of vector, say vector<int> or
vector<float> or vector<anything_else>, are simply instantiations of the
base template.
The issue is that the compiler has to know about all specializations seen so
far in the translation unit, in case maybe you're going to use one of them.
When you instantiate the template for a particular set of template
arguments, they may match a specialization, and then you have to instantiate
that instead of the base tempalte.
>"Squirrel away any compiled specializations" to me means "squirrel
>away std::vector<int>, std::vector<float>, and std::vector<double>,
>already compiled and ready to go, but nothing else."
>
>You seem to be arguing that because the template export feature is not
>implemented (which is actually incorrect -- at least one compiler
>already implements it),
AFAIK, not a single shipping compiler on the planet implements export. One,
EDG/Comeau, is in beta testing and will be released eventually (I don't know
the date, it's taken longer than they thought; it was supposed to be last
year, but they must be close now).
>-- the dynamic template repository you are describing has all the
>complexity of export and more, thus implementing it will take even
>longer than implementing export
I didn't read that repository proposal, but don't be quick to assume any
alternative will be harder than export -- export is really hard. Ask EDG,
and see Daveed Vandevoorde's posts here and in comp.std.c++. He ought to
know, he implemented it.
It took some really smart guys a fairly long time to do export (well, it's
"almost" done, anyway). Guys who arguably understand standard C++ templates
better than anyone on the planet, except possibly for one or two exceptional
people like Bjarne. Daveed is in fact working on the definitive book on C++
templates, with Nico Josuttis. More power to 'em! I'm looking forward to
reading prepub manuscripts of it. I'm positive I'll learn something in every
chapter.
Herb
---
Herb Sutter (www.gotw.ca)
Secretary, ISO WG21/ANSI J16 (C++) standards committee (www.gotw.ca/iso)
Contributing editor, C/C++ Users Journal (www.gotw.ca/cuj)
C++ community liaison, Microsoft (www.gotw.ca/microsoft)
Check out "THE C++ Seminar" - Boston, March 2002 (www.gotw.ca/cpp_seminar)
Ed L Cashin
...
> Correct me if I'm wrong, but I think that when you say
> specializations you mean something different than what I mean. When
> I say specializations, I mean a specific instantiation like
> std::vector<int>. I think you mean something like an intermediate
> representation from which a specific instantiation can be generated.
> "Squirrel away any compiled specializations" to me means "squirrel
> away std::vector<int>, std::vector<float>, and std::vector<double>,
> already compiled and ready to go, but nothing else."
After reading Francis Glassborow's response, I'm not sure I understand
the subtleties, and after reading Herb Sutter's response, I see I am
not distinguishing between specialization and instantiation, but when
I say "specialization", I'm not talking about an intermediate form but
rather a completely defined type, with all components completely
defined, like std::vector<int>, std::vector<float>, etc. Your
definition looks like what I have in mind:
A template generates an unbounded family of classes or functions. Each
member of such a family is a "specialization" of the template. Some
specializations are written by hand, like vector<bool>, and are called
"explicit specializations." Other specializations are generated from
the template by the compiler, like vector<int>, and are called
"implicit specializations." When you use a specialization, the
compiler "instantiates" it, generating the right code and including it
in the compiler's output.
It sounds like Herb Sutter would not consider compiler-created
specializations to be specializations at all but rather just
instantiations. In that case, I was talking about all of the
type-distinct instantiations.
...
> Well, considering that
>
> -- the standard has nothing to say about shared libraries, and
> changing it to address this would be a non-trivial undertaking,
> something I would possibly expect for C++0x but definately not sooner.
I wouldn't expect any of these implementation ideas to be
standardized.
> -- the dynamic template repository you are describing has all the
> complexity of export and more, thus implementing it will take even
> longer than implementing export
Then one problem here is that I don't understand what implementing
export entails.
...
[Ed C. says shared libraries are status quo]
> That status quo you're talking about ain't so great. We've also
> gotten used to fixing a bug in a shared library for one app breaking
> another app. We've gotten used to having multiple versions of the
> same shared libraries installed but not knowing which one is being
> actually used. We've gotten used to different apps requiring
> different versions of the same shared library.
All good points.
> The disk-space argument is moot, because, as you say, disk space is
> cheap.
I don't know -- it's never a good idea to waste finite resources. On
the other hand, if the complexity of the alternative necessitates the
sprinkling of object code around, then it's not exactly waste, so I
see your point.
> Another argument for shared libraries, that the code must be
> loaded exactly once, thus saving on RAM usage and VM thrashing, is not
> compelling in the general case because the amount of code that is
> identical between apps is relatively small compared to the amount of
> code that differs; thus avoiding the thrashing resulting from
> repeating the common code will not have a huge impact.
I forgot about that. Another instance where it would be nice to
minimize the use of a finite resource. In general, people buy disk
and memory because they want their software to go faster or do more.
Again, if there is no feasible alternative, then it's not really
wasteful to do populate memory with duplicated code, but it's not
optimal in an ideal sense.
...
> > Apps that use STL specializations don't enjoy those two benefits, and
> > your last suggestion doesn't address that because there's no central
> > location for the compiled object code.
>
> My suggestion addresses it the same way ordinary shared libraries
> addresses it. I can place std::vector<int>, std::list<int>, and
> std::deque<int> into a shared library called IntContainers.so. I can
> place std::vector<float>, etc into another called FloatContainers.so.
> How is this different than having a debug C runtime shared library, a
> non-debug C runtime shared library, a debug C++ runtime shared
> library, etc.?
If IntContainers.so goes somewhere where *all* the apps on the system
can use it, then there is no difference.
...
> I ask you once more: what problem does instantiation at dynamic-link
> time solve?
I wasn't really advocating that it's better for the instantiation to
happen at any particular time. ... After re-reading this thread I see
that my idea is not very interesting given that shared libraries
aren't necessarily a good thing.
When I asked about how export works, I learned from Michiel Salters
that of course libraries will not contain every possible
fully-compiled template instantiation. However, I was surprised to
learn that they will instead contain only partially-compiled
templates.
Which begs the question, "What happens when the template is
instantiated?" One approach would be to cache the compiled code for
the fully-instantiated templates, like happens in the manpage system
and with tetex bitmap fonts. That would speed up compile times quite
a bit, and you could even put the compiled code into shared libraries.
When I asked whether anyone is doing that, you asked,
How can an implementor (or anyone else) do that? How can they place
all specializations into a shared library, when the set of all
specializations is unbounded?
So I came up with something. The problem was the same one we've been
discussing: if the object code for the fully-instantiated templates
isn't placed in shared libraries, then the user loses the advantages
of shared libraries (if you believe in those advantages).
--
--Ed L Cashin | PGP public key:
eca...@uga.edu | http://noserose.net/e/pgp/
[ Send an empty e-mail to c++-...@netlab.cs.rpi.edu for info ]
James Kanze
|> ka...@gabi-soft.de (James Kanze) wrote:
|> > Francis Glassborow <francis.g...@ntlworld.com> wrote in
|> > message news:<A8tl43D$vfh8...@robinton.ntlworld.com>...
|> > [...]
|> > > FWIW some form of separate template compilation (STC) was the
|> > > only supported mechanism in early drafts of the Standard.
|> Well the "mechanism" wasn't actually described. All the standard
|> said was "templates can have external linkage". It was this lack of
|> description that caused a number of people to ask for the removal of
|> the statement.
|> [...]
|> > The results, of course, is that we have a name lookup that no one
|> > really understands, coupled with a separate compilation model that
|> > no one implements.
|> The EDG front end implements the standard export model. (And I like
|> to think that I "really understand" C++ name lookup, and I know
|> several others who do too. Without that, it would have been even
|> harder to implement. ;-)
Yes. I should have been clearer (and less categoric). Most people
today don't have access to a version of EDG which implements export, and
I'm not 100% sure that EDG implements export as it is described in the
standard. (Partially, of course, because I'm not quite sure what a
fully compliant compiler should do.) Similarly, of course, I imagine
that at least John Spicer understands chapter 14 (since he wrote most of
it), and that the same is probably true for some of the other
implementers.
Still, for the vast majority of users, like myself, export doesn't exist
(yet), and chapter 14 may as well be in Chinese for all its implications
are understood. I've gotten to the point where at least when the
compiler does something other than what I expect, I can more or less
find my way around the chapter, to figure out why. But I would hardly
say that I fully understand it, and with no false modesty, I suspect
that I'm better than average at understanding such things.
|> > > export was a consensus forming solution four months latter.
|> I don't think the final vote could be called a consensus, BTW. It
|> was a "majority against opposition" vote at an oversees meeting
|> (which typically has fewer of the regular attendees).
|> [...]
|> > More to the point, perhaps, is that there are different types of
|> > users. In C++, most of the projects I've worked on have been very
|> > big, and controlling compile time dependancies was a critical
|> > issue. For such users, the current status of templates
|> > practically means that we cannot define templates in our
|> > application level code. It's as simple as that.
|> I don't think you can generalize that observation. I worked with
|> several projects involving tens of millions of line of C++ code
|> _and_ heavy use of templates (including huge header files, multi-
|> vendor compiler setups, weird-and-wonderful build processes, etc.).
|> Build-times were an issue, but not to the point that they didn't
|> want application-level templates. And in fact, the build-time
|> issues could be resolved in all the cases I was involved in.
A lot depends on how the templates are used. My last project was
actually pretty small (about 40 KLOC); my collegue had just read Andrei,
and came into the project as a template enthusiast. At the end, he was
worried that I was using too many templates -- I had to specifically
show him that I had taken pains to avoid dependancies. And this in a
class which almost by definition cannot change. (It calculated an MD5
hash -- the probability of its changing is about the same as the
probability of a change in the number of days in the week.)
I don't doubt that if I could put together a team consisting of, say,
Herb Sutter, Scott Meyers, Andy Koenig and yourself, we could do a lot
with templates, with inheritance, etc. Regretfully, most of the team
members I have to work with aren't quite at that level. I've not found
a simple way to define what you can and cannot do with templates without
creating too many dependancies. So the simple guideline is not to
define templates at the application level, except as an implementation
technique where they are hidden by the compilation firewall. Like all
guidelines, of course, this one is subordonnate to the principal
guideline, which is that any guideline can be violated if the violation
is justified.
--
James Kanze mailto:ka...@gabi-soft.de
Conseils en informatique orientée objet/
Beratung in objektorientierter Datenverarbeitung
Ziegelhüttenweg 17a, 60598 Frankfurt, Germany Tel. +49(0)179 2607481