Is all this fancy C++ stuff used in the real world?

877 views
Skip to first unread message

Alexy Maykov2035754861

unread,
Nov 2, 2002, 1:01:56 PM11/2/02
to
Hello!

I was trying to output a unicode string using C++ IOStream. And I failed in
this. It turnes out that it's a known bug in Visual Studio V6.0 which is
declared to be fixed by the SP5 but it exists in VS7.0! If it's really so
then it means that people do not really use the IOStream! All C++ projects
which I worked with didn't systematically use STL and IOStream. Neither they
systematically used all this fancy teamplate stuff. The impression was that
they used C++ of 10 years ago.

So, this is how it all seems to me:

<C++ library developer>: I'd like to develop a very generic class. It'll be
universal task solver. Since any software developer solves tasks everyone is
in desperate need of the class.
(template<typename Task, typename TaskData, typename
TaskDataProperties<TaskData>> class UniversalTaskSolver;). Unfortunately
this is not supported in the language right now :(

<C++ standard comittee>: Hmmm, this wants to develop this library. We should
allow this flexibility in the language. Language should be flexible enough
to accomodate this.

<C++ compiler developer>: Hmmmm, this new feature is apporved. We should be
the first company to implement this in our compiler. Let's ship version 21.0
with support for this. We must be able to declare 95.6% standard
conformance. This is good for marketing.

<C++ library developer [on the seminar]>: Here is my universal task solver
library! It is supported by the compiler version 21.0 and it's the standard.
Everyone should use it.

<average C++ developer [on the seminar]>: Yeah, yeah whatever. This looks
cool. But I have a project , I have a deadline. I know such functions as
printf, sscanf and I know how to implement my own bubble sort. And the rest
of my group isn't on this seminar. No way we can use this library in our
code. But it was an excellent and entertaining talk.

In the end, the new feature is implemented and the language is more
powerful, but average C++ developer isn't using it. Language is more and
more complicated while only 20% percent is used by 80%. At the same time
other 80% of the language features are indeed needed and 50% of them are
implemented in different languages like Java and C#. And some people switch
to those languages because of this.

Is it really so? And if yes, what is the best way to convince peers to use
C++ features? And is it worth it?

[ Send an empty e-mail to c++-...@netlab.cs.rpi.edu for info ]
[ about comp.lang.c++.moderated. First time posters: do this! ]

P.J. Plauger

unread,
Nov 3, 2002, 6:45:15 AM11/3/02
to
"Alexy Maykov2035754861" <may...@usstories.com> wrote in message
news:aq081q$5he4t$1...@ID-149195.news.dfncis.de...

> I was trying to output a unicode string using C++ IOStream. And I failed in
> this. It turnes out that it's a known bug in Visual Studio V6.0 which is
> declared to be fixed by the SP5 but it exists in VS7.0!

You haven't stated what you think the bug is. I've done lots of Unicode I/O
using VC++ from V4.2 on, so I have reason to believe the machinery basically
works.

> If it's really so

> then it means that peoplw do not really use the IOStream!

What an interesting conclusion. The iostreams machinery is indeed widely
used. Wide streams are much less often used and are poorly understood by
most programmers. But the latter does *not* negate the former.

> All C++ projects
> which I worked with didn't systematically use STL and IOStream. Neither they
> systematically used all this fancy teamplate stuff. The impression was that
> they used C++ of 10 years ago.

That's also true for many programmers to date. But just because you haven't
used something, that doesn't mean that all people do not really use it.
And just because you failed to use something the way you hoped, that doesn't
mean it's a persistent bug.

P.J. Plauger
Dinkumware, Ltd.
http://www.dinkumware.com

Bjarke Dahl Ebert

unread,
Nov 3, 2002, 7:02:23 AM11/3/02
to
"Alexy Maykov2035754861" <may...@usstories.com> wrote in message
news:aq081q$5he4t$1...@ID-149195.news.dfncis.de...

> So, this is how it all seems to me:
> <C++ library developer>: I'd like to develop a very generic class. It'll
be
> universal task solver. Since any software developer solves tasks everyone
is
> in desperate need of the class.
> (template<typename Task, typename TaskData, typename
> TaskDataProperties<TaskData>> class UniversalTaskSolver;). Unfortunately
> this is not supported in the language right now :(

[Long, funny description removed]

This is exactly how I feel about the more "functional" aspects of STL, i.e.
the function binders and so on.
It is as if they want us to never write a for-loop again. Instead we have to
lookup everytime the names of bind_1st, mem_ptr, etc.

In my opinion, it is a lot better to write a direct for-loop than depending
on these wrappers.
If they wanted us to code in a more functional style, they should have given
us a real lambda.

Someone is going to suggest Boost, but they don't have lambdas either, even
though they want it to look that way. The favorite example of boost is
something with "cout << _1 << endl;", which of course fails miserably if
they do "cout << "Blah: " << _1 << endl;".


Bjarke

Shannon Barber

unread,
Nov 3, 2002, 7:05:31 AM11/3/02
to
"Alexy Maykov2035754861" <may...@usstories.com> wrote in message news:<aq081q$5he4t$1...@ID-149195.news.dfncis.de>...
> Hello!
>
> I was trying to output a unicode string using C++ IOStream. And I failed in
> this. It turnes out that it's a known bug in Visual Studio V6.0 which is
> declared to be fixed by the SP5 but it exists in VS7.0! If it's really so
> then it means that people do not really use the IOStream! All C++ projects
> which I worked with didn't systematically use STL and IOStream. Neither they
> systematically used all this fancy teamplate stuff. The impression was that
> they used C++ of 10 years ago.
>
Post the code, I have to think you are doing something wrong. I've
use the wide versions of the streams a little bit. However, when
coding for windows you usually have a GUI, not too many apps are
console/command line ones, and of the ones that are, most probably use
the ACSII streams.

Hillel Y. Sims

unread,
Nov 3, 2002, 3:12:55 PM11/3/02
to

"Alexy Maykov2035754861" <may...@usstories.com> wrote in message
news:aq081q$5he4t$1...@ID-149195.news.dfncis.de...
> <average C++ developer [on the seminar]>: Yeah, yeah whatever. This
looks
> cool. But I have a project , I have a deadline. I know such functions
as
> printf, sscanf and I know how to implement my own bubble sort. And the
rest
> of my group isn't on this seminar. No way we can use this library in
our
> code. But it was an excellent and entertaining talk.

If the "average C++ developer" is impressed by his/her own skills with
printf and sscanf and the ability to write a custom bubble sort
algorithm,
then I think C++ in general may be a bit much for this developer.

I attended The C++ Seminar #1 last year (if this is representative of
the
style of seminar you are referring to), and I have to say I have since
frequently used and even expanded upon many of the techniques presented
throughout production level systems, and have trained others in my
department on the use of some of these techniques, which they are
beginning
to incorporate in their own projects as well. Perhaps our system (VMS)
has a
more compliant compiler than others and we do not have to worry about
LCD-quality compilers, as writers of portable library code do, so these
"fancy" topics are indeed relevant to our work (and I like to consider
it
"real world" programming). Most of us don't exactly have PhD's...

>
> In the end, the new feature is implemented and the language is more
> powerful, but average C++ developer isn't using it. Language is more
and
> more complicated while only 20% percent is used by 80%. At the same
time
> other 80% of the language features are indeed needed and 50% of them
are
> implemented in different languages like Java and C#. And some people
switch
> to those languages because of this.

If Java or C# can effectively serve the needs of programmer(s) for a
particular problem, and the programmer(s) find it to be more
straightforward, easier, or cost-effective to work with those languages
inst
ead of C++, then they should certainly consider them. If they persist in
using those languages even in cases where it would clearly be a better
choice to use C++ (most, imnsho ;-), then they are simply exercising
poor
or uninformed judgement. The correct definitions of "effective" and
"better"
may vary. ;-)

>
> what is the best way to convince peers to use
> C++ features?
>

Actual working non-trivial-but-simple proof-of-concept code can be a
quite
powerful stimulus.

hys

--
(c) 2002 Hillel Y. Sims, all rights reserved.
FactSet Research Systems
hsims AT factset.com

Hyman Rosen

unread,
Nov 3, 2002, 5:54:55 PM11/3/02
to
Alexy Maykov2035754861 wrote:
> If it's really so then it means that people do not really use the IOStream!

No, it just means that people do not output Unicode. I'm not surprised.

> All C++ projects which I worked with didn't systematically use STL and IOStream.

I do. My group does. They did before I joined.

> Neither they systematically used all this fancy teamplate stuff.

We do. As much as we can, anyway, given the dodgy compilers.

> And some people switch to those languages because of this.

Language wars are juvenile. By all means, people should switch if doing
so makes them more productive, or happier, or whatever. Meanwhile, the
designers of any particular language should be working to perfect it in
its own style.

> what is the best way to convince peers to use C++ features?

Demonstrate that a feature helps get the job done more easily, or faster,
or more reliably.

Michael Ivey

unread,
Nov 3, 2002, 5:58:33 PM11/3/02
to
>In the end, the new feature is implemented and the language is more
>powerful, but average C++ developer isn't using it. Language is more and
>more complicated while only 20% percent is used by 80%. At the same time
>other 80% of the language features are indeed needed and 50% of them are
>implemented in different languages like Java and C#. And some people switch
>to those languages because of this.

>Is it really so? And if yes, what is the best way to convince peers to use
>C++ features? And is it worth it?

I'm convinced that the best way to convince peers and the masses is by
example. If a particular feature serves a purpose better than an older
feature, use it. As far as you or I should be concerned, let other people
do it the 'old' way that's slower and less reliable. That makes my code and
applications seem that much more impressive in comparison when the
executable is half the size, twice as fast, and updated in a few hours
instead of weeks.

Nothing speaks so loudly about your competency on the job as when you're
five times as productive as your peers given the same tools to work with.

Joke myJoke()
{
Joke thejoke = "Now leave me alone, I've a lot more printf statements to
write.";
return thejoke;
}

Michael

--
To email me, remove nospam/ from my email address.

Mirek Fidler

unread,
Nov 3, 2002, 8:50:00 PM11/3/02
to
> This is exactly how I feel about the more "functional" aspects of STL,
i.e.
> the function binders and so on.
> It is as if they want us to never write a for-loop again. Instead we have
to
> lookup everytime the names of bind_1st, mem_ptr, etc.
>
> In my opinion, it is a lot better to write a direct for-loop than
depending
> on these wrappers.
> If they wanted us to code in a more functional style, they should have
given
> us a real lambda.

Indeed. Personally, I do not like STL, moreover I do not like all that
'metaprogramming' buzz. Yes, templates are very good thing, containers are
very good thing. But STL is hard to use, unortoghonal, cryptic,
performance-wise crippled. Instead of solving real problems, you end up
devicing better ways how to express single two-lines loop using ten-lines
long for_each-like idiom and template crap. Result is unreadable, hard to
maintain and often slow code.

Mirek

David Abrahams

unread,
Nov 3, 2002, 8:51:14 PM11/3/02
to
"Bjarke Dahl Ebert" <beb...@worldonline.dk> writes:

> Someone is going to suggest Boost, but they don't have lambdas either, even
> though they want it to look that way. The favorite example of boost is
> something with "cout << _1 << endl;", which of course fails miserably if
> they do "cout << "Blah: " << _1 << endl;".

I don't know how miserably, really, but yes, it fails. You have to
write:

cout << constant("Blah") << _1 << endl

Best we can do, sorry. C++ doesn't really have lazy evaluation. If you
want that you'll need to pick up some other language.

--
David Abrahams
da...@boost-consulting.com * http://www.boost-consulting.com

Alisdair Meredith

unread,
Nov 4, 2002, 11:23:53 AM11/4/02
to
Alexy Maykov2035754861 wrote:

> ... All C++ projects


> which I worked with didn't systematically use STL and IOStream.
Neither they
> systematically used all this fancy teamplate stuff. The impression was
that
> they used C++ of 10 years ago.

[Examples snipped for space]

> Is it really so? And if yes, what is the best way to convince peers to
use
> C++ features? And is it worth it?

I certainly make much use of STL, and some use of IOStreams, although
only the narrow-char versions. The eagerness of my colleagues to adopt
such techniques seems to be proportional to their familiarity with 'C'
and their willingness to learn new techniques in general.

I've had an interesting time trying to wean people off the container
classes in the library that comes with our toolset [VCL] in favour of
the STL as a first step. It's surprising how quickly they came over to
the standard the second they had to support two compilers!

AFAICT, the 'majority' of programmers are happy using their
tried-and-testing trick and techniques and will go to quite some lengths
to avoid taking time out to learn something new, especially if they are
being productive. In order to get such developers to embrace the last
10 years, you need to throw them into a new environment where the old
tricks are not so easy/reliable as they once were.

OTOH, another group of developers will adopt anything/everything new
that comes along. They were quick into Java, currently playing with
C#/.NET, and probably a host of other languages as well. The problem
here is pinning them down long enough to learn any new tricks in an
'old' language.

--
AlisdairM

Aleksey Gurtovoy

unread,
Nov 4, 2002, 1:16:28 PM11/4/02
to
"Bjarke Dahl Ebert" <beb...@worldonline.dk> wrote in message news:<tjZw9.205408$Qk5.7...@news010.worldonline.dk>...

> In my opinion, it is a lot better to write a direct for-loop than depending
> on these wrappers.
> If they wanted us to code in a more functional style, they should have given
> us a real lambda.
>
> Someone is going to suggest Boost, but they don't have lambdas either, even
> though they want it to look that way. The favorite example of boost is
> something with "cout << _1 << endl;", which of course fails miserably if
> they do "cout << "Blah: " << _1 << endl;".

Which, of course, is clearly documented [1] and should be written as

cout << constant("Blah: ") << _1 << endl;

(or "var(cout) << "Blah: " << _1 << endl;")

If you really want to see "a real lambda" in the language, writing a
well thought formal proposal is probably a better way to get around to
it than blackening the libraries that try to make your life at least a
little bit easier.

Aleksey

[1] http://www.boost.org/libs/lambda/doc/ar01s05.html#sect:delaying_constants_and_variables

Andrea Griffini

unread,
Nov 4, 2002, 1:40:36 PM11/4/02
to
On 3 Nov 2002 20:51:14 -0500, David Abrahams
<da...@boost-consulting.com> wrote:

>I don't know how miserably, really, but yes, it fails. You have to
>write:
>
> cout << constant("Blah") << _1 << endl
>
>Best we can do, sorry. C++ doesn't really have lazy evaluation. If you
>want that you'll need to pick up some other language.

The problem is... does this nice lambda library will all
its limits (the need to wrap constants is just one of them)
really helps C++ ? Wouldn't be better trying to approach
the problem radically instead to keep trying to dig
huge holes using once again tea spoons ?
Seeing that library (that half-solves the problem in a
funky way with a lot of problems) creeping in the future
"standard" library is something that scares me a bit.
Get past the feeling "hey ma... look... doing lisp in C++"
and I think you'll agree that the lambda library is not
a serious solution. Just three arguments because STL only
needs three... c'mon. We passed the ludicrous line.

Implementing "lazy evaluation" or to say it differently
just being able to specify nameless functions inline
with bindings to the local context of where the function
is specified should be just a bit more than trivial at
the compiler level. That's IMO what is needed at *minimum*
to make any serious use of the algorithm library.
IMO C++ has a problem on this front with syntax too, but
adding some context type deduction may be can lead to
a reasonable syntax too (I'm thinking to have some form
of type inference that allows me to avoid the full type
description for the nameless function at the point of
definition). The use of prescribed name arguments
done in the lambda library looks interesting (it's used
also in PERL for sort comparision functions, but _1 and
_2 look to me less limiting than $a and $b in the view
of allowing access from the nameless function to local
context variables).

It still remains IMO quite questionable if
specifying algorithms that way is better for any
reason. But at least with the proper machinery can
be viable. IMO using algorithms nowdays (with a
couple of exceptions) is a form of masochism, and
sadism to the compiler and collegues.
Wanna talk about what are the error messages you
get if you throw in a typo ? Who uses that stuff
with current technology limits looks to me pretty
much like a fanatic.

Also I think explicit imperative code is a nice way
to specify algorithms and proved its ability to scale
well with complexity too. It's not something that
stinks and that should be removed.

Just my 0.02
Andrea

Thomas Hansen

unread,
Nov 4, 2002, 11:07:20 PM11/4/02
to
On 3 Nov 2002 15:12:55 -0500, there came a drop of sanity from "Hillel
Y. Sims" <use...@phatbasset.com> containing:

>
[snip]
5 years ago when I started to work with C++, I allways felt it like a
problem that I couldn't scale a project up, that was my biggest
problem.
The reason to this was that I didn't know enough "advanced" c++
features, neither did I know enough Design Patterns and Best Practises
etc...

Today I sometime "miss" those days since I "have" to implement a class
as a singleton, bridge or implement it generic testing against all
different iterator categories etc...
This makes even the most trivial task as the hello world program a
"problem" since I have to make a billion dollar framework to spit out
an "Hello World" to the screen.

consider...

int main( int argc, char * argv[] )
{
try {
CmdLineParser cmd<int, char*[]>( argc, argv );
// Generic class for parsing command line paramaters,
// probably about 200 lines big...
OutputBuilder<std::ostream, CmdLineParser>
myBuilder (std::cout, cmd);
// Probably yet another 3-400 lines of class spread
// into one .h and one .cpp file...
myBuilder.Write();
// Can write given input string in given format to given
// output stream...
// BUT!
// Who cares?!?
}
catch( std::exception err )
{
std::cerr << err.what();
}
}

No this was submitted as a "joke" but seriously this is a BIG problem
for me since I can't seem to solve the most trivial task without
spending weeks just to check for exception_safety,
mutithreading_safety or_some_other_safety in my
xxx_sub_function_contained_in_a_nearly_not_used_class...
Or since I wan't to make it generic, reusable and versatile enough to
use in fifty different un-imagenary ways...

Man I miss the Amstrad CPC464 and it's Basic sometimes...

When that's said, the reason to why I allways get stuck into my
"scalable hell" is because I off course thinks it's a "kick" when I
can do something nobody has done before... So it's kind of my own
fault...
;)
--
"FOOT-AND-MOUTH BELIEVED TO BE FIRST VIRUS
UNABLE TO SPREAD THROUGH MICROSOFT OUTLOOK"

Hyman Rosen

unread,
Nov 4, 2002, 11:09:05 PM11/4/02
to
Andrea Griffini wrote:
> does this nice lambda library will all its limits really helps C++?

Yes. It allows simple and straightfowrad expression of common cases
while staying within the language standard.

> Wouldn't be better trying to approach the problem radically instead

No. The lambda library allows for working solutions to problems *now*.
Radical changes in the language would take years to be promulgated,
approved, implemented, and become widespread. And that is assuming
that anyone actually has such a design.

Hillel Y. Sims

unread,
Nov 5, 2002, 1:33:19 PM11/5/02
to

"Thomas Hansen" <thomas.hansenNOSP...@adramatch.com> wrote in
message news:urqcsu8qd9770ndj6...@4ax.com...

> On 3 Nov 2002 15:12:55 -0500, there came a drop of sanity from "Hillel
> Y. Sims" <use...@phatbasset.com> containing:
>
> >
> [snip]
> 5 years ago when I started to work with C++, I allways felt it like a
> problem that I couldn't scale a project up, that was my biggest
> problem.
> The reason to this was that I didn't know enough "advanced" c++
> features, neither did I know enough Design Patterns and Best Practises
> etc...
>
> Today I sometime "miss" those days since I "have" to implement a class
> as a singleton, bridge or implement it generic testing against all
> different iterator categories etc...
> This makes even the most trivial task as the hello world program a
> "problem" since I have to make a billion dollar framework to spit out
> an "Hello World" to the screen.

#include <iostream>
using std::cout;
int main()
{
cout << "hello world!\n";
}
(c) 2002 hys , $1 billion framework

Please send royalty checks or money orders to ...

[..]

What are you trying to accomplish in that program? Are you trying to build a
production-strength system which has to handle errors and event logging and
command-line parsing, or just a toy? Software has gotten a lot bigger since
Basic on an Amstrad CPC464 (whatever that is), and programming languages and
idioms must grow as well, or become irrelevant.

Through appropriate design patterns, componentization, and layering, it is
possible to write powerful programs in C++ using few lines of code. I wrote
a nearly production-strength program to demonstrate how to read a text file,
sort it by lines, and do binary search for specific lines for someone today
in about 20 minutes and around 50 lines of code (20% of which is skeleton
code), using std::lower_bound and std::lexicographical_compare (I guess they
couldn't think of a good abbr for that name..) and various library toolkits
which had already been written and packaged generically (which use various
other std:: algorithms, etc.). I haven't found any bugs yet in my program,
and I'm pretty sure there aren't even any. I don't care if it's 10000 lines
of code involved in the background, but if the machinery is already verified
to work and I can use it here in just a few lines of code, I don't see why
that is a problem.

hys

--
Hillel Y. Sims


FactSet Research Systems
hsims AT factset.com

Andrea Griffini

unread,
Nov 5, 2002, 1:57:50 PM11/5/02
to
On 4 Nov 2002 13:16:28 -0500, agur...@meta-comm.com (Aleksey
Gurtovoy) wrote:

>If you really want to see "a real lambda" in the language, writing a
>well thought formal proposal is probably a better way to get around to
>it than blackening the libraries that try to make your life at least a
>little bit easier.

A question however is... does this pseudo-lambda
actually makes having a real lambda a nearer
possibility or a farther possibility ?

If lambda is important is this a step in the right
direction ? Are we going to get there this way ?

I'm scared that having that in the next standard
(a real possibility if I understand what boost is)
is going to make all us poor programmers to crawl
for ten more years (if not forever) instead of
giving us a better tool in that area.

Aren't there already enough little ugly parts,
asymmetries, strange limitations and quirks in this
C++ language to keep adding new ones by overusing
tools like templates far beyond the scope (i think)
they were designed for ?

Ok... ok ... templates are a new shiny hammer; and
a beautiful one, honestly. But does this mean
everything now is a nail ?

Just a doubt from a confused scared programmer...

Andrea

Bjarke Dahl Ebert

unread,
Nov 5, 2002, 1:59:18 PM11/5/02
to
"Andrea Griffini" <agr...@tin.it> wrote in message
news:3dc60fbe...@news.tin.it...

> The problem is... does this nice lambda library will all
> its limits (the need to wrap constants is just one of them)
> really helps C++ ?

I think that one of the best thing Boost provides, is the ability to
handle
"callables" more abstractly.
I.e., we have the type boost::function<void, int> (or something...) for
"functions" taking int returning void. It can then hold not only
functions,
but also "closures" ("bound functions"), member functions and so on.
This repairs C++, in my opinion, in that all "callables" in C++ belong
to
their own concrete type, instead of being special cases of a more
abstract
"can be called with one int parameter".

Doing tricks to enable fancy syntax for binders etc. is a matter of
taste.
Personally, I think they are bending the syntax of C++ a little to much
-
because constructs can easily become incomprehensible to ordinary
programmers.
It is very confusing - unless you know why - that
cout << _1 << " - hello"
works but
cout << "hello -" << _1
doesn't.

I almost think it would be better to require us to be explicit when
building
"functions" (or what are they called?). Like:
ref(cout) << _1 << constant("- hello")

> Get past the feeling "hey ma... look... doing lisp in C++"

LOL!! :-)

> IMO C++ has a problem on this front with syntax too, but
> adding some context type deduction may be can lead to
> a reasonable syntax too (I'm thinking to have some form
> of type inference that allows me to avoid the full type
> description for the nameless function at the point of
> definition).

Yes, with the more and more complex types that are involved in C++
programs,
it would be really nice with some mechanism to deduce types.

Example:
Why must I write:
std::map<MyFoo, std::vector<const MyBar*>>::iterator it = m.begin();

why not:
let it = m.begin();

The compiler knows the type anyway - it just wants me to write it down
so
that it can complain when I didn't get it right ;-)

Of course I could use a typedef, but then I have to find out where to
find
the typedef corresponding to the m-iterator. When coding the above, I
just
want to say "Hold this, please", without caring about its type. Just
like I
can do "m.begin()==m.end()" without worrying about the types involved.

There are maybe some issues regarding: should "let foo=..." become
"const
Foo& foo=...", "Foo& foo=..." or "Foo foo=...", but hopefully, this can
be
clarified.

> It still remains IMO quite questionable if
> specifying algorithms that way is better for any
> reason. But at least with the proper machinery can
> be viable.

Yes, I think other languages show that with sufficient syntax support,
it is
very nice to be able to give code as a parameter to a function.
For this to work, we need proper closures of some sort. Boost gives some
kind of closure, but with a somewhat confusing syntax (the cout<<"hello"
thing already mentioned). Boost is fine, given C++, but I would rather
like
to see C++ enhance.
Not because I miss a language with the features I want (for that I have
Python :-), but because I am sometimes required to write a solution in
C++.


Bjarke

Andy Sawyer

unread,
Nov 5, 2002, 2:59:10 PM11/5/02
to
In article <urqcsu8qd9770ndj6...@4ax.com>,
Thomas Hansen <thomas.hansenNOSP...@adramatch.com> wrote:

> Today I sometime "miss" those days since I "have" to implement a
> class as a singleton, bridge or implement it generic testing against
> all different iterator categories etc...

You don't "have" to implement a class as any of those. In fact, if you
don't want to, you don't "have" to use classes at all. C++ is a
"multi-paradigm"น language, which _allows_ you to do all the things
you talk about, but dones _not_ require them.

> This makes even the most trivial task as the hello world program a
> "problem" since I have to make a billion dollar framework to spit
> out an "Hello World" to the screen.

Only if you choose to. You can write "Hello world" as:

#include <cstdio>

int main()
{
std::puts( "Hello, world" );
}

Scarcely a "billion dollar framework" (Of course, if you want to pay
me a billion dollars for the above, I won't complain TOO loudly :-)

> catch( std::exception err )

I really hope you mean "const std::exception& err" here...

> No this was submitted as a "joke" but seriously this is a BIG
> problem for me since I can't seem to solve the most trivial task
> without spending weeks just to check for exception_safety,
> mutithreading_safety or_some_other_safety in my
> xxx_sub_function_contained_in_a_nearly_not_used_class... Or since I
> wan't to make it generic, reusable and versatile enough to use in
> fifty different un-imagenary ways...

That's your choice. It sounds to me like you may be over-engineering
your solutions - not _everything_ needs to be templated, generic or
re-useable.

> Man I miss the Amstrad CPC464 and it's Basic sometimes...

I still have a CPC464 which I'd be happy to sell you :-)

> When that's said, the reason to why I allways get stuck into my
> "scalable hell" is because I off course thinks it's a "kick" when I
> can do something nobody has done before... So it's kind of my own
> fault...

Exactly :-)

Regards,
Andy S.
Footnotes:
น I really hate that expression - it's a bit too much like
sales-speak for my liking.
--
"Light thinks it travels faster than anything but it is wrong. No matter
how fast light travels it finds the darkness has always got there first,
and is waiting for it." -- Terry Pratchett, Reaper Man

James Kanze

unread,
Nov 5, 2002, 4:50:44 PM11/5/02
to
David Abrahams <da...@boost-consulting.com> wrote in message
news:<un0oqb...@boost-consulting.com>...

> "Bjarke Dahl Ebert" <beb...@worldonline.dk> writes:

> > Someone is going to suggest Boost, but they don't have lambdas
> > either, even though they want it to look that way. The favorite
> > example of boost is something with "cout << _1 << endl;", which of
> > course fails miserably if they do "cout << "Blah: " << _1 << endl;".

> I don't know how miserably, really, but yes, it fails. You have to
> write:

> cout << constant("Blah") << _1 << endl

> Best we can do, sorry. C++ doesn't really have lazy evaluation. If you
> want that you'll need to pick up some other language.

You mean that C++ doesn't have lambda expressions, functions, classes or
whatever. The boost solution is impressive in what it manages to do
within the framework of the current language, but true lambda whatevers
is a language issue, and it cannot reasonably be solved by the library.

--
James Kanze mailto:jka...@caicheuvreux.com
Conseils en informatique orientée objet/
Beratung in objektorientierter Datenverarbeitung

Francis Glassborow

unread,
Nov 5, 2002, 4:56:51 PM11/5/02
to
In message <urqcsu8qd9770ndj6...@4ax.com>, Thomas Hansen
<thomas.hansenNOSP...@adramatch.com> writes

>No this was submitted as a "joke" but seriously this is a BIG problem
>for me since I can't seem to solve the most trivial task without
>spending weeks just to check for exception_safety,
>mutithreading_safety or_some_other_safety in my
>xxx_sub_function_contained_in_a_nearly_not_used_class...
>Or since I wan't to make it generic, reusable and versatile enough to
>use in fifty different un-imagenary ways...

Then I think you have been caught in one of the traps that lie waiting
those who believe all that some authors will say.

Every design should be made for a purpose. If you are not writing
multi-threaded code, don't support MT safety. Clearly mark your code as
not for use in a MT environment.

Do not make code generic until you have a reasonable expectation that
that added complexity (a great deal) is going to be repaid in the
reasonable future.

Do, however, write exception safe code because that cannot easily be
bolted on as an afterthought.

Have a look at the 'Agile Development' family of methodologies. They are
a good counter balance to those who advocate writing monolithic, one
template fits all conceivable uses.

Note that writing fundamental components (such as the containers in the
Standard Library) are exactly the area where genericity + exception
safety makes excellent sense but even there adding MT safety is
inappropriate if your implementation is for a single threaded
environment.

--
Francis Glassborow ACCU
64 Southfield Rd
Oxford OX4 1PA +44(0)1865 246490
All opinions are mine and do not represent those of any organisation

Hyman Rosen

unread,
Nov 5, 2002, 5:08:59 PM11/5/02
to
Andrea Griffini wrote:
> A question however is... does this pseudo-lambda
> actually makes having a real lambda a nearer
> possibility or a farther possibility ?

Irrelevant. "Pseudo-lambda" is legal and standard C++,
and therefore becomes a viable programming technique
in the language.

> If lambda is important is this a step in the right
> direction ? Are we going to get there this way ?

As usual, if a "true lambda" is desired, the way to
go about getting it is to thoroughly specify it,
perhaps create a sample implementation, and then
champion it. It can then be contrasted with
pseudo-lambda, and presumably would have some
advantages over it.

> Just a doubt from a confused scared programmer...

I don't know why you feel that it is confusing or scary
to implement libraries in C++ by using standard features
of the language. Isn't that what the language and the
standard are for? I think the Java folks are doing pretty
much the same thing with their unique feature, reflection,
to create aspect-oriented programming.

Andrea Griffini

unread,
Nov 6, 2002, 2:47:23 PM11/6/02
to
On 5 Nov 2002 17:08:59 -0500, Hyman Rosen <hyr...@mail.com> wrote:

>Andrea Griffini wrote:
>> A question however is... does this pseudo-lambda
>> actually makes having a real lambda a nearer
>> possibility or a farther possibility ?
>
>Irrelevant. "Pseudo-lambda" is legal and standard C++,
>and therefore becomes a viable programming technique
>in the language.

Of course I'm not scared by the idea of someone
using that pseudo-lambda. You may believe it or not
but I was experimenting with that kind of stuff
myself a few months ago... Actually it was a quite
more domain-specific experiment, but the ideas and
I suppose the tricks were the same.

My fear is about seeing that in the next *standard*:
that's the problem. It's partial, it's an hack,
it has a lot of drawbacks, it may seem working but
only does for very simple cases. Not the kind of
things there should be IMO in the standard library.

That pseudo-lambda is of course impressive, and better
than nothing (pratically nothing is what the standard
library currently provides) but it is not even near the
quality level that it should be (and, the real issue,
it *can't* get there).

>I don't know why you feel that it is confusing or scary
>to implement libraries in C++ by using standard features
>of the language.

No problems with libraries. A problem with the standard
library. However may be I largely overestimated the
role boost will play.

>Isn't that what the language and the standard are for?
>I think the Java folks are doing pretty much the same
>thing with their unique feature, reflection, to create
>aspect-oriented programming.

The Java standard library seemed to me it was trying
to solve all known problems instead of providing well
thought tools to use to fight yet unknown ones.
But it's long I'm not looking at Java so probably now
everthing is different ... (and will be shortly
completely different again ;) ).

Andrea

Hillel Y. Sims

unread,
Nov 7, 2002, 11:59:30 AM11/7/02
to

"Andy Sawyer" <an...@evo6.com.NoSpam> wrote in message
news:65vctd...@ender.evo6.com...

[..]


> > catch( std::exception err )
>
> I really hope you mean "const std::exception& err" here...

How about "catch (std::exception& err)" ?

hys

--
(c) 2002 Hillel Y. Sims

FactSet Research Systems
hsims AT factset.com

Hillel Y. Sims

unread,
Nov 7, 2002, 11:59:49 AM11/7/02
to

"Francis Glassborow" <francis.g...@ntlworld.com> wrote in message
news:8DxOWyIk...@robinton.demon.co.uk...
[..]


>
> Every design should be made for a purpose. If you are not writing
> multi-threaded code, don't support MT safety. Clearly mark your code
as
> not for use in a MT environment.
>

[..]


>
> Do, however, write exception safe code because that cannot easily be
> bolted on as an afterthought.
>

Interesting...

hys

--
(c) 2002 Hillel Y. Sims

FactSet Research Systems
hsims AT factset.com

Thomas Hansen

unread,
Nov 7, 2002, 12:01:23 PM11/7/02
to
On 5 Nov 2002 16:56:51 -0500, there came a drop of sanity from Francis
Glassborow <francis.g...@ntlworld.com> containing:
[snip]

>>use in fifty different un-imagenary ways...
>
>Then I think you have been caught in one of the traps that lie waiting
>those who believe all that some authors will say.
Agree!!

>
>Every design should be made for a purpose. If you are not writing
>multi-threaded code, don't support MT safety. Clearly mark your code as

>not for use in a MT environment.

Agree!!


>
>Do not make code generic until you have a reasonable expectation that
>that added complexity (a great deal) is going to be repaid in the
>reasonable future.

Agree!


>
>Do, however, write exception safe code because that cannot easily be
>bolted on as an afterthought.

Agree!!


>
>Have a look at the 'Agile Development' family of methodologies. They
are
>a good counter balance to those who advocate writing monolithic, one
>template fits all conceivable uses.

Haven't seen those...


>
>Note that writing fundamental components (such as the containers in the

>Standard Library) are exactly the area where genericity + exception
>safety makes excellent sense but even there adding MT safety is
>inappropriate if your implementation is for a single threaded
>environment.

...hmm...
Most STL vendors have in fact rewritten their std::string classes
quite recently removing the reference counted parts since it breaks
MT...


--
"FOOT-AND-MOUTH BELIEVED TO BE FIRST VIRUS
UNABLE TO SPREAD THROUGH MICROSOFT OUTLOOK"

[ Send an empty e-mail to c++-...@netlab.cs.rpi.edu for info ]

Raoul Gough

unread,
Nov 7, 2002, 9:58:56 PM11/7/02
to
"Andrea Griffini" <agr...@tin.it> wrote in message
news:3dc60fbe...@news.tin.it...
> On 3 Nov 2002 20:51:14 -0500, David Abrahams
> <da...@boost-consulting.com> wrote:
>
> >I don't know how miserably, really, but yes, it fails. You have to
> >write:
> >
> > cout << constant("Blah") << _1 << endl
> >
> >Best we can do, sorry. C++ doesn't really have lazy evaluation. If
you
> >want that you'll need to pick up some other language.
[snip]

> Implementing "lazy evaluation" or to say it differently
> just being able to specify nameless functions inline
> with bindings to the local context of where the function
> is specified should be just a bit more than trivial at
> the compiler level.

I know I've mentioned this here before, but I once wrote a
preprocessor that converts lambda expressions into C++ code. It
actually gets away with surprisingly little knowledge of the language,
and handles a broad range of situations. e.g.

// function returning a function object - this
// can actually be templated pretty easily

lambda bool (int) (int) // return type
match (int target) {
return lambda bool (int candidate) {
return candidate == __ctx(target);
};
}

// Use a returned function object
std::vector<int>::iterator
i = find_if (v.begin(), v.end(), match (7));

Of course, if the preprocessor were smarter, it wouldn't need the
"__ctx" helper, since it could determine the scope of the
identifier for itself. I won't subject you to what the C++ compiler
sees when the preprocessor is finished, but the interested can see
some more detail (or download the preprocessor) from
http://home.clara.net/raoulgough/clamp/.

> That's IMO what is needed at *minimum*
> to make any serious use of the algorithm library.
> IMO C++ has a problem on this front with syntax too, but
> adding some context type deduction may be can lead to
> a reasonable syntax too (I'm thinking to have some form
> of type inference that allows me to avoid the full type
> description for the nameless function at the point of
> definition). The use of prescribed name arguments
> done in the lambda library looks interesting (it's used
> also in PERL for sort comparision functions, but _1 and
> _2 look to me less limiting than $a and $b in the view
> of allowing access from the nameless function to local
> context variables).

Personally, I don't think this kind of "Magic" name fits very well
with the C++ style. If I want to call a template's type parameters T1,
T2 and T3, that's up to me. Maybe I want to call them Element,
Comparator and Allocator. The same goes for function parameter names.
I think a real extension to the language would have to provide some
way of assigning names explicitly, rather than requiring predetermined
ones.

Now that I think about it, I think I'll add some extra support to the
preprocessor, along these lines:

algo (x, y, lambda<T> (T p) { /*...*/ });

[snip]


> Also I think explicit imperative code is a nice way
> to specify algorithms and proved its ability to scale
> well with complexity too. It's not something that
> stinks and that should be removed.

I agree - but I have my doubts about C++ ever being particularly easy
to work with in an imperative programming style. Are there any
strongly typed languages that are? For instance, I'd be interested
to see how you would you do currying if you have to fully specify
the types of all function parameters.

Regards,
Raoul Gough.

Francis Glassborow

unread,
Nov 7, 2002, 10:05:29 PM11/7/02
to
In message <s89ksus1fmvudr5hs...@4ax.com>, Thomas Hansen
<thomas.hansenNOSP...@adramatch.com> writes

>>Standard Library) are exactly the area where genericity + exception
>>safety makes excellent sense but even there adding MT safety is
>>inappropriate if your implementation is for a single threaded
>>environment.
>...hmm...
>Most STL vendors have in fact rewritten their std::string classes
>quite recently removing the reference counted parts since it breaks
>MT...

Yes, but you need to tell the whole story. Reference counting was in
fact providing little if any benefit in most existing code and the
replacement techniques of using internal arrays for short strings made
reference counting of very doubtful benefit for this type.


--
Francis Glassborow ACCU
64 Southfield Rd
Oxford OX4 1PA +44(0)1865 246490
All opinions are mine and do not represent those of any organisation

[ Send an empty e-mail to c++-...@netlab.cs.rpi.edu for info ]

JKB

unread,
Nov 7, 2002, 10:11:05 PM11/7/02
to

"Francis Glassborow" wrote:
> Every design should be made for a purpose. If you are not writing
> multi-threaded code, don't support MT safety. Clearly mark your code
as
> not for use in a MT environment.

But there are coding practices that you should still be careful about.
For
instance, any time you have (mutable) static data, you make it very
difficult to ever convert the code to MT.

MT isn't a yes-or-no thing. Offhand, I'd say to mark your code as one
of:
internally synchronized against MT access
safe to externally synchronize
unsafe to ever use in a MT program
And I wouldn't be surprised to find some other distinction expands this
set.


In a sense, I am agreeing with you. Don't support every conceivable
behavior. Decide which ones you're going to support, which you're going
to
leave a place for somebody to implement later, and which you're going to
block off forever. Then document that decision.
-- jkb

Andy Sawyer

unread,
Nov 7, 2002, 10:12:19 PM11/7/02
to
In article <lTny9.8443$h4....@news4.srv.hcvlny.cv.net>,
on 7 Nov 2002 11:59:30 -0500,

"Hillel Y. Sims" <use...@phatbasset.com> wrote:

> "Andy Sawyer" <an...@evo6.com.NoSpam> wrote in message
> news:65vctd...@ender.evo6.com...
> > In article <urqcsu8qd9770ndj6...@4ax.com>,
> > Thomas Hansen <thomas.hansenNOSP...@adramatch.com>
> wrote:
> [..]
> > > catch( std::exception err )
> >
> > I really hope you mean "const std::exception& err" here...
>
> How about "catch (std::exception& err)" ?

Thomas's code only calls the "what()" function which is a const
member, so catching non-const buys you nothing here. Note also that
Thomas's code slices any derived type, so the "what" string may not
relate to the exception actually thrown, whilst catching a reference
(const or otherwise) will call the correct version of what.

The only non-const member function in std::exception is operator=,
and I think I can honestly say I've never seen any need to do that in
an exception handler (although that doesn't mean there isn't any need
to do it)

Regards,
Andy S.


--
"Light thinks it travels faster than anything but it is wrong. No matter
how fast light travels it finds the darkness has always got there
first,
and is waiting for it." -- Terry Pratchett, Reaper Man

[ Send an empty e-mail to c++-...@netlab.cs.rpi.edu for info ]

James Kanze

unread,
Nov 8, 2002, 1:43:00 PM11/8/02
to
Francis Glassborow <francis.g...@ntlworld.com> wrote in message
news:<VQo0CsEM...@robinton.demon.co.uk>...

> In message <s89ksus1fmvudr5hs...@4ax.com>, Thomas Hansen

> <thomas.hansenNOSP...@adramatch.com> writes
> >>Standard Library) are exactly the area where genericity + exception
> >>safety makes excellent sense but even there adding MT safety is
> >>inappropriate if your implementation is for a single threaded
> >>environment.

> >...hmm...
> >Most STL vendors have in fact rewritten their std::string classes
> >quite recently removing the reference counted parts since it breaks
> >MT...

> Yes, but you need to tell the whole story. Reference counting was in
> fact providing little if any benefit in most existing code and the
> replacement techniques of using internal arrays for short strings made
> reference counting of very doubtful benefit for this type.

Could you indicate some actual studies which have been made to prove
this? In pre-standard days, I did some tests with my own String class,
and found that a reference counted implementation made a significant
difference. Now I know that the standard string class has an interface
which inhibits copying in some cases, but those cases don't occur very
often in my code. Also, a fair number of my strings tend to be larger
than the 32 bytes which are often used for caching. So I have my doubts
for single threading.

What is true is that the standard interface requires the implementation
to look at the reference count in a lot of functions where one might not
expect it (althoug again, I don't think I make much use of those
functions in my code). And in a multi-threaded environment, you must
acquire a lock just to look at the count. That's expensive. More
important, however, is the fact that it is difficult to get right. A
quick look at the implementation of std::basic_string in g++ (3.1), for
example, shows that it doesn't attempt to protect read only accesses to
the reference count, even though other threads might access them. This
is undefined behavior; although it happens to work on my platform (a Sun
Sparc), it may fail on platforms like the Alpha (or the upcoming Intel
Merced). (There are also weaknesses in the implementation specific part
for Sparc, which doesn't work correctly if threads are given different
priorities. This is a limitation which, if it were documented, I could
live with, however.) In the end, the real alternatives for a
multi-threaded environment seem to be a reference counted version which
doesn't work, a reference counted version which is several orders of
magnitude slower, or a non-reference counted version. Faced with those
options, the choice is obvious.

--
James Kanze mailto:jka...@caicheuvreux.com
Conseils en informatique orientée objet/
Beratung in objektorientierter Datenverarbeitung

[ Send an empty e-mail to c++-...@netlab.cs.rpi.edu for info ]

Mirek Fidler

unread,
Nov 8, 2002, 10:26:00 PM11/8/02
to

Recently I came to conclusion that possible way how to solve this is
to
simply keep using 'reference counted version that does not work',
because it
works well within single thread. But in cases when value is being passed
to
another thread (e.g. when assigning or retrieving value of global shared
variable), special 'deep-copy' operation would be used.

Primitive form of this is something like:

std::string shared;

void SetShared(const std::string& data)
{
aquire_lock_for_shared;
shared = data.c_str();
release_lock_for_shared;
}

Yes, this would be rather error-prone, but it could be very well
encapsulated in special class 'shared_string', which could also include
thread locks, something like

std::shared_string shared;

void SetShared(const std::string& data) { shared = data; }

This could work well...

It is just an idea, anyway.

Mirek

P.S.: Other possibility is to use my optional garbage collector for
string
data :)

Shannon Barber

unread,
Nov 9, 2002, 11:25:05 AM11/9/02
to
"Hillel Y. Sims" <use...@phatbasset.com> wrote in message
news:<2iIx9.17757$0x....@news4.srv.hcvlny.cv.net>...
echo "Hello World"

http://www.ariel.com.au/jokes/The_Evolution_of_a_Programmer.html

Francis Glassborow

unread,
Nov 9, 2002, 11:31:25 AM11/9/02
to
In message <aqhatf$cp9$1...@news.vol.cz>, Mirek Fidler <c...@centrum.cz>
writes

> Recently I came to conclusion that possible way how to solve this
is
>to
>simply keep using 'reference counted version that does not work',
>because it
>works well within single thread.

Like all optimisations, we should measure before choosing. Many
applications do not use many 'long' strings (i.e. longer than 32 bytes).

In such cases the use of an internal buffer may well outperform other
options.

However be aware that advances in technology can have surprising
results. For example we all 'know' that inserting an element into a
sequence is quickest when the sequence is a list, and probably slowest
when it is a vector. However I was listening to Andy Koenig recently
when he reported the results of actual measurements. Given 'short'
containers, i.e. ones small enough to fit in some level of CPU cache,
inserts to vectors usually outperformed insertions to lists. The reason
being that the contiguous nature of a vector works better with caches.

A good deal of popular theory is based on the assumption that memory
access times are constant. In the real world this is very far from true
when systems perhaps have three levels of cache, main RAM and virtual
storage on disk.

Having a whole object present in cache can provide extra performance
benefits, but to determine if this is the case for a specific system
requires good measurement.


--
Francis Glassborow ACCU
64 Southfield Rd
Oxford OX4 1PA +44(0)1865 246490
All opinions are mine and do not represent those of any organisation

[ Send an empty e-mail to c++-...@netlab.cs.rpi.edu for info ]

Mirek Fidler

unread,
Nov 9, 2002, 5:54:13 PM11/9/02
to
> >simply keep using 'reference counted version that does not work',
> >because it
> >works well within single thread.
>
> Like all optimisations, we should measure before choosing. Many
> applications do not use many 'long' strings (i.e. longer than 32 bytes).

That is right, as long as you use strings only to contain character
data. Anyway, I am acustomed to using things like

String LoadFile(const char *filename);

that loads content of file and returns it as string. Without some form
of optimization, creating two full 'deep copies' of string would cost too
much here.

Mirek

Francis Glassborow

unread,
Nov 9, 2002, 9:01:04 PM11/9/02
to
In message <aqjkbb$1sne$1...@news.vol.cz>, Mirek Fidler <c...@centrum.cz>
writes

>> >simply keep using 'reference counted version that does not work',
>> >because it
>> >works well within single thread.
>>
>> Like all optimisations, we should measure before choosing. Many
>> applications do not use many 'long' strings (i.e. longer than 32 bytes).
>
> That is right, as long as you use strings only to contain character
>data. Anyway, I am acustomed to using things like
>
>String LoadFile(const char *filename);
>
> that loads content of file and returns it as string. Without some form
>of optimization, creating two full 'deep copies' of string would cost too
>much here.

I do not think we should all pay for your choice of programming
mechanism. As you can do nothing but read from such a string (without
triggering a copy) I can think of many other ways of achieving the same
benefits.


--
Francis Glassborow ACCU
64 Southfield Rd
Oxford OX4 1PA +44(0)1865 246490
All opinions are mine and do not represent those of any organisation

[ Send an empty e-mail to c++-...@netlab.cs.rpi.edu for info ]

Joshua Lehrer

unread,
Nov 10, 2002, 10:47:55 AM11/10/02
to
"Mirek Fidler" <c...@centrum.cz> wrote in message news:<aqhatf$cp9$1...@news.vol.cz>...

> > What is true is that the standard interface requires the
> implementation
> > to look at the reference count in a lot of functions where one might
> not
> > expect it (althoug again, I don't think I make much use of those
> > functions in my code). And in a multi-threaded environment, you must
> > acquire a lock just to look at the count. That's expensive. More

This is simply not true. As I have posted before, a reference counted
string class can be written using only atomic_increment and
atomic_decrement, and no other semaphors, locks, or mutexes. You are
all set as long as atomic_* each return either the pre- or
post-modification value. Search google for my name along with
"string" and you should find the posts.

joshua lehrer
factset research systems

Hillel Y. Sims

unread,
Nov 11, 2002, 1:53:11 AM11/11/02
to
{This part of the thread is drifting off-topic for clc++m.
Please redirect followups to the appropriate system-specific
or advocacy newsgroups, or to email. Thanks! -mod}

"Shannon Barber" <shannon...@myrealbox.com> wrote in message
news:de001473.02110...@posting.google.com...


> "Hillel Y. Sims" <use...@phatbasset.com> wrote in message
> news:<2iIx9.17757$0x....@news4.srv.hcvlny.cv.net>...
> >

> > #include <iostream>
> > using std::cout;
> > int main()
> > {
> > cout << "hello world!\n";
> > }
> > (c) 2002 hys , $1 billion framework
> >
> > Please send royalty checks or money orders to ...
> >
> echo "Hello World"
>
> http://www.ariel.com.au/jokes/The_Evolution_of_a_Programmer.html
>

Yeah.. too bad we were specifically talking about C++ programming! ;-)

hys

--
(c) 2002 Hillel Y. Sims


FactSet Research Systems
hsims AT factset.com

Andrei Alexandrescu

unread,
Nov 12, 2002, 5:44:48 PM11/12/02
to
"Mirek Fidler" <c...@centrum.cz> wrote in message
news:aqjkbb$1sne$1...@news.vol.cz...

> Anyway, I am acustomed to using things like
>
> String LoadFile(const char *filename);
>
> that loads content of file and returns it as string. Without some
form
> of optimization, creating two full 'deep copies' of string would cost
too
> much here.

Of the possible optimizations, best is, of course, to use Mojo :oD.

By the way, my draft article (http://moderncppdesign.com/mojo) makes the
conjecture that many textbooks and many people use reference counting
solely
because it optimizes *one* copy (the return value), not because there
are
tons of strings with the same content lying around.


Andrei

Thaddeus L Olczyk

unread,
Nov 12, 2002, 5:54:12 PM11/12/02
to
gOn 9 Nov 2002 11:31:25 -0500, Francis Glassborow
<francis.g...@ntlworld.com> wrote:

>However be aware that advances in technology can have surprising
>results. For example we all 'know' that inserting an element into a
>sequence is quickest when the sequence is a list, and probably slowest
>when it is a vector. However I was listening to Andy Koenig recently
>when he reported the results of actual measurements. Given 'short'
>containers, i.e. ones small enough to fit in some level of CPU cache,
>inserts to vectors usually outperformed insertions to lists. The reason

>being that the contiguous nature of a vector works better with caches.

That's not the reason.
Assume ( just as a number to demonstrate ) that a cache hit makes an
instruction a 100 times longer. Let us also assume that an algorithm
that does everything in the cache is O(n*n) and one that involves
a lot of cache hits O(n*log n).

Then you incur an over head in the second algorithm which is
approximately 100. For the second algorithm to catch up ( and overtake
as such an algorithm must eventually do ) the first you have to let
n become at least 250. Since you assume that the algorithm "runs in
cahce" that makes it hard to find a situation where n ( which if too
large will run out of cache ) allows the second algorithm to rule,
while still running in cache in the first algorithm.

PS: I tend to use vectors of pointers to objects as it is not that
expensive to copy a block of pointers.

Joshua Lehrer

unread,
Nov 13, 2002, 5:00:34 PM11/13/02
to
I can't prove that my numbers aren't only due to optimizing away the
copy on the return value, but I'll give you them anyway.

Turning off reference counting of my String class, a common run
through of my application creates over 2.2 million Strings.

Turning reference counting on reduces that number to just over 560
thousand created that didn't reference other Strings.

This was a huge win for us here. Switching to our own custom
reference counted String class, and off of the vendor supplied one,
showed a marked decrease in CPU consumption of our applications.

joshua lehrer
factset research systems

[ Send an empty e-mail to c++-...@netlab.cs.rpi.edu for info ]

James Kanze

unread,
Nov 13, 2002, 6:26:23 PM11/13/02
to
Francis Glassborow <francis.g...@ntlworld.com> wrote in message
news:<Ev6AtgMA...@robinton.demon.co.uk>...

> In message <aqhatf$cp9$1...@news.vol.cz>, Mirek Fidler <c...@centrum.cz>
> writes

> > Recently I came to conclusion that possible way how to solve this
> >is to simply keep using 'reference counted version that does not
> >work', because it works well within single thread.

> Like all optimisations, we should measure before choosing. Many
> applications do not use many 'long' strings (i.e. longer than 32
> bytes).

And many use longer strings. Is it our intention to normalize a string
class which cannot reasonably be used for strings longer than 32 bytes?
(I've never seen an application which didn't have some strings longer
than 32 bytes. How many, and what operations were done with them, is
another story.)

> In such cases the use of an internal buffer may well outperform other
> options.

If you're copying a lot, it will still depend somewhat on the hardware
architecture, at least in a single threaded environment. In one case,
you copy a pointer (four bytes?), and increment a word. In the other,
you copy 32 bytes.

There is also the case that some compilers might optimize a class
consisting of a single pointer into a register, but won't do the same
for a class with a 32 byte array.

> However be aware that advances in technology can have surprising
> results. For example we all 'know' that inserting an element into a
> sequence is quickest when the sequence is a list, and probably slowest
> when it is a vector. However I was listening to Andy Koenig recently
> when he reported the results of actual measurements. Given 'short'
> containers, i.e. ones small enough to fit in some level of CPU cache,
> inserts to vectors usually outperformed insertions to lists. The
> reason being that the contiguous nature of a vector works better with
> caches.

Even with longer containers. A list requires a dynamic memory
allocation at each insertion; a vector doesn't. Depending on the
relative costs of shifting and dynamic allocation, vector can be
significantly faster even without the effect on the cache.

--
James Kanze mailto:jka...@caicheuvreux.com
Conseils en informatique orientée objet/
Beratung in objektorientierter Datenverarbeitung

[ Send an empty e-mail to c++-...@netlab.cs.rpi.edu for info ]

James Kanze

unread,
Nov 13, 2002, 6:26:59 PM11/13/02
to
usene...@lehrerfamily.com (Joshua Lehrer) wrote in message
news:<31c49f0d.02111...@posting.google.com>...

> "Mirek Fidler" <c...@centrum.cz> wrote in message
> news:<aqhatf$cp9$1...@news.vol.cz>...
> > > What is true is that the standard interface requires the
> > > implementation to look at the reference count in a lot of
> > > functions where one might not expect it (althoug again, I don't
> > > think I make much use of those functions in my code). And in a
> > > multi-threaded environment, you must acquire a lock just to look
> > > at the count. That's expensive. More

> This is simply not true. As I have posted before, a reference counted
> string class can be written using only atomic_increment and
> atomic_decrement, and no other semaphors, locks, or mutexes.

This has often been claimed, but
- no one has yet shown an implementation which does so and is conform,
and
- under Posix, at least, there is no atomic_increment nor
atomic_decrement, and the only way to implement these functions uses
a mutex.

Concering the first point: you don't want to systematically make copies
in string::begin and string::end, and indeed, you can't and remain
conform. This means that you have to be able to *read* the count,
without modifying it. Or can you show me another implementation?

> You are all set as long as atomic_* each return either the pre- or
> post-modification value. Search google for my name along with
> "string" and you should find the posts.

I find a post where you claim this is true, with a vague demonstration
based on operator=, and a bit of hand-waving concerning the other
functions. The operator= function is NOT a problem; even if you acquire
a lock, it is likely to be less expensive than allocating a new block,
since the allocation routine must also acquire a lock. The real
problems are the functions in which the implementation doesn't know
whether a modification will take place or not: the non-const versions of
begin, end, at and operator[].

Of course, this is easily solved: we'll also assume a function which can
read atomically (and synchronized -- the atomicity is only part of the
problem). While it would be nice if such primitives exist, they don't
under Posix, and the only way to implement them, according to Posix, is
to use a mutex or some other synchronizing system call. (Some Posix
systems, such as Solaris on Sparc or Linux on Intel, may have additional
possibilities, but from what I have been told, this is NOT the case for
such architectures as the Alpha and Merced. And I've yet to see an
atomic increment or decrement without a Mutex which works under Solaris
on a Sparc.)

--
James Kanze mailto:jka...@caicheuvreux.com
Conseils en informatique orientée objet/
Beratung in objektorientierter Datenverarbeitung

[ Send an empty e-mail to c++-...@netlab.cs.rpi.edu for info ]

Roland Pibinger

unread,
Nov 14, 2002, 8:09:59 AM11/14/02
to
On 9 Nov 2002 17:54:13 -0500, "Mirek Fidler" <c...@centrum.cz> wrote:

>> >simply keep using 'reference counted version that does not work',
>> >because it
>> >works well within single thread.
>>
>> Like all optimisations, we should measure before choosing. Many
>> applications do not use many 'long' strings (i.e. longer than 32
bytes).
>
> That is right, as long as you use strings only to contain character
>data. Anyway, I am acustomed to using things like
>
>String LoadFile(const char *filename);
>
> that loads content of file and returns it as string. Without some
form
>of optimization, creating two full 'deep copies' of string would cost
too
>much here.

No need for optimization, just avoid pessimization:

String& LoadFile(const char *filename, String& out);
loads a file into out and returns it without a copy.

Best regards,
Roland Pibinger

Silicon Graphics

unread,
Nov 14, 2002, 8:14:56 AM11/14/02
to
James Kanze <ka...@gabi-soft.de> wrote:
: usene...@lehrerfamily.com (Joshua Lehrer) wrote in message

: news:<31c49f0d.02111...@posting.google.com>...
:> "Mirek Fidler" <c...@centrum.cz> wrote in message
:> news:<aqhatf$cp9$1...@news.vol.cz>...
:> > > What is true is that the standard interface requires the
:> > > implementation to look at the reference count in a lot of
:> > > functions where one might not expect it (althoug again, I don't
:> > > think I make much use of those functions in my code). And in a
:> > > multi-threaded environment, you must acquire a lock just to look
:> > > at the count. That's expensive. More

:> This is simply not true. As I have posted before, a reference
counted
:> string class can be written using only atomic_increment and
:> atomic_decrement, and no other semaphors, locks, or mutexes.

: This has often been claimed, but
: - no one has yet shown an implementation which does so and is
conform,
: and
: - under Posix, at least, there is no atomic_increment nor
: atomic_decrement, and the only way to implement these functions
uses
: a mutex.

If you make the concession that all ref counted objects are immutable,
the complexity level drops considerably. Whether that is useful for
your
environment is a different question. Obtaining the ref count as a rhs
isn't that hard as long as you have the appropriate memory barrier
primitives.

<elided>

: Of course, this is easily solved: we'll also assume a function which


can
: read atomically (and synchronized -- the atomicity is only part of the
: problem). While it would be nice if such primitives exist, they don't
: under Posix, and the only way to implement them, according to Posix,
is
: to use a mutex or some other synchronizing system call. (Some Posix
: systems, such as Solaris on Sparc or Linux on Intel, may have
additional
: possibilities, but from what I have been told, this is NOT the case
for
: such architectures as the Alpha and Merced. And I've yet to see an
: atomic increment or decrement without a Mutex which works under
Solaris
: on a Sparc.)

:
I don't know if you consider CAS or its 64-bit bother CASX a mutex or
not:

http://soldc.sun.com/articles/stl-new.html

Adam
ze...@best.com

: --
: James Kanze mailto:jka...@caicheuvreux.com

[ Send an empty e-mail to c++-...@netlab.cs.rpi.edu for info ]

Joseph Seigh

unread,
Nov 14, 2002, 8:14:37 AM11/14/02
to

James Kanze wrote:
>
> usene...@lehrerfamily.com (Joshua Lehrer) wrote in message
> news:<31c49f0d.02111...@posting.google.com>...
> > "Mirek Fidler" <c...@centrum.cz> wrote in message
> > news:<aqhatf$cp9$1...@news.vol.cz>...
> > > > What is true is that the standard interface requires the
> > > > implementation to look at the reference count in a lot of
> > > > functions where one might not expect it (althoug again, I don't
> > > > think I make much use of those functions in my code). And in a
> > > > multi-threaded environment, you must acquire a lock just to look
> > > > at the count. That's expensive. More
>
> > This is simply not true. As I have posted before, a reference
counted
> > string class can be written using only atomic_increment and
> > atomic_decrement, and no other semaphors, locks, or mutexes.
>
> This has often been claimed, but
> - no one has yet shown an implementation which does so and is
conform,
> and
> - under Posix, at least, there is no atomic_increment nor
> atomic_decrement, and the only way to implement these functions
uses
> a mutex.
>

You could implement it using hardware dependent assembly code. Most
platforms
have a compare and swap or something that can used for that.

But the previous poster is wrong. You cannot make a thread-safe
reference count
using atomic increment/decrement, or POXIX mutex for that matter, to
modify the
reference count. There's the problem of the count going to zero before
you can
increment it. This happens to be a fairly active are of research which
you
will notice if you search on reference counting in
http://citeseer.nj.nec.com/cs/
Most of the solutions involve non-existent instructions or doing exotic
things with
memory pools.

Now I have a thread-safe reference count based smart pointer. The
prototype
uses POSIX mutexes for atomicity, but if a doubleword compare and swap
is
available, which seems to be the case for most 32 bit platforms, it can
be made both lock-free and wait-free.

So, in principle it can be done. Whether anyone wants it to be done
seems
to be another question though.

Joe Seigh

Joshua Lehrer

unread,
Nov 14, 2002, 8:19:28 AM11/14/02
to
ka...@gabi-soft.de (James Kanze) wrote in message
news:<d6651fb6.02111...@posting.google.com>...

> Of course, this is easily solved: we'll also assume a function which
can
> read atomically (and synchronized -- the atomicity is only part of the
> problem). While it would be nice if such primitives exist, they don't
> under Posix, and the only way to implement them, according to Posix,
is
> to use a mutex or some other synchronizing system call. (Some Posix
> systems, such as Solaris on Sparc or Linux on Intel, may have
additional
> possibilities, but from what I have been told, this is NOT the case
for
> such architectures as the Alpha and Merced. And I've yet to see an
> atomic increment or decrement without a Mutex which works under
Solaris
> on a Sparc.)

On my platform, ALPHAS, we have builtin operations

__ATOMIC_INCREMENT_LONG
and
__ATOMIC_DECREMENT_LONG

also, reads of 32 byte integral values are atomic.

Yes, this is NOT portable. But, given these restraints, as you have
demonstrated in this post, it is possible to write such a class.

joshua lehrer
factset research systems

[ Send an empty e-mail to c++-...@netlab.cs.rpi.edu for info ]

James Kanze

unread,
Nov 14, 2002, 8:24:45 AM11/14/02
to
"Andrei Alexandrescu" <andre...@hotmail.com> wrote in message
news:<aqres2$d1ur8$1...@ID-14036.news.dfncis.de>...

> "Mirek Fidler" <c...@centrum.cz> wrote in message
> news:aqjkbb$1sne$1...@news.vol.cz...

> > Anyway, I am acustomed to using things like

> > String LoadFile(const char *filename);

> > that loads content of file and returns it as string. Without
> > some form of optimization, creating two full 'deep copies' of string
> > would cost too much here.

> Of the possible optimizations, best is, of course, to use Mojo :oD.
> By the way, my draft article (http://moderncppdesign.com/mojo) makes
> the conjecture that many textbooks and many people use reference
> counting solely because it optimizes *one* copy (the return value),
> not because there are tons of strings with the same content lying
> around.

Murray did some actual measurements. The critical factor is the average
number of copies per string -- the one return value copy is a copy, just
like any other. I've never seen it claimed that this copy is in any way
different than any other copy. It might be mentionned, however, as a
place where you cannot easily avoid the copy -- you can pass a parameter
by const reference to avoid the copy, but if the result of a function is
a local variable or a temporary, returning by const reference is not a
solution. Using strings in containers (STL, of course, but also most
pre-STL containers) also involves copies.

--
James Kanze mailto:jka...@caicheuvreux.com
Conseils en informatique orientée objet/
Beratung in objektorientierter Datenverarbeitung

[ Send an empty e-mail to c++-...@netlab.cs.rpi.edu for info ]

James Kanze

unread,
Nov 15, 2002, 12:16:15 PM11/15/02
to
Joseph Seigh <jsei...@xemaps.com> wrote in message
news:<3DD2F394...@xemaps.com>...

> James Kanze wrote:
> > usene...@lehrerfamily.com (Joshua Lehrer) wrote in message
> > news:<31c49f0d.02111...@posting.google.com>...
> > > "Mirek Fidler" <c...@centrum.cz> wrote in message
> > > news:<aqhatf$cp9$1...@news.vol.cz>...
> > > > > What is true is that the standard interface requires the
> > > > > implementation to look at the reference count in a lot of
> > > > > functions where one might not expect it (althoug again, I
> > > > > don't think I make much use of those functions in my
> > > > > code). And in a multi-threaded environment, you must acquire a
> > > > > lock just to look at the count. That's expensive. More

> > > This is simply not true. As I have posted before, a reference
> > > counted string class can be written using only atomic_increment
> > > and atomic_decrement, and no other semaphors, locks, or mutexes.

> > This has often been claimed, but
> > - no one has yet shown an implementation which does so and is
conform,
> > and
> > - under Posix, at least, there is no atomic_increment nor
> > atomic_decrement, and the only way to implement these functions
uses
> > a mutex.

> You could implement it using hardware dependent assembly code. Most
> platforms have a compare and swap or something that can used for that.

Most, but not all. As far as I know, for example, Sparc's don't. And
while current Intel processors do, Intel has announced that this may not
be the case in the future.

The problem is that it isn't just a question of CPU. It is a question
of synchronizing ALL of the processor caches which may be concerned.
Thus, Intel currently guarantees that if an instruction is preceded by a
lock prefix will ensure cache consistency before and after the
instruction; it creates memory barriers around the instruction. And
since the hardware has an increment memory instruction, atomic increment
is easy (albeit slow).

I'm not sure how a compare and swap can be used for an atomic increment;
Sparc's have a swap instruction which makes the above guarantees, but
not an increment, at least that I know of. But I don't see off-hand how
a swap can be used to implement an increment. (The g++ library uses the
swap to implement a spin lock, and the spin lock to protect the
increment. Which would be fine, except that spin locks deadlock with
Posix threads, if a thread with a higher priority than the one holding
the lock attempts to acquire it.)

> But the previous poster is wrong. You cannot make a thread-safe
> reference count using atomic increment/decrement, or POXIX mutex for
> that matter, to modify the reference count. There's the problem of
> the count going to zero before you can increment it.

Whoever accesses the count, whether to modify it or just to read it,
must acquire the same lock. If you hold the lock, the count cannot go
to zero, because no one else can access it. And acquiring or releasing
a lock ensures cache consistency (according to Posix).

> This happens to be a fairly active are of research which you will
> notice if you search on reference counting in
> http://citeseer.nj.nec.com/cs/ Most of the solutions involve
> non-existent instructions or doing exotic things with memory pools.

Like the one proposed here:-).

Seriously, if all accesses to the reference count (including read
accesses) are protected by a Posix mutex, it should be safe. Ungodly
slow, but save.

> Now I have a thread-safe reference count based smart pointer. The
> prototype uses POSIX mutexes for atomicity, but if a doubleword
> compare and swap is available, which seems to be the case for most 32
> bit platforms, it can be made both lock-free and wait-free.

I'd be interested in seeing how. (We suppose, of course, that the
doubleword compare and swap also ensures cache consistency. Complete
cache consistency, and not just that of the words involved, because
otherwise, it is easy to show that it cannot work.)

And I've been told that such an instruction doesn't exist for some 64
bit platforms.

--
James Kanze mailto:jka...@caicheuvreux.com
Conseils en informatique orientée objet/
Beratung in objektorientierter Datenverarbeitung

[ Send an empty e-mail to c++-...@netlab.cs.rpi.edu for info ]

James Kanze

unread,
Nov 15, 2002, 12:16:34 PM11/15/02
to
Silicon Graphics <ze...@zell.best.vwh.net> wrote in message
news:<piEA9.133261$Yb1.1...@sea-read.news.verio.net>...

> James Kanze <ka...@gabi-soft.de> wrote:
> > usene...@lehrerfamily.com (Joshua Lehrer) wrote in message
> > news:<31c49f0d.02111...@posting.google.com>...
> >> "Mirek Fidler" <c...@centrum.cz> wrote in message
> >> news:<aqhatf$cp9$1...@news.vol.cz>...

> >> > > What is true is that the standard interface requires the
> >> > > implementation to look at the reference count in a lot of
> >> > > functions where one might not expect it (althoug again, I don't
> >> > > think I make much use of those functions in my code). And in a
> >> > > multi-threaded environment, you must acquire a lock just to
> >> > > look at the count. That's expensive. More

> >> This is simply not true. As I have posted before, a reference
> >> counted string class can be written using only atomic_increment and
> >> atomic_decrement, and no other semaphors, locks, or mutexes.

> > This has often been claimed, but
> > - no one has yet shown an implementation which does so and is
conform,
> > and
> > - under Posix, at least, there is no atomic_increment nor
> > atomic_decrement, and the only way to implement these functions
uses
> > a mutex.

> If you make the concession that all ref counted objects are immutable,
> the complexity level drops considerably.

Quite. Regretfully, the standard string class does not allow this.

> Whether that is useful for your environment is a different question.
> Obtaining the ref count as a rhs isn't that hard as long as you have
> the appropriate memory barrier primitives.

As long as... :-)

--
James Kanze mailto:jka...@caicheuvreux.com

Conseils en informatique orientée objet/
Beratung in objektorientierter Datenverarbeitung

[ Send an empty e-mail to c++-...@netlab.cs.rpi.edu for info ]

James Kanze

unread,
Nov 15, 2002, 12:18:23 PM11/15/02
to
usene...@lehrerfamily.com (Joshua Lehrer) wrote in message
news:<31c49f0d.02111...@posting.google.com>...
> ka...@gabi-soft.de (James Kanze) wrote in message
> news:<d6651fb6.02111...@posting.google.com>...

> > Of course, this is easily solved: we'll also assume a function which
> > can read atomically (and synchronized -- the atomicity is only part
> > of the problem). While it would be nice if such primitives exist,
> > they don't under Posix, and the only way to implement them,
> > according to Posix, is to use a mutex or some other synchronizing
> > system call. (Some Posix systems, such as Solaris on Sparc or Linux
> > on Intel, may have additional possibilities, but from what I have
> > been told, this is NOT the case for such architectures as the Alpha
> > and Merced. And I've yet to see an atomic increment or decrement
> > without a Mutex which works under Solaris on a Sparc.)

> On my platform, ALPHAS, we have builtin operations

> __ATOMIC_INCREMENT_LONG
> and
> __ATOMIC_DECREMENT_LONG

Do they ensure cache consistency as well?

> also, reads of 32 byte integral values are atomic.

I don't have access to the actual documentation for the Alpha, so I
can't be sure, but I have been informed that it is one of the platforms
where there wasn't cache write-through -- that it occasionally would
prefetch.

> Yes, this is NOT portable. But, given these restraints, as you have
> demonstrated in this post, it is possible to write such a class.

I'm not 100% convinced of anything for the moment. I know that
analysing the possibilities is extremely difficult. I've just been
going over the code in the library which comes with g++, for example; I
know that they *don't* generally protect the reads. For the moment,
I've found a case where they may make a copy when it isn't necessary,
but I've not found a sure case of error (supposing that their system
dependent "atomic" functions really do work atomically. And the
reasoning as to why they work is very subtle: I'd feel a lot better
about it if they had some sort of comments concerning the invariants,
which showed that they had really considered things like reading stale
data in the absense of a lock -- for the moment, all of the cases I've
encountered can only result in considering the string shared when it
isn't.

Note that for the user, there are some subtle issues to consider as
well. Normally, if no thread modifies the string (once it has been
created), it should not be necessary for accesses to be protected. In
the case of std::string, however (with a reference counted
implementation like that of g++), I think that the user needs protection
for any non-const function; there are at least subtle cases where he
might.

--
James Kanze mailto:jka...@caicheuvreux.com
Conseils en informatique orientée objet/
Beratung in objektorientierter Datenverarbeitung

[ Send an empty e-mail to c++-...@netlab.cs.rpi.edu for info ]

Joseph Seigh

unread,
Nov 15, 2002, 5:42:36 PM11/15/02
to

James Kanze wrote:
>
> Joseph Seigh <jsei...@xemaps.com> wrote in message
> news:<3DD2F394...@xemaps.com>...

....


> > You could implement it using hardware dependent assembly code. Most
> > platforms have a compare and swap or something that can used for
that.
>
> Most, but not all. As far as I know, for example, Sparc's don't. And
> while current Intel processors do, Intel has announced that this may
not
> be the case in the future.
>

Current sparcs do. And if Microsoft uses what they patent then Intel
may
want to keep them in also.

> The problem is that it isn't just a question of CPU. It is a question
> of synchronizing ALL of the processor caches which may be concerned.
> Thus, Intel currently guarantees that if an instruction is preceded by
a
> lock prefix will ensure cache consistency before and after the
> instruction; it creates memory barriers around the instruction. And
> since the hardware has an increment memory instruction, atomic
increment
> is easy (albeit slow).
>

You're confusing cache with memory access ordering. Most cache is
transparent
and you cannot tell it's there programatically, execpt for performance
effects.
Memory barriers are needed because the order of accesses by one
processor appear to another processor are not necessarily be the order
they
were performed in.

Atomic operations don't require memory barriers unless they're used in
conjunction with other memory operations. For instance, if you push
something onto a queue with compare and swap, you want memory barriers
not for queue integrity but because you want the memory operations that
created what you pushed onto the queue to complete so some other thread
won't see that object in an inconsistent state.

> I'm not sure how a compare and swap can be used for an atomic
increment;
> Sparc's have a swap instruction which makes the above guarantees, but
> not an increment, at least that I know of. But I don't see off-hand
how
> a swap can be used to implement an increment. (The g++ library uses
the
> swap to implement a spin lock, and the spin lock to protect the
> increment. Which would be fine, except that spin locks deadlock with
> Posix threads, if a thread with a higher priority than the one holding
> the lock attempts to acquire it.)

For increment, if the old/compare value is n, you make the swap value
n+1;


>
> > But the previous poster is wrong. You cannot make a thread-safe
> > reference count using atomic increment/decrement, or POXIX mutex for
> > that matter, to modify the reference count. There's the problem of
> > the count going to zero before you can increment it.
>
> Whoever accesses the count, whether to modify it or just to read it,
> must acquire the same lock. If you hold the lock, the count cannot go
> to zero, because no one else can access it. And acquiring or
releasing
> a lock ensures cache consistency (according to Posix).

Depends on where the lock is. If the lock is part of the object, then
you have an interval between the time the pointer is accessed and before
the time
the the lock is aquired and the reference count incremented where the
object
could be deleted, in which case you are in trouble.

You could use a global lock to protect that whole interval. Up to a
point that
might be fairly efficient. It wouldn't scale up though.
>
....


>
> Seriously, if all accesses to the reference count (including read
> accesses) are protected by a Posix mutex, it should be safe. Ungodly
> slow, but save.

Of course. Unsafe code always runs much faster.


>
> > Now I have a thread-safe reference count based smart pointer. The
> > prototype uses POSIX mutexes for atomicity, but if a doubleword
> > compare and swap is available, which seems to be the case for most
32
> > bit platforms, it can be made both lock-free and wait-free.
>
> I'd be interested in seeing how. (We suppose, of course, that the
> doubleword compare and swap also ensures cache consistency. Complete
> cache consistency, and not just that of the words involved, because
> otherwise, it is easy to show that it cannot work.)

I posted an early version to c.p.t. The basic logic works. I'm
just messing with c++ design issues. When I get a little more resolution
on those, I'll post it here.


>
> And I've been told that such an instruction doesn't exist for some 64
> bit platforms.
>

They may change their minds.

Joe Seigh

Silicon Graphics

unread,
Nov 16, 2002, 3:31:08 PM11/16/02
to
James Kanze <ka...@gabi-soft.de> wrote:
:> If you make the concession that all ref counted objects are immutable,

:> the complexity level drops considerably.

: Quite. Regretfully, the standard string class does not allow this.

For copy-on-write, I don't see a way out without either DCAS or a
mutex. Which, I assume, is the prevailing sentiment.

:> Whether that is useful for your environment is a different question.


:> Obtaining the ref count as a rhs isn't that hard as long as you have
:> the appropriate memory barrier primitives.

: As long as... :-)

Depending on how many platforms you need to support, this may be quite
viable. Implementing the appropriate assembly instructions for SPARC,
x86, Alpha, is relatively straightforward. Failing that, one could
fall back on a mutex. I believe RogueWave does this for their STL
implementation of std::string.

I am not sure where, outside of a garbage collector, code would encounter
a ref counted object with count 0. By definition, a count of 0 signals
no available references.

Assuming that objects are created with count == 1, and that all threads
increment the count if storing the object past the function call (and
performing the consequent decement when finished), what is the scenario
that this happens?

Adam
ze...@best.com

: --
: James Kanze mailto:jka...@caicheuvreux.com

[ Send an empty e-mail to c++-...@netlab.cs.rpi.edu for info ]

Alexander Terekhov

unread,
Nov 17, 2002, 10:19:32 AM11/17/02
to

Silicon Graphics wrote:
>
> James Kanze <ka...@gabi-soft.de> wrote:
> :> If you make the concession that all ref counted objects are
immutable,
> :> the complexity level drops considerably.
>
> : Quite. Regretfully, the standard string class does not allow this.
>
> For copy-on-write, I don't see a way out without either DCAS or a
> mutex.

<http://tinyurl.com/2r88> [c.l.c++.mod "Subject: Re: Ref-counted
strings not thread-safe -- looking for clarification"]

> Which, I assume, is the prevailing sentiment. ...

My "sentiment" is this: < Forward Inline >

-------- Original Message --------
Message-ID: <3DD68A0B...@web.de>
Date: Sat, 16 Nov 2002 19:10:19 +0100
From: Alexander Terekhov <tere...@web.de>
Newsgroups: comp.programming.threads
Subject: Re: std::string, reference counting and multithreading
References: <snip>

< the referenced article(s) can be found on c.l.c++.mod >

Joseph Seigh wrote:
[...]


> I posted an early version to c.p.t. The basic logic works. I'm
> just messing with c++ design issues. When I get a little more
resolution
> on those, I'll post it here.

C++ design issues aside, you are aiming at solving this:

http://www.hpl.hp.com/personal/Hans_Boehm/gc/example.html
("Expensive Explicit Deallocation: An Example". Note that even with GC
one would still need memory barriers to insure proper visibility
[Java's revised volatiles would work really good here], but GC makes
it possible to get rid of rather expensive locking/ref.counting.)

or similar problem(s) using DCAS/whatever (avoiding blocking, making
it "lock-free"/"non-blocking"[1]), I believe.

AFAICT, this isn't the problem that is commonly solved by "reference
counting". ``shared_ptr is supposed to be as thread-safe as an int.''

Well, OTOH, it would be really great if POSIX would provide something
along the lines of: <SKETCHv2>

Opaque "mutex-like" pthread_refcount_t [and pthread_refcount_attr_t]
type for reference counting. The only reference counting operations
that should "synchronize memory with respect to other threads" [4.10,
but "execution" aside -- <http://tinyurl.com/2jb9>, except destruction
of course] would be pthread_refcount_decrement_msync() and
pthread_refcount_sub_msync(). Note that pthread_refcount_increment()
and pthread_refcount_add() need *NOT* "synchronize memory with respect
to other threads". For immutable stuff (like COW), and for "double"
counting ala strong/weak counts, one could also use "optimized"
memory-sync-less pthread_refcount_decrement_nomsync() and
pthread_refcount_sub_nomsync() operations. The rest is as follows:

PTHREAD_REFCOUNT_MAX // upper bound
PTHREAD_REFCOUNT_INITIALIZER(N) // macro for statics INIT
pthread_refcount_init() // mutex like but with initial value
pthread_refcount_destroy() // mutex like apart from EBUSY
pthread_refcount_getvalue() // just in case someone will need it
pthread_refcount_setvalue() // without any sync whatsoever
pthread_refcount_attr_*() // PROCESS_SHARED and etc. attr-stuff
std::size_t as "value type" // for get/set/add/sub "value" param
PTHREAD_REFCOUNT_DROPPED_TO_ZERO // for pthread_refcount_decrement*()
// and pthread_refcount_sub() to
// indicate "dropped to zero" condition
// that MAY be used to safely destoy
// associated ref.counted object or
// simply continue -- increment/setval

regards,
alexander.

[1]
http://www.informatik.uni-trier.de/~ley/db/indices/a-tree/d/Detlefs:Davi
d.html
(Even Better DCAS-Based Concurrent Deques. DISC 2000: 59-73)

"A non-blocking implementation is one for which any history that has
invocations of some set O of higher-level operations but no associated
responses may contain any number of responses for high-level operations

concurrent with those in O. That is, even if some higher-level
operations
(each of which may be continuously taking steps, or not) never
complete,
other invoked operations may nevertheless continually complete. Thus
the
system as a whole can make progress; individual processors cannot be
blocked, only delayed, by other processors continuously taking steps or

failing to take steps. Using locks would violate the above condition,
hence the alternate name lock-free."

Joseph Seigh

unread,
Nov 17, 2002, 7:38:36 PM11/17/02