Replacement for MS STL?

[Garry W]

Anybody know of any really fresh and clean STL implementation out there?

I need something like the STL for a big new project that I've started in on.

And I've given up on the Microsoft implementation. When I tried to use it I
immediately fell over several bugs in it that took a lot of time to find. And
working with it has been really difficult because the code is kind of
illegible. At least for the uninitiated. (Apologies if there are authors
here!)

So I've been "rolling my own", as the need arises. I can tune to serve my
needs, and I have a (greater) prayer of figuring out the bugs and of
understanding those lovely VStudio template error messages. But rolling my
own, and making it right, is getting to be too much of a time-consuming
process too.

I did download StlPort back at the beginning of the problems. But I quickly
discovered that StlPort doesn't compile (yet) under VStudio 8 (2005). And
it's so heavily encrusted with preprocessor stuff that it's rather hard to
work with too. The coding has much of the same "encryption".

Anything else out there that's nice? Doesn't need to be free. Doesn't need to
follow the STL spec as long as it does many of the equivalent things. Does
there exist a "super sweet" STL (or equiv) for Visual Studio?

thanks much -

Garry

PS - If anyone is interested, I started building a "de-mystifier" for the MS
code, to translate the mysterious internal names over to ones that more
resemble English words. Helped me with the debugging.

[P.J. Plauger]

"Garry W" <List...@extremelyserious.org> wrote in message
news:3rg4h2lafjdds8hg9...@4ax.com...

> Anybody know of any really fresh and clean STL implementation out there?

Hasn't been a new one for years. Alexandrescu began YASLI (Yet Another
Standard Library Implementation) nearly five years ago. It's goal was
to redo the Standard C++ library properly, using what we had learned
about template metaprogramming, etc. You can read the entire history
of that project at:

http://www.stlport.com/cgi-bin/forum/dcboard.cgi

It peaked out at several postings a year [sic]. Hasn't been a new
one in 2006.

> I need something like the STL for a big new project that I've started in
> on.
>
> And I've given up on the Microsoft implementation. When I tried to use it
> I
> immediately fell over several bugs in it that took a lot of time to find.
> And
> working with it has been really difficult because the code is kind of
> illegible. At least for the uninitiated. (Apologies if there are authors
> here!)

He is.

> So I've been "rolling my own", as the need arises. I can tune to serve my
> needs, and I have a (greater) prayer of figuring out the bugs and of
> understanding those lovely VStudio template error messages. But rolling my
> own, and making it right, is getting to be too much of a time-consuming
> process too.

Perhaps you're beginning to understand why you have to suffer those
fools who write commercial software you have to use. BTW, did you
report any of those bugs you found, so they might get fixed?

> I did download StlPort back at the beginning of the problems. But I
> quickly
> discovered that StlPort doesn't compile (yet) under VStudio 8 (2005). And
> it's so heavily encrusted with preprocessor stuff that it's rather hard to
> work with too. The coding has much of the same "encryption".

STLport is also moribund. stlport.org gave up on it years ago,
leaving it to a couple of determined volunteers to keep it going.
See:

http://sourceforge.net/projects/stlport

for their ongoing efforts. We test it regularly, and found that it
hasn't improved in compliance (or correctness) for the past several
years.

> Anything else out there that's nice? Doesn't need to be free. Doesn't need
> to
> follow the STL spec as long as it does many of the equivalent things. Does
> there exist a "super sweet" STL (or equiv) for Visual Studio?

You might also check out:

http://incubator.apache.org/stdcxx

It's the old Rogue Wave library made open source and hosted by
Apache. And if you're a traditionalist, there's still:

http://www.sgi.com/tech/stl/

which is the original STL from H-P reworked by the very capable
Matt Austern while he was at SGI. Or even:

http://www.recursionsw.com/cplus.htm

Recursion Sowtware's STL Toolkit is the remains of the old
ObjectSpace implementation.

The best free implementation of STL these days comes with
libstdc++, which ships with gcc. The only problem is, you
have a nontrivial porting job to get it to work with VC++.

There's only one commercial supplier of Standard C and C++
libraries left today, and that's Dinkumware, Ltd. Our latest
product, the Dinkum Compleat Libraries, works out of the box
with V6 through V8 and provides the only full implementation
of TR1 (which will be a large chunk of the next Standard C++
library). But it's written by the same folks who brought
you the buggy, unreadable code you're trying to avoid, so
you might not want to try it.

> thanks much -
>
> Garry
>
> PS - If anyone is interested, I started building a "de-mystifier" for the
> MS
> code, to translate the mysterious internal names over to ones that more
> resemble English words. Helped me with the debugging.

Glad to hear it.

P.J. Plauger
Dinkumware, Ltd.
http://www.dinkumware.com

[Alex Blekhman]

Hi Garry,

It will be quite an arrogance to defend Dinkum's C++ Library
after the author himself put his weight already. Anyway, see
comments inline.

"Garry W" wrote:
> Anybody know of any really fresh and clean STL
> implementation out there?
>
> I need something like the STL for a big new project that
> I've started in on.
>
> And I've given up on the Microsoft implementation. When I
> tried to use it I
> immediately fell over several bugs in it that took a lot
> of time to find. And
> working with it has been really difficult because the code
> is kind of
> illegible. At least for the uninitiated. (Apologies if
> there are authors
> here!)

In my opinion, amount of bugs in MS C++ Library (which is
made by Dinkumware) is at least no greater than in any other
implementaion just due to simple fact that MS Lib is the
most widely used implementation in the world today. With
such huge base of users you find trivial bugs immediately
and nontrivial ones quite quickly. Prevalence of the library
is great advantage of it.

The code style is unreadable only at first glance. While
wandering there you'll get used to it rather quickly. The
good news that the code is really polished over generations.
It's by no means old project's hairy horror code. After
accustoming myself with Library's coding style I find
debugging within it quite pleasant (comparing to other code
that I have to deal with). Also, VC8 debugger's ability to
unroll containers and show actual types makes debugging
easier. Remember that once you adopt other library you will
need to write containter expansion rules (in `autoexp.dat')
from scratch.

> So I've been "rolling my own", as the need arises. I can
> tune to serve my
> needs, and I have a (greater) prayer of figuring out the
> bugs and of
> understanding those lovely VStudio template error
> messages. But rolling my
> own, and making it right, is getting to be too much of a
> time-consuming
> process too.

Don't you feel it like reinventing the wheel while others
drive cars already? It can take several man-years to develop
anything decent that will be able to compete with existsing
libraries.

> I did download StlPort back at the beginning of the
> problems. But I quickly
> discovered that StlPort doesn't compile (yet) under
> VStudio 8 (2005). And
> it's so heavily encrusted with preprocessor stuff that
> it's rather hard to
> work with too. The coding has much of the same
> "encryption".
>
> Anything else out there that's nice? Doesn't need to be
> free. Doesn't need to
> follow the STL spec as long as it does many of the
> equivalent things. Does
> there exist a "super sweet" STL (or equiv) for Visual
> Studio?

"Nice" is vague definition. There are many project, which
based on MFC/ATL and use its built in collection classes for
their humble needs and happy with that. Regarding other
generic libraries. I used once RogueWave C++ library (which
constantly changes names). Now it called SourcePro C++
(http://www.roguewave.com/products/sourcepro/). It's decent
rich library with many goodies already built in (threading,
XML, DB connectivity, etc..). However, it's not free.
Another nice library is ACE
(http://www.cs.wustl.edu/~schmidt/ACE.html). If you have to
do heavy networking and thread lifting, then it definitely
worth a look.

HTH
Alex

[Stephen Howe]

> And I've given up on the Microsoft implementation. When I tried to use it
I
> immediately fell over several bugs in it that took a lot of time to find.

Do you mean in VS2005 (since you mention it below)?
And what bugs are they?

I have found that quite a few so-called "bugs" turn out to be because
programmers forget the rules that the STL imposes, like iterator validity.
It is all in the ISO C++ standard.

In terms of bugs in the STL, there were quite a few in VS 5.0/6.0, some
caused because the standard was not set in stone at that point in time.
Dinkumware have some downloadable patches for this.
There was a bug in vector<bool> in 7.0 IIRC.

> So I've been "rolling my own", as the need arises. I can tune to serve my
> needs, and I have a (greater) prayer of figuring out the bugs and of
> understanding those lovely VStudio template error messages. But rolling my
> own, and making it right, is getting to be too much of a time-consuming
> process too.

Right. I would re-think your descision to give up on MS STL.
It is likely to be more correct than your "rolling my own" (but I am not
disparaging your efforts - it is a noble task).
I have used the MS STL for the past 7 years and in general it does look
solid quality both from in-use and lookng under the hood. I am not claiming
it is bug-free. But if you posted your "bugs" here, I am sure the authors
can provide fixes or workarounds for any genuine bugs you find.

> PS - If anyone is interested, I started building a "de-mystifier" for the
MS
> code, to translate the mysterious internal names over to ones that more
> resemble English words. Helped me with the debugging.

Not necessary. A good programmer can work out what a variable does by
context even if they were called u0001, u0002, ... Never found this a
problem.

Cheers

Stephen Howe

[Ulrich Eckhardt]

Garry W wrote:
> And I've given up on the Microsoft implementation. When I tried to use it
> I immediately fell over several bugs in it that took a lot of time to
> find.

Hmmm, I'm only aware of one bug, though that one was a real showstopper for
me. And no, I'm not referring to the crippled variant MS ships for CE,
which was another point.

> And working with it has been really difficult because the code is
> kind of illegible. At least for the uninitiated. (Apologies if there are
> authors here!)

I think it's a lack of understanding on your side. The point is that I must
be able to do this:

#define i 7*6
#define x 23,
#include <vector>

and still have a working vector afterwards. That is why that code, except
for the interfaces required by the standard, uses names reserved to the
compiler and standardlibrary implementation like __foo and _Bar.

> I did download StlPort back at the beginning of the problems. But I
> quickly discovered that StlPort doesn't compile (yet) under VStudio 8
> (2005).

When were the beginnings of your problems? I'm using STLport (not StlPort,
please!) with VC8 shortly after I started migrating to VC8, around half a
year back or so. Also, STLport people tend to respond to requests and
bugreports (I'm one of them, FYI), so you don't have to do everything on
your own.

> And it's so heavily encrusted with preprocessor stuff that it's
> rather hard to work with too. The coding has much of the same
> "encryption".

The preprocessor stuff is tough but necessary for portability.
The "encryption" is also necessary, as explained above.

> Anything else out there that's nice? Doesn't need to be free. Doesn't need
> to follow the STL spec as long as it does many of the equivalent things.
> Does there exist a "super sweet" STL (or equiv) for Visual Studio?

One last thing: the term 'STL' is typically used to refer to the
container/iterator/algorithm parts of the standardlibrary, because those
parts have their origin in the STL (the real, original one) which is now
dead. Otherwise, the term STL doesn't describe anything, you should rather
say C++ standardlibrary if you mean that.

cheers

Uli

[peter.ko...@gmail.com]

Garry W wrote:
> Anybody know of any really fresh and clean STL implementation out there?
>
> I need something like the STL for a big new project that I've started in on.
>
> And I've given up on the Microsoft implementation. When I tried to use it I
> immediately fell over several bugs in it that took a lot of time to find. And
> working with it has been really difficult because the code is kind of
> illegible. At least for the uninitiated. (Apologies if there are authors
> here!)

[snip]

This comes as a big surprise for me. I use the standard library in
Visual Studio 2005 ("STL") and havent found any bugs. Most certainly
the problem lies in your understandig of the library. Do feel free to
post examples here if you believe I'm wrong.

/Peter

[Garry W]

"P.J. Plauger" <p...@dinkumware.com> wrote:
>>(Apologies if there are authors here!)
>He is.

Ah! Pleased to meet you. And *thank you* for a very helpful post. I did not
know that StlPort was dying/dead. I did not know about the Apache and the
new/newer SGI implementations -- both look quite readable at first
inspection, although the Apache is pretty "frugal" with the comment lines.

At this point I'm about an hour into trying to get one or the other to
compile under VisualStudio. I'll keep working on it. (I would be most happy
to switch to the compiler that the whole open-source world seems to be using
- if only Gnu C++ did have a decent debugger and IDE.)

>The best free implementation of STL these days comes with
>libstdc++, which ships with gcc. The only problem is, you
>have a nontrivial porting job to get it to work with VC++.

I =noticed=. I was retired from the software business for about 12 years
before deciding, this summer, to go back on "active duty". I love C++ but
I've been noticing that using it for real projects can be just as hairy now
as it was years ago. (If not more so, as template usage gets fancier and
fancier.) In the code I see, the preprocessor is still very much in evidence
despite the greater abilities of the "real" language. Sigh.

(I'm thinking that for this project I might need to use other languages
(e.g., Python) as much as I can, even though they're not as "beautiful" as
C++. :)

>There's only one commercial supplier of Standard C and C++
>libraries left today, and that's Dinkumware, Ltd.

Didn't know there was a commercial version. I haven't figured out yet whether
my project will be open-source or not -- i.e., I don't know yet whether I can
go commercial. I need to dig into everything a little deeper yet.

>But it's written by the same folks who brought
>you the buggy, unreadable code you're trying to avoid, so
>you might not want to try it.

Ah...... I will go ahead and mention a couple of the bugs, and you can tell
me who was responsible.:) But, on the off-chance that they really were
yours, maybe we should do that offline?

> BTW, did you report any of those bugs you found, so they might get fixed?

Nope. I just searched (again) for a bug-submission process on the MS website.
The best I could find was the line "If you think you have found a bug in one
of Microsoft's products, please call 1-800-MICROSOFT". Plus numerous mentions
on various pages of what to do if you're a paid-support customer.

I don't have paid support. Does Microsoft want to hear about my bugs? Is
there a link hiding somewhere on that giant website?

And... when you do report a bug as an unpaid customer (assuming it's
possible), do you get a polite response and followup back from Microsoft?
Back in the old days my unpaid bug reports would simply disappear silently,
never to be heard from again. It's wasn't very encouraging.

Thank you again for the help about the alternative STLs.

Garry

[Carl Daniel [VC++ MVP]]

Garry W wrote:
> Nope. I just searched (again) for a bug-submission process on the MS
> website.

http://connect.microsoft.com/feedback/default.aspx?SiteID=210

>
> I don't have paid support. Does Microsoft want to hear about my bugs?

Yes, they do. Bug reports that you enter into the feedback site go directly
to the developers. The people that read the bug reports have no way of
knowing who's a paid support customer and who isn't. Basically, paid
support just lets you talk to the paid support people without a credit card.
Bug reports have their own path (more direct than going through support, in
some cases).

> Is there a link hiding somewhere on that giant website?

It's a bit hard to find, yes. Microsoft is moving towards having all
feedback across the board go through Connect (see URL above), so it should
be easier to find in the future.

>
> And... when you do report a bug as an unpaid customer (assuming it's
> possible), do you get a polite response and followup back from
> Microsoft? Back in the old days my unpaid bug reports would simply
> disappear silently, never to be heard from again. It's wasn't very
> encouraging.
>

You can monitor the status/progress of your bug reports on the web site.
It's not uncommon for one of the devs to ask a question of the poster, in
which case you'll also get email.

-cd

[P.J. Plauger]

"Garry W" <List...@extremelyserious.org> wrote in message

news:mc0bh2thi3tb1d77m...@4ax.com...

>>There's only one commercial supplier of Standard C and C++
>>libraries left today, and that's Dinkumware, Ltd.
>
> Didn't know there was a commercial version. I haven't figured out yet
> whether
> my project will be open-source or not -- i.e., I don't know yet whether I
> can
> go commercial. I need to dig into everything a little deeper yet.

Fine. One of the reasons we tell everybody about our free
competition (which we do at our web site too) is to shorten
the time that potential customers take to get back to us.
Not everybody can afford "free".

>>But it's written by the same folks who brought
>>you the buggy, unreadable code you're trying to avoid, so
>>you might not want to try it.
>
> Ah...... I will go ahead and mention a couple of the bugs, and you can
> tell
> me who was responsible.:) But, on the off-chance that they really were
> yours, maybe we should do that offline?

I don't care who was responsible, I just want to know about any
potential bugs so they can get fixed. Feel free to post them
on this forum, for maximum exposure. Ofttimes the MVPs answer
faster and better than I can, because of their broad experience.
But in any case, I assure you that any real bugs will get the
attention they deserve, and will be put on track to get fixed
in a future SP or release by Microsoft.

[Ulrich Eckhardt]

P.J. Plauger wrote:
> STLport is also moribund. stlport.org gave up on it years ago,
> leaving it to a couple of determined volunteers to keep it going.
> See:
>
> http://sourceforge.net/projects/stlport
>
> for their ongoing efforts. We test it regularly, and found that it
> hasn't improved in compliance (or correctness) for the past several
> years.

Hmmm, I did fix a bug in it, so either your testing methods are flawed or I
did introduce another bug by doing so....how about a concrete example?

Seriously, indeed STLport lay dormant for some time, but calling it dead
(which it isn't) or claiming that it didn't improve at all (which it did)
is downright respectless to the people that are actively putting efforts
into it. Just the fact that you can compile it out of the box with several
modern compilers already shows that this project is far from dead.

Uli
(Who is one of those developers, in case anybody wonders.)

[Tom Widmer [VC++ MVP]]

Ulrich Eckhardt wrote:
> P.J. Plauger wrote:
>> STLport is also moribund. stlport.org gave up on it years ago,
>> leaving it to a couple of determined volunteers to keep it going.
>> See:
>>
>> http://sourceforge.net/projects/stlport
>>
>> for their ongoing efforts. We test it regularly, and found that it
>> hasn't improved in compliance (or correctness) for the past several
>> years.
>
> Hmmm, I did fix a bug in it, so either your testing methods are flawed or I
> did introduce another bug by doing so....how about a concrete example?

Dinkumware used to sell their Proofer, though I don't see it on their
site now. You could contact them about the possibility of licencing it
to test STLport (if you have any money available to the project).

It's certainly the case that in the past the proofer can give a lot of
false negatives, due to each test being reliant on multiple features (as
I recall from the compliance roundup in cuj a few years back -
http://web.archive.org/web/20050218201324/www.cuj.com/documents/s=8193/cuj0104sutter/).
If you had a particular header missing, that might mean that a large
proportion of tests fail, despite the fact that the header in question
only applies directly to a small number of tests.

Tom

[P.J. Plauger]

"Ulrich Eckhardt" <eckh...@satorlaser.com> wrote in message
news:puqku3-...@satorlaser.homedns.org...

> P.J. Plauger wrote:
>> STLport is also moribund. stlport.org gave up on it years ago,
>> leaving it to a couple of determined volunteers to keep it going.
>> See:
>>
>> http://sourceforge.net/projects/stlport
>>
>> for their ongoing efforts. We test it regularly, and found that it
>> hasn't improved in compliance (or correctness) for the past several
>> years.
>
> Hmmm, I did fix a bug in it, so either your testing methods are flawed or
> I
> did introduce another bug by doing so....how about a concrete example?

The percentage count has stayed essentially unchanged for years.
A few fixes here and there won't necessarily be significant
among thousands of tests. If you license the source code of our
library you get access to our Quick Proofer, which will tell
you quite a bit about what's wrong with STLport; but we don't
give away test results, particularly to nominal competitors.

> Seriously, indeed STLport lay dormant for some time, but calling it dead
> (which it isn't)

I said moribund, which is not quite the same as dead. See below.

> or claiming that it didn't improve at all (which it did)

I said it hasn't improved in compliance, and I merely implied
"measurably". It's clear that work has been done on tracking
gcc and VC++ releases for packaging; it's less clear that
any serious effort has been made to address conformance
issues.

> is downright respectless to the people that are actively putting efforts
> into it. Just the fact that you can compile it out of the box with several
> modern compilers already shows that this project is far from dead.

I mean no disrespect for the people involved, but I have rather
less respect for the overall system. Go to stlport.org and it'll
tell you that the "Current release version" is 4.6.2, dating from
April 2004. That will *not* work out of the box on quite a few
current compilers. Go to the support forum at that web site and it'll
tell you that it is now in "read-only mode", and has been for much
of the past year. It steers you to sourceforge for "new discussions".

Ask for help there with 4.6.2 and you'll be told in no uncertain
terms that it's obsolete. You're advised instead to get the latest
sourceforge version, called 5.1 RC2, and then you can read about
the problems getting it to work with various systems. New threads
start at the rate of less than one per day, and that't *not*
because it's flawless. While I give full marks to Dumont and
Ovtchenkov for trying to organize new releases, it's pretty clear
that they lack both the time and some of the skills needed to
properly maintain a package of this nature.

Boris Fomitchev benefitted tremendously from the excellent work
done by Matt Austern while at SGI. Fomitchev started STLport with
a nearly complete Standard C++ library acquired at no cost. But
Austern has moved on, and so obviously has Fomitchev. With all
due respect to you and other contributors, the rate of improvement
has fallen way off, and the reasons for choosing STLport over
most other libraries have largely gone away.

From dictionary.com:

moribund \MOR-uh-bund\, adjective:

1. In a dying state; dying; at the point of death.
2. Becoming obsolete or inactive.

I meant what I said and I said what I meant.

[P.J. Plauger]

"Tom Widmer [VC++ MVP]" <tom_u...@hotmail.com> wrote in message
news:uu8RJLK4...@TK2MSFTNGP05.phx.gbl...

> Ulrich Eckhardt wrote:
>> P.J. Plauger wrote:
>>> STLport is also moribund. stlport.org gave up on it years ago,
>>> leaving it to a couple of determined volunteers to keep it going.
>>> See:
>>>
>>> http://sourceforge.net/projects/stlport
>>>
>>> for their ongoing efforts. We test it regularly, and found that it
>>> hasn't improved in compliance (or correctness) for the past several
>>> years.
>>
>> Hmmm, I did fix a bug in it, so either your testing methods are flawed or
>> I
>> did introduce another bug by doing so....how about a concrete example?
>
> Dinkumware used to sell their Proofer, though I don't see it on their site
> now. You could contact them about the possibility of licencing it to test
> STLport (if you have any money available to the project).

We license our Proofer only to our OEM customers now. We include
our Quick Proofer, which is broader but not as deep as the Proofer,
along with our source code these days.

> It's certainly the case that in the past the proofer can give a lot of
> false negatives, due to each test being reliant on multiple features (as I
> recall from the compliance roundup in cuj a few years back -
> http://web.archive.org/web/20050218201324/www.cuj.com/documents/s=8193/cuj0104sutter/).

Nope. We peel the tests apart to avoid having one compile failure
take down unrelated tests.

> If you had a particular header missing, that might mean that a large
> proportion of tests fail, despite the fact that the header in question
> only applies directly to a small number of tests.

All the headers are there, they just don't do everything they
should.

[Garry W]

Pardon the pause in replying. It was one of those weeks where the code
started flowing =really= well...

Here's one of the STL bugs things I found. Rather inelegantly and rapidly
written up.

#1: _HAS_ITERATOR_DEBUGGING.

(This was discussed here recently, but only after I had to find it on my
own.)

Background: Because I'm dealing with a huge data collection, I had been
compiling most of my program "Release", for needed speed, while compiling
only the bit that I'm actively working on as "Debug". Fair enough. But, as
recently discussed here, the slimly-documented switch _HAS_ITERATOR_DEBUGGING
- and, in particular, the completely-undocumented changing memory layout that
depends on _DEBUG/NDEBUG - can cause all he.. to break loose when you mix
modes. Because the switch triggers different, incompatible, memory layouts of
STL data.

By "undocumented" (it could be argued that the lack of documentation is the
real bug here) I mean:

a) The switch, and the feature, are virtually unmentioned in the
documentation that shipped with my compiler (VS Standard 2005). Specifically,
_HAS_ITERATOR_DEBUGGING comes up "not found" in when I try F1 help, "Search"
help, and "Index" help. After digging for a while, to prepare this bug
report, I did eventually find a link to a page that mentioned it.

b) The switch =is= mentioned on the Microsoft website - when you Google for
it - but even there the data-structure-altering behaviour is never mentioned
and the incompatibility between _DEBUG and NDEBUG builds is never mentioned.

c) Nor does Microsoft provide much general documentation of all the things a
user might expect to be different with _DEBUG/NDEBUG... I noticed in a simple
grep of the MS STL that there are =numerous= other mentions of "#ifdef
_DEBUG". I did not go through to figure out which items were implemented
safely and which are unsafe. Here's the documentation I found about
_DEBUG/NDEBUG and the RTL in the shipped "help". This is under "CRT Debug
Library Use".

The C run-time library provides extensive debugging support.
To use one of the CRT debug libraries, you must link with
/DEBUG and compile with /MDd, /MTd, or /LDd.

Remarks
The main definitions and macros for CRT debugging can be found
in the CRTDBG.h header file.

The functions in the CRT debug libraries are compiled with
debug information (/Z7, /Zd, /Zi, /ZI (Debug Information Format))
and without optimization. Some functions contain assertions to
verify parameters that are passed to them, and source code is
provided. With this source code, you can step into CRT
functions to confirm that the functions are working as you
expect and check for bad parameters or memory states.

Nothing in those words would lead me to suspect that _DEBUG and NDEBUG
template-library code might be unmixable. Nothing at all is mentioned about
C++ or the C++ STL.

My workaround is now to completely delete the _DEBUG and NDEBUG symbols from
my projects, while hoping that there are not any other ways for the MS
headers to behave differently in "Debug" and "Release" modes.

I love the existence of iterator debugging. Nice idea - the default iterators
lend themselves to mistakes. My suggested fix would simply be to have the
user explicitly turn the feature on and off, rather than doing it by
implication.

Garry

PS - There have been rumors that the MS "_DEBUG" C++ operator-new/delete is
incompatible with the MS "NDEBUG" C++ operator-new/delete. Are the rumors
true?

(It would be simple to make the debug/release heap calls mutually mixable and
compatible (at a "normal case" cost of, hmmmmm.... one memory 'read' and
three Intel instructions) but I do not know whether Microsoft has chosen to
do that.)

[Garry W]

Here's another STL bug I found -

#2: Instance of illegal "pointer" usage (or non-usage).

I am doing something unusual: I'm providing my own definitions for
allocator::pointer and allocator::const_pointer. (I have a persistent - i.e,
memory-mapped - database to store that "huge" amount of data I have. The
persistent database requires the unusual pointers.) Doing something unusual
always provides excellent opportunities for interesting bugs.

Specifically -

In the file "xstring" near line 2045 occurs this definition of std::string
character storage:

union _Bxty
{ // storage for small buffer or pointer to larger one
_Elem _Buf[_BUF_SIZE];
_Elem * _Ptr;
} _Bx;

First error is that the intrinsic-pointer usage "_Elem *" is obviously wrong
- it has to be rebind<_Elem>::pointer or something. There shouldn't be any
naked pointers to elements in stored STL data.

More subtle error is that even with that change the program would still be
wrong: the arbitrary type "pointer" =may= have constructors (or destructors
or assignment ops.) Mine do. And types that have any of those things are
illegal within C++ unions.

To achieve the equivalent effect of constructor-in-union, you might instead
do a bit of fancy footwork with alignment guarantees, manual
construction/destruction at the proper moments, and accessors that do
reference casts. At least, that's what I did in my quick work-around, except
for putting in the alignment guarantee (my own pointers didn't need it.)

Hope this helps.

Garry

[Garry W]

Ulrich Eckhardt <eckh...@satorlaser.com> wrote:
>I think it's a lack of understanding on your side. The point is that I must
>be able to do this:
>
> #define i 7*6
> #define x 23,
> #include <vector>
>
>and still have a working vector afterwards. That is why that code, except
>for the interfaces required by the standard, uses names reserved to the
>compiler and standardlibrary implementation like __foo and _Bar.

Well, the MS STL -almost- always remembers to put the _'s in there. :)

Here's the beginning of the MS STL "de-obfuscator" that I had, by necessity,
started working on. Old internal names are on the left, replacement ones are
on the right.

_Tptr _ValuePointerType
_Ctptr _ConstValuePointerType
_Reft _ValueReferenceType
_Dift _ValueDifferenceType
_Ax _AllocatorTypeArgument
_Mfl _EqualKeysAllowedArgument
_Multi _EqualKeysAllowed
_Elem _BaseElement
_Alloc _BaseValueAllocator
_Myt _MyType
_Ptr _pointer
_Num _count
_Mysize _current_count
_Bxty _BaseTypeStorageUnionTag
_Bx _BaseTypeStorage
_Eos _SetLengthAndNullTerminator
_Myres _reserved_space
_Newres _new_reserved_space
_Alty _value_allocator
_Alval _AllocatorValue
_Kty _KeyTypeArgument
_Ty _ValueTypeArgument
_Pr _ComparatorArgument
comp _comparator_instance
..., etc, & so forth

With a real understanding of what was going I could do better -- a lot of the
above substitutions (_Ptr, _Num, ...) were just place-holders until I could
figure out what the original names =really= meant.

But that gives you an idea what I'd like to see at those moments when I have
run the debugger through the MS STL.

(Yes, there are (rare) times when short names are plainer in the long run --
if you're the code maintainer. But when I'm a mere passer-by, wandering
through someone else's wild things, I'd prefer to have all the help that I
can get.)

yours,
Garry

[Carl Daniel [VC++ MVP]]

Garry W wrote:
> Pardon the pause in replying. It was one of those weeks where the code
> started flowing =really= well...
>
> Here's one of the STL bugs things I found. Rather inelegantly and
> rapidly written up.
>
> #1: _HAS_ITERATOR_DEBUGGING.
>
> (This was discussed here recently, but only after I had to find it on
> my own.)

Yeah, it's subtle. The (mostly, if not completely) undocumented truth is:
You can't reliably mix Debug and Release in a single build. There are too
many things that are simply undefined. In any case, this can't really be
blamed on the STL implementation.

> PS - There have been rumors that the MS "_DEBUG" C++
> operator-new/delete is incompatible with the MS "NDEBUG" C++
> operator-new/delete. Are the rumors true?

If you're talking about a single image (i.e. the result of a single
invocation of the linker), then there aren't any issues, because your image
will be linked with either the debug heap OR the release heap. Exactly
which one will depend on a bunch of things not worth going into here
(suffice it to say, you should never do this on purpose unless you really
know what you're doing).

If you're talking about multiple images, e.g. two different DLLs, then you
do indeed have an incompatibility. If you link two DLLs, both with the DLL
runtime library, but one's a release build and the other is a debug build,
then an object allocated in one of the DLLs can't be freed in the other (and
vice-versa). This is because you're now running a program with two
completely separate implementations of the runtime library in memory. It's
no different in principle than allocating something with CoTaskMemAlloc and
then freeing it with free() - they're simply two different heaps that can't
be mixed.

-cd

[P.J. Plauger]

"Garry W" <List...@extremelyserious.org> wrote in message

news:0ef0i2hs7apgsec86...@4ax.com...

> Pardon the pause in replying. It was one of those weeks where the code
> started flowing =really= well...
>
> Here's one of the STL bugs things I found. Rather inelegantly and rapidly
> written up.
>
> #1: _HAS_ITERATOR_DEBUGGING.
>
> (This was discussed here recently, but only after I had to find it on my
> own.)
>
> Background: Because I'm dealing with a huge data collection, I had been
> compiling most of my program "Release", for needed speed, while compiling
> only the bit that I'm actively working on as "Debug". Fair enough. But,
> as
> recently discussed here, the slimly-documented switch
> _HAS_ITERATOR_DEBUGGING
> - and, in particular, the completely-undocumented changing memory layout
> that
> depends on _DEBUG/NDEBUG - can cause all he.. to break loose when you mix
> modes. Because the switch triggers different, incompatible, memory layouts
> of
> STL data.
>
> By "undocumented" (it could be argued that the lack of documentation is
> the
> real bug here)

Then it's not a bug. There are *oodles* of macros and names beginning
with an underscore and a capital letter in the Standard C++ library.
Clause 17 of the C++ Standard tells you that these are part of the
implementation. Attempting to fiddle with such names, or second guess
how you might change them, is a dangerous and unsupported exercise.

> .....

> Nothing in those words would lead me to suspect that _DEBUG and NDEBUG
> template-library code might be unmixable. Nothing at all is mentioned
> about
> C++ or the C++ STL.

Well, they are. Perhaps Microsoft's top-level documentation might
make such things clearer for beginners, but I'm astounded that a
putative expert would think it wise to mix libraries compiled with
different settings.

> My workaround is now to completely delete the _DEBUG and NDEBUG symbols
> from
> my projects, while hoping that there are not any other ways for the MS
> headers to behave differently in "Debug" and "Release" modes.
>
> I love the existence of iterator debugging. Nice idea - the default
> iterators
> lend themselves to mistakes. My suggested fix would simply be to have the
> user explicitly turn the feature on and off, rather than doing it by
> implication.

There are compiler switches for doing just that.

> Garry
>
> PS - There have been rumors that the MS "_DEBUG" C++ operator-new/delete
> is
> incompatible with the MS "NDEBUG" C++ operator-new/delete. Are the rumors
> true?

Dunno. That's not my diocese.

> (It would be simple to make the debug/release heap calls mutually mixable
> and
> compatible (at a "normal case" cost of, hmmmmm.... one memory 'read' and
> three Intel instructions) but I do not know whether Microsoft has chosen
> to
> do that.)

It ain't that simple.

[P.J. Plauger]

"Garry W" <List...@extremelyserious.org> wrote in message

news:ioh0i2h2nsh8l2ge4...@4ax.com...

> Here's another STL bug I found -
>
> #2: Instance of illegal "pointer" usage (or non-usage).
>
> I am doing something unusual: I'm providing my own definitions for
> allocator::pointer and allocator::const_pointer. (I have a persistent -
> i.e,
> memory-mapped - database to store that "huge" amount of data I have. The
> persistent database requires the unusual pointers.) Doing something
> unusual
> always provides excellent opportunities for interesting bugs.
>
> Specifically -
>
> In the file "xstring" near line 2045 occurs this definition of std::string
> character storage:
>
> union _Bxty
> { // storage for small buffer or pointer to larger one
> _Elem _Buf[_BUF_SIZE];
> _Elem * _Ptr;
> } _Bx;
>
> First error is that the intrinsic-pointer usage "_Elem *" is obviously
> wrong
> - it has to be rebind<_Elem>::pointer or something. There shouldn't be any
> naked pointers to elements in stored STL data.

No, it doesn't. Template class basic_string can be instantiated only
on PODs -- see Clause 21 of the C++ Standard. That precludes any
fancy tricks with redefining pointers or addressing. Note that template
class vector does not use the small-string optimization, because it
lacks this restriction.

> More subtle error is that even with that change the program would still be
> wrong: the arbitrary type "pointer" =may= have constructors (or
> destructors
> or assignment ops.) Mine do. And types that have any of those things are
> illegal within C++ unions.

Then your program is ill formed. Not a bug in the implementation.

> To achieve the equivalent effect of constructor-in-union, you might
> instead
> do a bit of fancy footwork with alignment guarantees, manual
> construction/destruction at the proper moments, and accessors that do
> reference casts. At least, that's what I did in my quick work-around,
> except
> for putting in the alignment guarantee (my own pointers didn't need it.)
>
> Hope this helps.

It does.

[P.J. Plauger]

"Garry W" <List...@extremelyserious.org> wrote in message

news:gbi0i2h5qksje13ij...@4ax.com...

> Ulrich Eckhardt <eckh...@satorlaser.com> wrote:
>>I think it's a lack of understanding on your side. The point is that I
>>must
>>be able to do this:
>>
>> #define i 7*6
>> #define x 23,
>> #include <vector>
>>
>>and still have a working vector afterwards. That is why that code, except
>>for the interfaces required by the standard, uses names reserved to the
>>compiler and standardlibrary implementation like __foo and _Bar.
>
> Well, the MS STL -almost- always remembers to put the _'s in there. :)

Nope, it *always* does. The only names you'll find that aren't
"obfuscated" (defined in the implementer's name space) are the
ones mandated by the C++ Standard.

> Here's the beginning of the MS STL "de-obfuscator" that I had, by
> necessity,
> started working on. Old internal names are on the left, replacement ones
> are
> on the right.

> .....

So far, you've reported:

1) that you don't like the documented way that debugging gets
turned on and off, so you experimented at length and got in
some trouble

2) that you tried to do something with strings that's documented
as not supported, and you got in some trouble

3) that you *think* the names internal to the library are
inconsistent, and you *feel* they are in bad style

But you began this thread by saying:

: And I've given up on the Microsoft implementation. When I tried to use it

I
: immediately fell over several bugs in it that took a lot of time to find.

And
: working with it has been really difficult because the code is kind of
: illegible. At least for the uninitiated. (Apologies if there are authors
: here!)

So far you are zero for three, and seem to have trouble
reading documentation. But if you have other bugs to report,
I'm still very interested in hearing about them.

[Tom Widmer [VC++ MVP]]

Garry W wrote:
> Here's another STL bug I found -
>
> #2: Instance of illegal "pointer" usage (or non-usage).
>
> I am doing something unusual: I'm providing my own definitions for
> allocator::pointer and allocator::const_pointer. (I have a persistent - i.e,
> memory-mapped - database to store that "huge" amount of data I have. The
> persistent database requires the unusual pointers.) Doing something unusual
> always provides excellent opportunities for interesting bugs.
>
> Specifically -
>
> In the file "xstring" near line 2045 occurs this definition of std::string
> character storage:
>
> union _Bxty
> { // storage for small buffer or pointer to larger one
> _Elem _Buf[_BUF_SIZE];
> _Elem * _Ptr;
> } _Bx;
>
> First error is that the intrinsic-pointer usage "_Elem *" is obviously wrong
> - it has to be rebind<_Elem>::pointer or something. There shouldn't be any
> naked pointers to elements in stored STL data.
>
> More subtle error is that even with that change the program would still be
> wrong: the arbitrary type "pointer" =may= have constructors (or destructors
> or assignment ops.) Mine do. And types that have any of those things are
> illegal within C++ unions.

The implementation is allowed to assume that allocator::pointer ==
allocator::value_type* (see 20.1.5/4). Certainly, STLport makes that
assumption AFAIK, though Dinkumware comes quite close to avoiding making it.

I only know of one implementation that works with completely custom
pointer/reference types, and that's the one in the boost shared memory
library, which hasn't been released yet (see
http://ice.prohosting.com/newfunk/boost/libs/shmem/doc/html/shmem/quick_guide.html#shmem.qg_shmem_container)

> To achieve the equivalent effect of constructor-in-union, you might instead
> do a bit of fancy footwork with alignment guarantees, manual
> construction/destruction at the proper moments, and accessors that do
> reference casts. At least, that's what I did in my quick work-around, except
> for putting in the alignment guarantee (my own pointers didn't need it.)

Right. Dinkumware should perhaps investigate what changes (or "fixes")
they need to make to allow their containers to work with Boost::shmem.

You've reported a lot of non-bugs! Other STL implementations have all
the problems you mention, and more (e.g. STLport can't mix debugging and
non-debugging iterators either).

Tom

[Ivan Vecerina]

"P.J. Plauger" <p...@dinkumware.com> wrote in message
news:UsWdnc3maZdXkLzY...@giganews.com...
: So far you are zero for three, and seem to have trouble

: reading documentation. But if you have other bugs to report,
: I'm still very interested in hearing about them.

Sorry for jumping in, but I have been annoyed by
a real bug, that has been reported an acknowledged by MS
last March. It is now marked as "closed", but there seems
to be no patch or resolution available from Microsoft.

The bug report can be found here:
http://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx?FeedbackID=106322
I also described a failing test case here:
http://www.codecomments.com/archive373-2006-4-888237.html

Has Dinkumware solved this problem internally ?
What is the process for a patched library to reach end-users,
and when/how can I expect to receive a bug fix ?

Thanks,
Ivan
--
http://ivan.vecerina.com/contact/?subject=NG_POST <- email contact form

[Stephen Howe]

> PS - There have been rumors that the MS "_DEBUG" C++ operator-new/delete
is
> incompatible with the MS "NDEBUG" C++ operator-new/delete. Are the rumors
> true?

Yes

> (It would be simple to make the debug/release heap calls mutually mixable
and
> compatible (at a "normal case" cost of, hmmmmm.... one memory 'read' and
> three Intel instructions)

How?
The debug heap has extra bookkeeping to spot all kinds of ghastly errors
made by programmers using the heap.
The release heap has no such bookeeping, it is designed so that new/delete/
new[]/delete []/malloc/calloc/realloc/free
run as fast as possible.
The two are totally incompatible.
Now how are you going satisfy the criteria of both heaps _AND_ your simple
goal?

Stephen Howe

[P.J. Plauger]

"Ivan Vecerina" <INVALID_u...@ivan.vecerina.com> wrote in message
news:u6zqlRj5...@TK2MSFTNGP05.phx.gbl...

The _SECURE_SCL code was added by Microsoft after our last delivery.
They are responsible for making fixes like this.

[Carl Daniel [VC++ MVP]]

"Ivan Vecerina" <INVALID_u...@ivan.vecerina.com> wrote in message
news:u6zqlRj5...@TK2MSFTNGP05.phx.gbl...

Based on the close date, this is probably fixed in Orcas. It's possible
that it's fixed in VS 2005 SP1 (now in beta). You might want to download
the beta and try it.

There is only one process for getting a patch to an end user: you must
contact Microsoft Product Support and request it. I don't know whether
there's a hotfix available for this particular bug or not, but PSS should be
able to find out.

Worst case scenario - you wait for Orcas.

-cd

[Ivan Vecerina]

"Carl Daniel [VC++ MVP]"
<cpdaniel_remove...@mvps.org.nospam> wrote in message
news:OAJB0Ho5...@TK2MSFTNGP03.phx.gbl...
: "Ivan Vecerina" <INVALID_u...@ivan.vecerina.com> wrote in
message
: news:u6zqlRj5...@TK2MSFTNGP05.phx.gbl...
: >

http://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx?FeedbackID=106322
: > I also described a failing test case here:
: > http://www.codecomments.com/archive373-2006-4-888237.html
: >
: > Has Dinkumware solved this problem internally ?
: > What is the process for a patched library to reach end-users,
: > and when/how can I expect to receive a bug fix ?
:
: Based on the close date, this is probably fixed in Orcas. It's
possible
: that it's fixed in VS 2005 SP1 (now in beta). You might want to
download
: the beta and try it.
:
: There is only one process for getting a patch to an end user: you must
: contact Microsoft Product Support and request it. I don't know
whether
: there's a hotfix available for this particular bug or not, but PSS
should be
: able to find out.
:
: Worst case scenario - you wait for Orcas.

I did implement a workaround in my code (comparing element addresses
instead of iterators), so I will wait for Orcas.

Interesting to hear, though, that this is an MS-only extension
to the Dinkumware lib...

Thanks -Ivan