vc71 compiler seems to be very buggy
exl...@ml1.net
threads and uses several STL vectors and maps of simple objects
(most of which get a compiler generated assignment operator and copy
constructor). There is nothing fancy in that code...
The following combinations *always* crash rather quickly (30 to 70
seconds):
vc71 / STL either default Dinkumware or STLport/ -MDd or -MD.
cl.exe has version 13.10.3077.
The crashes occur either in destructors upon return from main
or upon insertion of a new element into the map/vector and
then reading it in another thread.
When I build with visual studio 6 SP6 or mingw gcc, then the program
works *always* perfectly in all of these combinations:
vc6 / STL either default Dinkumware or STLport / -MDd or -MD
gcc 3.4.2/ GNU STL lib or STLPORT / -g or -O3
At first I thought that maybe the stl library was the problem,
but by performing the above mentioned experiments I ruled that out.
Another possibility is that the vc71 compiler is good but the
debug/release libraries msvcrt71?.dll are buggy.
I've rarely seen compilers/runtime as buggy as vc71 or Sun Forte,
and for a while was contemplating abandoning c++ altogether and
moving to Ada or Pascal or C or ...
Anyway I'd be interested to hear if vc71 has been successfully
used to build any large non trivial, multi-threaded applications
that DO NOT crash.
Sorry I cannot provide source code.
Pete Becker
Jeff F
<exl...@ml1.net> wrote in message
news:1117054639.5...@g14g2000cwa.googlegroups.com...
...
> Anyway I'd be interested to hear if vc71 has been successfully
> used to build any large non trivial, multi-threaded applications
> that DO NOT crash.
Yes, several.
> Sorry I cannot provide source code.
Which no doubt is where the bugs are. The other compiler/lib variations were
probably just lucky not to crash. Or you've got some flakey combination of
compiler options.
Jeff Flinn
Brian Muth
> used to build any large non trivial, multi-threaded applications
> that DO NOT crash.
>
Yes, including using STL.
Are you properly serializing access to your map/vector? Or are you allowing
threads to access the map/vector willy-nilly?
Brian
exl...@ml1.net
to a small test case. I am not trying to be arrogant, but simply can't
post all the source code to a public forum.
The point is that with the vc6sp6 cl.exe and mingw 3.4.2 compiler I've
never seen any crashes, yet with vc71 it happens all the time.
exl...@ml1.net
Arnaud Debaene
> Yes, I use spinlocks to ensure serial access.
Do you take into account that inserting/removing/sorting a container in a
thread while iterating over the same container into another thread is not
thread-safe, even if each operation is serialized? (basically, the whole
iteration over the container must be made atomic through serialization).
Arnaud
MVP - VC
Brian Muth
> The point is that with the vc6sp6 cl.exe and mingw 3.4.2 compiler I've
> never seen any crashes, yet with vc71 it happens all the time.
>
And therefore it must be the compiler's fault? That is a hasty conclusion.
Look again.
Brian
exl...@ml1.net
*ALL* of them:
NO crashes ever with
vc6, sp6
------------
default dinkumware STL, -MDd -Zi
default dinkumware STL,-MD -Ox
STLPORT 5.0RC2, -MDd -Zi
STLPORT 5.0RC2, -MD -Ox
....ditto for STLPORT 4.6.3.01
-----------
mingw gcc 3.4.2
------------
GNU STL, -g
GNU STL, -O3 -march=i686
STLPORT 5.0RC2, -g
STLPORT 5.0RC2, -O3 -march=i686
...ditto for STLPORT 4.6.3.01
------------
As far as vc71 is concerned no matter what the combinations of STL and
debug/release flags I *always* get a crash within seconds.
Furthermore, on my computer at work I have vc71 enterprise 2003 net
architect compiler available from an image which I have not prepared.
At home I have a vc71 compiler downloaded from MSDN. So you can't say
that a bad compiler install is to blame.
Also I've been writing OS kernel code for a living for a number of
years and don't suspect that a thread synchronization issue is to blame
either. Every function and line has been carefully scrutinized and
there aren't too many of them.
Brian Muth
> to a small test case.
If you mean when you reduce the code, the bug disappears, then that is an
important clue.
Brian
exl...@ml1.net
string &index = (*gshare.seclist)[j];
object_name &di = (*gshare.secmap)[index];
di.s_lock.lock();
vector<xhg_data> &ivars = di.s_intra;
xhg_data &xd = (*gshare.snapshot)[index];
xd.x_lock.lock();
ivars.push_back(xd);
xd.x_lock.unlock();
di.s_lock.unlock();
Never mind that I should use rwlocks instead of spinlocks...
That can be fixed later.
Brian Muth
but never a Win32 application. Usually one uses a mutex or critical section.
Can you elaborate?
Also, I encourage you to post code snippets (without creating problems of
intellectual property), otherwise we really have nothing to go on here...
Brian
exl...@ml1.net
Either of win32 critical section or mutex would prevent wasteful
spinning, as opposed to a spinlock (may spin wastefully if not
used judiciously) which is a long variable atomically changed from
0 to 1 or vice versa ("atomic" always implies memory barrier
operation). My code has to be portable and therefore can't
depend on too many windowisms.
In the example I gave the first vector and the map are
initialized once and remain unchanged forever afterwards.
Only push_back operations to the inner vector occur over time.
Unfortunately I can't say much more than that.
I am developing a fully automatic trading system which is the
most challenging project a programmer can tackle.
It separates the men from the boys because to be successful
one has to delve deeply into many areas - deductive, inductive
logic, philosophy, psychology, causality, bayesian networks,
AI, algorithm complexity theory, entropy of information
meta reasoning, .......
Anyone who is ignorant of these things may get robbed in
the markets.
adeb...@club-internet.fr
exl...@ml1.net a écrit :
> I am developing a fully automatic trading system which is the
> most challenging project a programmer can tackle.
> It separates the men from the boys because <snip> Anyone who is ignorant of these things may get robbed in
> the markets.
Just out of curiosity.... Do you *really* expect to get any help from a
public newsgroup with this kind of behaviour ????
Arnaud
MVP - VC
Tom Widmer
There are probably some thousands of such applications - a good
proportion of VC users upgraded long ago I think, and VC is a very
widely used compiler. It may be that VC7.1 is being the kindest of your
compilers by crashing due to the faulty code rather than hiding it with
a more subtle behavioural defect. Or it may indeed be a bug in the
compiler (though that seems very unlikely given that you say you are
just using multithreaded STL code, which works fine for me at least).
Have you tried debugging? What was the outcome?
Tom
Jason Winnebeck
> spinning, as opposed to a spinlock (may spin wastefully if not
> used judiciously) which is a long variable atomically changed from
> 0 to 1 or vice versa ("atomic" always implies memory barrier
> operation). My code has to be portable and therefore can't
> depend on too many windowisms.
Portable threading libraries exist. On first thought when I hear something
like this I get cautious. Creating an actual portable lock in user-space
(not kernel-space) is an extremely non-trivial task. I know you said you've
been coding OS software for years, and that you are a man and we are only
boys, but some thoughts come to mind:
* Did you properly use volatile where necessary?
* VC 7.1 has global optimization, whereas GCC and VC 6 do not. Does the
program crash only when optimization is on? If you've implemented your own
lock, VC might be inlining the code and see that lock and unlock do
seemingly menial tasks and might be reordering your code or doing something
to break memory barriers, if it thinks there are no side effects. Basically
I could think of a million things that the optimizer might feel safe in
doing that would break what you want, and VC7.1 is much more agressive than
VC6 at optimizing.
* Did you consider memory barrier issues in SMP systems?
Also another poster recently discussed debugging. Did you run the system in
a debugger to try to get a better sense of what is going on?
> I am developing a fully automatic trading system which is the
> most challenging project a programmer can tackle.
> It separates the men from the boys because to be successful
> one has to delve deeply into many areas - deductive, inductive
I'm glad you didn't mention social skills in that list, with which you seem
to be seriously deficient.
Jason
Peter Koch Larsen
<exl...@ml1.net> skrev i en meddelelse
news:1117060738.0...@g44g2000cwa.googlegroups.com...
[snip]
> I am developing a fully automatic trading system which is the
> most challenging project a programmer can tackle.
> It separates the men from the boys because to be successful
> one has to delve deeply into many areas - deductive, inductive
> logic, philosophy, psychology, causality, bayesian networks,
> AI, algorithm complexity theory, entropy of information
> meta reasoning, .......
Before you delve to deep into all that fancy stuff, could I suggest you
tried studying your own code as it is apparently bad?
To be kind, I will suggest you look at code-sections where behaviour might
be different on the new and the old compiler - e.g. difference in evaluation
in all the situations where the behaviour is either undefined or
implementation-defined.
Also - as mentioned by someone else - if you remove some code and this makes
your program work, the error is likely to be in the code you snipped.
[snip]
/Peter
Carl Daniel [VC++ MVP]
> threads and uses several STL vectors and maps of simple objects
> (most of which get a compiler generated assignment operator and copy
> constructor). There is nothing fancy in that code...
Your home-made spin lock is neatly encapsulated in a class (or at least
macros), right? Try a simple test: Replace your home-made spin lock with a
Win32 Ciritical section and see if your problems go away.
If you can't easily do that, then your code is already seriously in need of
some re-factoring to separate platform abstractions from your business
logic.
-cd
Stephen Howe
I have seen many, "This code works with vendor A, vendor B but does not work
vendor C" and 99 times out 100 is the programmers fault.
The programmer may be making some assumption that he assumes is true for C
or C++ or a library function when from existing C or C++ standards it is
unwarranted.
A case in point is that VC6 implemented vector iterators as pointers whereas
VC7 and VC7.1 implements them as classes. Therefore any programmer
concluding that "vendor C is buggy" because he can assign a vector iterator
to a pointer just because with 100 other vendor STL's have used pointers as
an implementation, is wrong. Nowhere in the C++ standard is this endorsed.
It is possible to write code that should work with _ALL_ vendors, assignment
from a vector iterator to a pointer is non-standard.
Why am I mentioning this?
Because without seeing your code, without a test case from yourself it is
impossible to conclude that you are right. For all I know, you may have made
a glaring mistake with your multi-threaded code that just happens to "work"
with other library/vendor combinations. The fact that you cannot isolate
where the problem is also does not assist your case.
Stephen Howe
exl...@ml1.net
> Portable threading libraries exist. On first thought when I hear something
> like this I get cautious. Creating an actual portable lock in user-space
> (not kernel-space) is an extremely non-trivial task. I know you said you've
> been coding OS software for years, and that you are a man and we are only
> boys, but some thoughts come to mind:
>
> * Did you properly use volatile where necessary?
Absolutely, I also use -GL, -ltgc for interprocedural optimizations in
release mode.
> * Did you consider memory barrier issues in SMP systems?
Absolutely, they are non issue on most x86 CPUs because mem op
ordering is strong. But I've written assembly for sparc64 where
cas + membar take care of atomic stuff. The instructions are not
atomic by themselves but the whole sequence from cas to membar
is *logically* atomic.
> Also another poster recently discussed debugging. Did you run the system in
> a debugger to try to get a better sense of what is going on?
Absolutely, crashes happen in the same place whether I use stlport
or dinkumware headers (somewhere in the destructors upon return from
main or upon making a reference to the map in another thread or
upon element insertion into the vector).
And yes, long years of experience have taught me that I should be
very cautios and patient before blaming the compiler.
It's usually the programmer's fault...But if vc6sp6 and gcc work
perfectly what does that say about the state of software engineering
with C++?
It's is scary to be lulled into false sense of security only to
have your confidence shattered when you upgrade to the next version
of the compiler.
> > I am developing a fully automatic trading system which is the
> > most challenging project a programmer can tackle.
> > It separates the men from the boys because to be successful
> > one has to delve deeply into many areas - deductive, inductive
>
> I'm glad you didn't mention social skills in that list, with which you seem
> to be seriously deficient.
Haha, you've misunderstood, I've never claimed that I have the answers.
On the contrary, the amount of difficulty to overcome is staggering.
What I have is a good idea about the fundamental limits I am up
against.
If you were a smug AI expert, would you agree to write a program that
attempts to predict an algorithmically random sequence of data?
Or solve a problem that is equivalent to the Post Correspondence
Problem?
Yet that is exactly what many AI and pattern recognition experts try to
do when attempting to decipher the market.
exl...@ml1.net
didn't exist 5 years ago. Most memory is managed too.
The scary thing is that since the code works perfectly with vc6 sp6 and
mingw gcc 3.4.2 I might be lulled into false sense of security that
it's correct. Yet when the next upgrade of the compiler takes place,
my confidence will be shattered.
What does that say about the state of software engineering with C++?
The reason I post to this forum is to get feedback from the bigger
collective of vc71 users whether they have seen bizarre problems like
these when they upgraded from vc6 sp6.
Tom Widmer
> There are no C++ facilities/techniques/keywords in this program that
> didn't exist 5 years ago. Most memory is managed too.
>
> The scary thing is that since the code works perfectly with vc6 sp6 and
> mingw gcc 3.4.2 I might be lulled into false sense of security that
> it's correct. Yet when the next upgrade of the compiler takes place,
> my confidence will be shattered.
> What does that say about the state of software engineering with C++?
That depends on what has caused the problem. Multithreaded code in any
language which has threading bugs is likely to suffer in exactly this way.
Can you post your spinlock code? Are you using
InterlockedCompareExchange or what? Have you tried using critical
sections instead, as Carl suggested?
Tom
Tom Widmer
> * Did you properly use volatile where necessary?
Where is it necessary? I've never found a use for volatile in
multithreaded code, even lock-free multithreaded code based on atomic
operations (Interlocked* functions).
Tom
exl...@ml1.net
I never claimed to have the answers...
Your answer sugests you suffer from the VHK syndrome.
VHS stands for Vertiginous Heights of Knowledge.
A lot of linux kernel developers and Prolog experts
exhibit signs of the VHK syndrome too, just watch their
angry outbursts as they berate less perfect beings.
About 5 of all techies suffer from VHK.
How is one diagnosed as suffering from the VHK syndrome?
These are some symptoms by which you shall know them
1) VHK sufferers, having attained unimaginable heights
of expertise, become angry that their coevals
and fellow programmers wallow in mediocrity.
Correspondingly they spew forth stinging criticism
at others and even their own code when it's not
written up to high standards.
Worse yet, their contemporaries and colleagues are
oftentimes unable to grasp their "simple" ideas.
2) VHK sufferers look forward to getting challenges,
much the same way as a crazy race driver looks
forward to the race track even if they may die on it.
3) Unable to resist a flood of creativity that bubbles
up within them they are constanty thinking of some
improvement or other and very much like manic depressives
are crestfallen when they are unable to realize these
ideas because their employer had them do something else.
Generally speaking VHK syndrome victims are as a rule
lonely, isolated and unable to function in large teams,
despite writing amazing code and exceeding expectations.
Imagine Kevin Lawton, Richard Stallman or Albert Einstein
as mid career employees at IBM, Microsoft or Lucent.
They would languish, suffocate and die a painful death in such
an environment, where hierarchies are bogus (not based
on merit), creativity is not appreciated (mandated) and architects
are oftentimes unremarkable thinkers who befriended their
boss 10 years ago.
So my advice to you friend is get out. Found your own company,
prosper and be happy.
Stephen Howe
> mingw gcc 3.4.2 I might be lulled into false sense of security that
> it's correct. Yet when the next upgrade of the compiler takes place,
> my confidence will be shattered.
> What does that say about the state of software engineering with C++?
Nothing really. It is a hard thing to prove that a program is "correct".
Static program tools only work for the simplest of programs with few
branches or loops or the domain they cover is so small as to be almost
worthless.
It seems that only in the field of mathematics is it possible for a proof
to span infinity (like Fermats Last Theorem).
Coming back to programming, correctness is hard to verify:
The program may be incorrect
The language may make insufficient guarantees for a solution to a problem.
The compiler may be faulty.
The linker may be faulty.
The libraries may be incorrect.
The OS support for the program may be faulty.
The processor may be faulty.
I have seen problems surface in every area I mentioned above. It is a
depressing thought.
Having said that, dynamic tools for catching bugs have improved over the
years.
> The reason I post to this forum is to get feedback from the bigger
> collective of vc71 users whether they have seen bizarre problems like
> these when they upgraded from vc6 sp6.
A reasonable position.
AFAIK, 7.1 is 7.0 with the bugs fixed.
I personally know of 1 bug with 7.1, 2 language support flaws and a few
flaws with libraries (there are probably more but I don't spend all my time
in Microsoft newsgroups with my ear to the ground).
But if I Google around I draw a blank on 7.1 and multithreading bugs.
If what you say is so, I would expect at least a few verifiable reports. But
nothing.
That suggests it is highly likely that your program has a bug that the other
compilers/libraries did not expose. Of course, it is not impossible that 7.1
does have a bug but so far with the information you have presented in terms
of maps/sets/destructors, there is little to go on. What about implementing
Carl's suggestion?
Stephen Howe
Brian Muth
> Carl's suggestion?
>
and/or... (and at risk of being a bit inflammatory)
What separates the men from the boys is the willingness to post relevant
code snippets for critical review by this panel.
Brian
Simon Trew
news:%23h%23rrQjY...@TK2MSFTNGP12.phx.gbl...
I did my apprenticeship at a defense company, originally in electronics, but
ended up writing missile guidance software. To supplement my income as a
struggling apprentice, I worked in a pub. It was a delightful moment when
after explaining to a grumpy punter why I couldn't legally serve him, he
asked "who do you think you are, a rocket scientist?" only to be told by a
regular customer, "yes, he is, actually."
Rocket science isn't as hard as it's made out to be*.
S.
* Or rather, using the usual engineering discipline of functional
decomposition, you work with a team to break down a problem until you can
solve it.
um
The InterlockedXXX functions expect a ptr to a right aligned variable.
On a 32-bit system something like the following:
#pragma pack(push, 4)
volatile long value;
#pragma pack(pop)
Citation (MSDN, SDK):
"The functions InterlockedDecrement, InterlockedCompareExchange,
InterlockedExchange, InterlockedExchangeAdd, and InterlockedIncrement
provide a simple mechanism for synchronizing access to a variable that is shared by multiple threads.
The threads of different processes can use this mechanism if the variable is in shared memory.
The variable pointed to [...] must be aligned on a 32-bit boundary; otherwise, this
function will fail on multiprocessor x86 systems and any non-x86 systems. "
um
adeb...@club-internet.fr
um a écrit :
> "Tom Widmer" <tom_u...@hotmail.com> wrote
> > Jason Winnebeck wrote:
> > > * Did you properly use volatile where necessary?
> >
> > Where is it necessary? I've never found a use for volatile in
> > multithreaded code, even lock-free multithreaded code based on atomic
> > operations (Interlocked* functions).
>
> The InterlockedXXX functions expect a ptr to a right aligned variable.
> On a 32-bit system something like the following:
We agree that the variable needs to be properly aligned, but this has
nothing to do with volatile...
Arnaud
MVP - VC
um
you that volatile is required:
LONG __cdecl InterlockedIncrement(IN OUT LONG volatile *lpAddend);
LONG __cdecl InterlockedDecrement(IN OUT LONG volatile *lpAddend);
LONG __cdecl InterlockedExchange(IN OUT LONG volatile *Target, IN LONG Value);
LONG __cdecl InterlockedExchangeAdd(IN OUT LONG volatile *Addend, IN LONG Value);
LONG __cdecl InterlockedCompareExchange (IN OUT LONG volatile *Destination,
IN LONG ExChange, IN LONG Comperand);
// etc.
<adeb...@club-internet.fr> wrote
exl...@ml1.net
post it to this forum.
Just yesterday I found out that if I remove 2 variables (double and
unsinged long) from the struct below then the map object doesn't crash
any more when building with vc71.
struct xxx { // map[string] = xxx;
double a;
unsigned long b;
unsigned char c:1;
unsigned char d:1;
};
Moved snippets out of larger code and couldn't reproduce any more :(.
release build: -W3 -nologo -G7 -Ox -MD -EHsc -GL for .cpp .c files
-ltcg for linker
debug build: -Zi -nologo -MDd -EHsc -W3
Tom Widmer
> I hope the following function prototypes from Winbase.h should convince
> you that volatile is required:
>
> LONG __cdecl InterlockedIncrement(IN OUT LONG volatile *lpAddend);
> LONG __cdecl InterlockedDecrement(IN OUT LONG volatile *lpAddend);
> LONG __cdecl InterlockedExchange(IN OUT LONG volatile *Target, IN LONG Value);
> LONG __cdecl InterlockedExchangeAdd(IN OUT LONG volatile *Addend, IN LONG Value);
> LONG __cdecl InterlockedCompareExchange (IN OUT LONG volatile *Destination,
> IN LONG ExChange, IN LONG Comperand);
> // etc.
Nope, you can pass the address of a non-volatile LONG and all of those
functions will work fine.
Tom
![Doug Harrison [MVP]'s profile photo](http://lh3.googleusercontent.com/a-/ALV-UjVvzhOQGbT9K0Re6Qg-OU4VGuHq0tAdTu01YlmIjUGm6t_o8g=s40-c)
Doug Harrison [MVP]
I once traced the evolution of those signatures:
http://groups-beta.google.com/group/microsoft.public.msdn.general/msg/d9853a418bfc3541?hl=en
<q>
The InterlockedXXX functions are interesting. Just a few years ago, they
took plain old LPLONG parameters. Then they changed to "LPLONG volatile",
which is a useless change. Nowadays, they take "volatile LONG*", and I'm
not sure why. When I looked at this recently, I thought maybe it's to
suppress optimizations in the face of the intrinsic versions of these
functions, which on x86, replace function calls with inline assembly
instructions that use the lock prefix. (That was just a total guess, and
some simple tests I've done since then suggest that's not the reason.) Then
I noticed the following in MSDN:
volatile LONG data = 1;
...
while (data < 100)
{
if (data < 100)
{
InterlockedIncrement(&data);
printf("Thread %d: %d\n", threadNum, data);
}
Sleep(time); // wait up to half of a second
}
Now I think it's just to enable people to more conveniently write wrong MT
code, that sometimes sorta works, especially on x86. Using an older SDK,
one would have to cast volatile away to call InterlockedIncrement, which
could be considered a bother.
</q>
--
Doug Harrison
Microsoft MVP - Visual C++
Tom Widmer
> On Fri, 27 May 2005 16:43:28 +0100, Tom Widmer wrote:
>
>
>>um wrote:
>>
>>>I hope the following function prototypes from Winbase.h should convince
>>>you that volatile is required:
>>>
>>>LONG __cdecl InterlockedIncrement(IN OUT LONG volatile *lpAddend);
>>>LONG __cdecl InterlockedDecrement(IN OUT LONG volatile *lpAddend);
>>>LONG __cdecl InterlockedExchange(IN OUT LONG volatile *Target, IN LONG Value);
>>>LONG __cdecl InterlockedExchangeAdd(IN OUT LONG volatile *Addend, IN LONG Value);
>>>LONG __cdecl InterlockedCompareExchange (IN OUT LONG volatile *Destination,
>>> IN LONG ExChange, IN LONG Comperand);
>>>// etc.
>>
>>Nope, you can pass the address of a non-volatile LONG and all of those
>>functions will work fine.
>
>
> I once traced the evolution of those signatures:
>
> http://groups-beta.google.com/group/microsoft.public.msdn.general/msg/d9853a418bfc3541?hl=en
Sounds about par for the course. Here is interesting:
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dllproc/base/about_synchronization.asp
I think that the "repaired" code at the bottom is not sufficient to fix
the race condition; you also need a memory barrier in FetchComputedValue
to prevent iValue from being read before fValueHasBeenComputed (as might
easily happen if branch prediction is taking place). Am I right in
saying the code needs to be:
BOOL FetchComputedValue(int *piResult)
{
if (fValueHasBeenComputed)
{
MemoryBarrier(); //server edition only, for multiCPU IA64
//actually, only an acquire barrier is required, but
//Microsoft doesn't provide such a thing AFAIK
*piResult = iValue;
return TRUE;
}
else
return FALSE;
}
Tom
Doug Harrison [MVP]
> Sounds about par for the course. Here is interesting:
> http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dllproc/base/about_synchronization.asp
>
> I think that the "repaired" code at the bottom is not sufficient to fix
> the race condition; you also need a memory barrier in FetchComputedValue
> to prevent iValue from being read before fValueHasBeenComputed (as might
> easily happen if branch prediction is taking place). Am I right in
> saying the code needs to be:
>
> BOOL FetchComputedValue(int *piResult)
> {
> if (fValueHasBeenComputed)
> {
> MemoryBarrier(); //server edition only, for multiCPU IA64
> //actually, only an acquire barrier is required, but
> //Microsoft doesn't provide such a thing AFAIK
> *piResult = iValue;
> return TRUE;
> }
> else
> return FALSE;
> }
>
> Tom
Yeah, I think you're right. In particular, this statement that follows the
inadequately corrected code is wrong but explains the thinking involved:
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dllproc/base/synchronization_and_multiprocessor_issues.asp
<q>
The InterlockedExchange function ensures that the value of iValue is
updated for all processors before the value of fValueHasBeenComputed is set
to TRUE.
</q>
Actually, this rings a bell. I've seen this discussed in
comp.programming.threads. A MSFT guy even acknowledged the error:
exl...@ml1.net
Very well. You asked me to shatter your confidence in the vc71
compiler/runtime right? You'll now get what you asked for.
I stripped down my sources and obfuscated them just for you.
The bug you are about to observe it TRULY bizarre and freakish!
Here is how to reproduce - extract 4 files: a.cpp, b.cpp, c.h and
build.bat and run build.bat. It will produce crash.exe, run it.
Bizarre thing #1 if you remove <winsock2.h> from a.cpp, rebuild, then
crash goes away (please note winsock2.h is not even relevant to the
needs of the program, I do use it though in my code for tcp/ip comm.).
Bizarre thing #2 if you restore <winsock2.h> in a.cpp the way it was
and remove member "double a; " from class hahaha in c.h then crash goes
away too!
Also it's important to stress that concurrency is NOT an issue with
this multithreaded program because the main thread waits for the other
thread to completely finish and disappear before returning (and running
the destructors).
And finally note that _endthreadex does NOT have to be run in the
subordinate thread because the return statement takes care of that
implicitly.
Also, comments about the logic of the program are useless because I've
stripped it and obfuscated it enough that suggestions about the logic
are meaningless.
My computer runs XP, latest service pack.
----------- build.bat begins ----------------------
cl -I. -nologo -MDd -EHsc -W3 -DWINVER=0x400 -c a.cpp -Foa.obj
cl -I. -nologo -MDd -EHsc -W3 -DWINVER=0x400 -c b.cpp -Fob.obj
cl -MDd -EHsc -nologo -Zi -W3 -Fecrash.exe a.obj b.obj
----------- build.bat ends ------------------------
----------- a.cpp begins --------------------------
#include <winsock2.h>
#include <string>
#include <map>
#include <stdio.h>
#include "c.h"
extern map<string,dainty> *ptr;
void funct1(void *arg)
{
try {
(*ptr)["xxxx"] = dainty();
} catch (...) {
printf("funct1: unknown exception\n");
}
printf("exiting funct1\n");
}
------------a.cpp ends ---------------------------
------------b.cpp begins ---------------------
#pragma warning(disable:4786)
#include <io.h>
#include <windows.h>
#include <process.h>
#include <stdio.h>
#include "c.h"
map<string,dainty> *ptr;
extern void funct1(void *arg);
static unsigned __stdcall wrapper_str(void *arg)
{
funct1(0);
return 0;
}
int main()
{
try {
map<string,dainty> mymap;
ptr = &mymap;
unsigned thrid;
HANDLE h = (HANDLE) _beginthreadex(0, 2*8192, wrapper_str, 0, 0,
&thrid);
WaitForSingleObject(h, INFINITE);
CloseHandle(h);
} catch (std::string &msg) {
printf("exception in main: %s\n", msg.c_str());
} catch (...) {
printf("main: unknown exception caught\n");
}
return 0;
}
------------b.cpp ends -----------------------
----------- c.h begins -----------------------
#ifndef _c_h_
#define _c_h_
#include <string>
#include <vector>
#include <map>
using namespace std;
class myobject {
private:
void *opaque;
volatile unsigned long *refcnt;
void delete_myobject() {
if(refcnt) {
(*refcnt)--;
if(*refcnt == 0) {
free(opaque);
free((void *) refcnt);
}
}
}
public:
myobject() {
opaque = malloc(4);
if(!opaque) {
out: throw string("myobject constructor failed");
}
refcnt = (volatile unsigned long *) malloc(sizeof *refcnt);
if(refcnt == 0) {
delete_myobject();
goto out;
}
*refcnt = 0;
(*refcnt)++;
}
myobject(const myobject &arg) : opaque(arg.opaque),
refcnt(arg.refcnt) {
(*refcnt)++;
}
~myobject() { delete_myobject(); }
myobject &operator=(myobject const& arg) {
if(this == &arg)
goto out;
if(refcnt != arg.refcnt) {
delete_myobject();
opaque = arg.opaque;
refcnt = arg.refcnt;
(*refcnt)++;
}
out: return *this;
}
};
class hahaha {
public:
double a;
unsigned long b;
unsigned char c;
unsigned char d;
};
class my_data {
public:
myobject x_obj;
};
class dainty {
public:
hahaha s_day;
vector<my_data> s_intra;
myobject s_obj;
};
#endif /* _c_h_ */
------------c.h ends--------------------------
Doug Harrison [MVP]
There's a bug in <winsock2.h> such that it changes the packing from the
default 8 to 4 when WIN32 isn't #defined. You're #including c.h in two
different ways; in a.cpp, it follows <winsock2.h>, while b.cpp doesn't
#include <winsock2.h>. I expect this is causing various classes to have
different layouts in these files. You can see the underlying problem if you
add the following line to the top of c.h and following all its #include
directives:
#pragma pack(show)
Then when I compile using your command line, I get:
C>cl -I. -nologo -MDd -EHsc -W3 -DWINVER=0x400 a.cpp b.cpp
a.cpp
c:\temp\c.h(4) : warning C4810: value of pragma pack(show) == 4
c:\temp\c.h(7) : warning C4810: value of pragma pack(show) == 4
c:\temp\c.h(9) : warning C4810: value of pragma pack(show) == 4
c:\temp\c.h(11) : warning C4810: value of pragma pack(show) == 4
b.cpp
c:\temp\c.h(4) : warning C4810: value of pragma pack(show) == 8
c:\temp\c.h(7) : warning C4810: value of pragma pack(show) == 8
c:\temp\c.h(9) : warning C4810: value of pragma pack(show) == 8
c:\temp\c.h(11) : warning C4810: value of pragma pack(show) == 8
I think that in general, if you're going to use the Windows SDK, you should
#include <windows.h> before any other Windows header, which would avoid
this problem without you having to remember to specify /DWIN32 for these
command lines you create yourself.
exl...@ml1.net
Why then when I build with vc6 sp6 I am not seeing this crash?
winsock2.h header fixed in that version???
Stephen Howe
> compiler/runtime right?
Having seen Doug's trenchant analysis:
Well well well.
You got caught by different levels of packing.
Windows programmers have known for years that the header files Microsoft
distributes for the Platform SDK are kludgy invidually and that despite the
obscenity of how much is pulled in by #include <windows.h> - it is best
practice.
So much for 7.1 bugginess and multi-threading.
Any more "bugs" to report???
It is not to say that there aren't any - I know there are - but a certain
amount of humility of reporting "bugs" saves a lot of embarassment - on
your part.
Stephen Howe
Carl Daniel [VC++ MVP]
When you compile with VC6 are you using the platform SDK header/libs that
VC6 installed?
It's entirely possible that the error in winsock2.h was introduced in a
later version.
-cd
exl...@ml1.net
>
> It is not to say that there aren't any - I know there are - but a certain
> amount of humility of reporting "bugs" saves a lot of embarassment - on
> your part.
>
> Stephen Howe
I am glad to report that Doug's analysis was correct and fixed the
problem.
On the subject of humility, it's embarrassing that amateur programmers
are employed by MSFT and allowed to contribute code that gets as much
visibility as visual studio.
Needless to say global #pragma scope is a bad idea.
Per struct pack attributes as in gcc are a much cleaner approach to
packing.
In any event if I were in charge of hiring for MSFT I'd take applicants
on a wrenching computer science test.
Only the best can survive it. Here are some simple questions that may
catch even smug programmers with 10+yrs experience off guard.
1) is priority inversion a good or a bad thing and what alternatives
to it exist?
2) when an interrupt occurs in the OS (smp or no smp), what stack does
it execute on and what happens if that interrupt hits a trap and is
preempted by another interrupt afterwards?
3) describe a virtualization technique whereby the program counter is
constantly under control in user space.
4) describe an implementation of read/write locks in 5 minutes or less.
5) describe an implementation of a FIFO that resides in memory that
is shared between user space and the kernel (note: obviously can't
contain pointers to memory outside that buffer).
the list of questions can go on and on.
It's embarrassing that a very large percentage of programmers nowadays
are too lazy or lack intellectual curiosity that would enable them to
think through simple mind exercises like these.
Bottom line is if you don't understand low level
kernel/memory/interrupt/compiler/linker issues you shouldn't be allowed
to write application code. Understanding
kernel/concurrency/compiler/linker issues on modern computers MUST be
made a PREREQUISITE before programmers are allowed to write production
code.
Bye.
Stephen Howe
> are employed by MSFT and allowed to contribute code that gets as much
> visibility as visual studio.
Agreed. I can think of quite a few issues to do with header files/libraries
that have remained unchanged by Microsoft for many years and I am amazed
that they have not been addressed. Given the volume of developers, you would
think it is priority to fix long standing problems rather than issue a new
version of VC or Platform SDK that has some new "gee whizz" features or
gimmicks. To me, it is travesty that the Platform SDK headers are not
"complete" like C & C++ headers.
> Needless to say global #pragma scope is a bad idea.
> Per struct pack attributes as in gcc are a much cleaner approach to
> packing.
Agreed. Been using "per struct pack" with all many compilers since 1986.
> In any event if I were in charge of hiring for MSFT I'd take applicants
> on a wrenching computer science test.
Sure.
> Only the best can survive it. Here are some simple questions that may
> catch even smug programmers with 10+yrs experience off guard.
>
> 1) is priority inversion a good or a bad thing and what alternatives
> to it exist?
> 2) when an interrupt occurs in the OS (smp or no smp), what stack does
> it execute on and what happens if that interrupt hits a trap and is
> preempted by another interrupt afterwards?
> 3) describe a virtualization technique whereby the program counter is
> constantly under control in user space.
> 4) describe an implementation of read/write locks in 5 minutes or less.
> 5) describe an implementation of a FIFO that resides in memory that
> is shared between user space and the kernel (note: obviously can't
> contain pointers to memory outside that buffer).
What does that prove?
I can ask questions in some areas that I am expert on, that I know are
extremely unlikely to be answered, even by the best. Should I then reject
programmers who fail my tests?
> the list of questions can go on and on.
> It's embarrassing that a very large percentage of programmers nowadays
> are too lazy or lack intellectual curiosity that would enable them to
> think through simple mind exercises like these.
> Bottom line is if you don't understand low level
> kernel/memory/interrupt/compiler/linker issues you shouldn't be allowed
> to write application code.
I find myself in partial agreement with you.
I agree with you the on the lack intellectual curiosity by many programmers.
Most programmers have little knowledge on data structures, they have never
been formally trained (at college, University etc) - single linked lists,
double linked lists, binary trees, AVL trees, red-black trees, B-trees (and
other variants), skip lists, heaps, stacks, queues, priority queues,
implementation of memory mangement structures etc, the use of sentinel
values.
Most programmers have little knowledge on algorithms - various sorts,
searches, selection, graphical algorithms or what research papers have been
presented in the past 15 years which improve existing algorithms.
But how far do we take this?
Do we not employ anyone that has not implemented a compiler/linker/librarian
for some language?
Do we not employ anyone that does not know by heart the C & C++ standard, or
owns a copy or can write quality code that for the best part is portable and
makes no OS assumptions?
Do we not employ anyone that is not uptodate with algorithmic research?
Do we not employ anyone for Intel platforms that cannot write 32-bit
assembler code and has no understanding on MMX and SIMD instruction set?
All of the above?
Anybody with an understanding of low level
kernel/memory/interrupt/compiler/linker issues is not going to be content
with writing mere application code.
> Understanding kernel/concurrency/compiler/linker issues on modern
computers MUST be
> made a PREREQUISITE before programmers are allowed to write production
> code.
Is that all? If you say that is necessary then I would many more requirement
s than that.
And we have longed passed the point when a programmer can know everything in
our small Universe.
There was a point when Microsoft issued their compilers in 1986 and before a
few weeks were out, I had a fairly complete grasp of every tool that was on
the disks, their limits, all documentation read etc. Not any more. It is
simply too big. And I spend a significant period of every day absorbing
knowledge.
Stephen Howe
um
> > Very well. You asked me to shatter your confidence in the vc71
> > compiler/runtime right?
>
> Having seen Doug's trenchant analysis:
>
> Well well well.
> You got caught by different levels of packing.
> Windows programmers have known for years that the header files Microsoft
> distributes for the Platform SDK are kludgy invidually and that despite the
> obscenity of how much is pulled in by #include <windows.h> - it is best
> practice.
> So much for 7.1 bugginess and multi-threading.
Does anybody know if analysing the linker map file would be
of any help in (programmatically) locating this nasty error?
(I mean by a simple cmdline tool which parses the map file).
An example map file:
0001:000034a0 ??0ItemRec@@QAE@_N@Z 004044a0 f i server.obj
0001:00003ce0 ??1ItemRec@@QAE@XZ 00404ce0 f i server.obj
"ItemRec" is a C++ struct.
I wonder why ItemRec is listed twice for the same module.
What does the character 0 and 1 in front of the name mean?
Some other structs have 4 there.
Code4u
>Stephen Howe wrote:
>>
>> It is not to say that there aren't any - I know there are - but a certain
>> amount of humility of reporting "bugs" saves a lot of embarassment - on
>> your part.
>>
>> Stephen Howe
>
>I am glad to report that Doug's analysis was correct and fixed the
>problem.
>On the subject of humility, it's embarrassing that amateur programmers
>are employed by MSFT and allowed to contribute code that gets as much
.
.
rambling rant snipped
.
.
>Bye.
All software has bugs, but if your application doesn't work, 99% of
the time it's your fault and not the system software. Or to put it
another way, when you hear the sound of hooves, think horses, not
zebra. You may consider yourself an expert but you made a rookie
mistake in assuming the compiler was at fault. Tip for the future:
never blame the compiler until you have proof, and remember, it's
quite unlikely that you will be the first to discover a particular
compiler bug.
exl...@ml1.net
> All software has bugs, but if your application doesn't work, 99% of
> the time it's your fault and not the system software. Or to put it
> another way, when you hear the sound of hooves, think horses, not
> zebra. You may consider yourself an expert but you made a rookie
> mistake in assuming the compiler was at fault. Tip for the future:
> never blame the compiler until you have proof, and remember, it's
> quite unlikely that you will be the first to discover a particular
> compiler bug.
True, if you had read the original post that spawned this discussion
then you would have realized that I suspected not only the compiler but
the runtime. What good is a compiler written by MSFT's brightest when
morons crafted the headers?
You guessed right that windows isn't my strong suit but I've jumped on
the windows bandwagon because of dependency on 3rd party sw. vendors
and their libraries. As a new arrival to windows it's putting me off to
see garbage like nonportable (D)WORD literals, build env. where
dependencies are hidden in project files and one is left guessing what
flags/directed acyclic graph generates the project (I use makefiles
only), and ill conceived pragma pack...
Stephen Howe
> morons crafted the headers?
Well I agree with you there.
Windows has been around since 1985, it is amazing to think that 20 years
later, we _STILL_ have poor SDK header files. Microsoft programmers should
hang their heads in shame.
After all both C and C++ standards make certain guarantees on header file
inclusion, why should it be necessary to do
#include <windows.h>
and suck in everything, because we cannot reliably include just the header
file where some Windows API is declared. Imagine if we have to write
#include <everything.h>
because we are unsure if just doing
#include <stdio.h>
is enough to make sure use of FILE works correctly with fgetc() etc.
Totally shameful
Stephen Howe
Code4u
If you specifically need platform portability then my advice would be
to isolate the platform independent code, do not try and make a single
code base work for multiple platforms. There are many problems with
win32 and you have hit upon a few, my advice to you would be to latch
on to a seasoned Windows developer (like me) and absorb his advice
like a sponge. Windows is tough for a newbie, even a smart newbie,
there's no need for you to rediscover old pitfalls. Regarding
makefiles, do youself a favor and use Visual Studio solution and
project files which provide higher level control over the build
process and can still be used from the command line.
Good luck with your project!
Hendrik Schober
Not to mention all the macros that pollute
the global namespace...
> Stephen Howe
Schobi
--
Spam...@gmx.de is never read
I'm Schobi at suespammers dot org
"Coming back to where you started is not the same as never leaving"
Terry Pratchett
Stephen Howe
> the global namespace...
Agreed
Those oh-so-brilliant, min(), max() macros have tripped up more programmers
than an undone shoelace.
Stephen Howe
Simon Trew
news:%230UPvuc...@TK2MSFTNGP15.phx.gbl...
> Imagine if we have to write
>
> #include <everything.h>
> because we are unsure if just doing
> #include <stdio.h>
> is enough to make sure use of FILE works correctly with fgetc() etc.
>
> Totally shameful
>
> Stephen Howe
Actually this technique has its uses. If FILE is only referenced (by
reference or pointer) in the header, one can avoid including the header file
by forward declaring it. There are pitfalls to this, e.g. you have to make
sure that FILE is indeed a struct/class and not a typedef, but it can be
useful to cut down unnecessary header-include cascades within a project and
so speed build times-- the header defining FILE is then included in the
appropriate .cpp file. This can cut down compile times on platforms that
aren't very clever about recognizing (or being told about) common header
inclusion patterns. It's especially useful when the header declares a class
that has smart pointer members-- generally one doesn't need to define (only
declare) the smart pointer type, though this may require declaration of a
copy constructor and assignment operator if the smart pointer makes any
calls on its wrapped object in its own copy constructor or assigment
operator.
I would not advocate this for headers that are used across projects, which I
agree should be 'complete' in the sense of declaring (via inclusion) all the
types that they need to be used, but within a project when there is total
control over where includes go, it can be useful.
It also cuts down on 'accidentally' including definitions through include
cascades, and so helps to ensure that a .h or .cpp file directly includes
all the stuff it relies on, which I think would generally be regarded as a
good thing. (Because you don't want your cpp to break just because a header
changes and no longer needs to include something that it did before.)
S.
Duane Hebert
"Stephen Howe" <sjhoweATdialDOTpipexDOTcom> wrote in message news:uzghtdsc...@TK2MSFTNGP09.phx.gbl...
Anyone using boost probably knows about this.
I'm not a big fan of all the typedefs either. What's
wrong with calling an unsigned char an unsigned char? <g>