void func(void)
{
static LONG Init = 0;
if (InterlockedOr(&Init, 1) == 0)
{
...do one time stuff here...
}
}
The problem is the inner code executes every single time the function
is invoked. I traced it down to a bug in the compiler. I then
recompiled the driver using the Visual Studio 2005 compiler and the
problem was fixed. I have attached the code each generated below. I'm
now concerned about shipping any future drivers compiled using the DDK
build environment because I don't know what all functions besides this
particular example would generate corrupt code.
;------------------------------------;
; Windows 2003 DDK SP1 (3790.1830)
;------------------------------------;
mov ecx, 1
mov edx, OFFSET FLAT:?Init
mov eax, DWORD PTR [edx]
$L14033:
mov esi, eax
or esi, ecx
lock cmpxchg DWORD PTR [edx], esi
jne $L14033
jne $L14011
;------------------------------------
; Visual Studio 2005
;------------------------------------
mov ecx, 1
mov edx, OFFSET ?Init
mov eax, DWORD PTR [edx]
$LN4
mov esi, eax
or esi, ecx
lock cmpxchg DWORD PTR [edx], esi
jne SHORT $LN4
test eax, eax ; <---------fixed
jne SHORT $LN2
>Hi guys, I ran into a compiler bug in the latest ddk that is very
>serious. I have a piece of code I want to execute exactly one time, but
>discovered it gets run every single time. It is written as follows:
>
>
>void func(void)
>{
> static LONG Init = 0;
>
> if (InterlockedOr(&Init, 1) == 0)
> {
> ...do one time stuff here...
> }
>}
>
>
>The problem is the inner code executes every single time the function
>is invoked. I traced it down to a bug in the compiler. I then
>recompiled the driver using the Visual Studio 2005 compiler and the
>problem was fixed. I have attached the code each generated below. I'm
>now concerned about shipping any future drivers compiled using the DDK
>build environment because I don't know what all functions besides this
>particular example would generate corrupt code.
Wait. I'm just learning how to write Windows drivers, but the
description of InterlockedOr() in the DDK I have says it should return
the original value of Init. With the setup above, isn't the original
value of Init always 0, so isn't executing the inner code always the
expected behavior?
>Hi guys, I ran into a compiler bug in the latest ddk that is very
>serious. I have a piece of code I want to execute exactly one time, but
>discovered it gets run every single time. It is written as follows:
>
>
>void func(void)
>{
> static LONG Init = 0;
>
> if (InterlockedOr(&Init, 1) == 0)
> {
> ...do one time stuff here...
> }
>}
>
>
>The problem is the inner code executes every single time the function
>is invoked. I traced it down to a bug in the compiler. I then
Oh, every time. Never mind. I see, static variable. That kind of
sucks.
Of course not, but the new WDK compiler is on the line with VS2005
( ver. 14.0.x ) - so it may have this bug fixed as well.
--PA
Serious? A showstopper? I think not. In 10 years of writting driver code,
that is the first time I have ever seen InterlockedOr used. In that time
frame I've found other compiler anomalies. In the 30 years I've been doing
this I've found them in every compiler or assembler I've ever used. I simply
adjust for them, or fix them if I am responsible for the build tools.
But thank you for publishing your findings, I'll be sure to not use
InterlockedOr, and use one of a dozen other ways to do the same thing.
--
The personal opinion of
Gary G. Little
<440...@email.com> wrote in message
news:1153830650....@h48g2000cwc.googlegroups.com...
>Serious? A showstopper? I think not. In 10 years of writting driver code,
>that is the first time I have ever seen InterlockedOr used. In that time
>frame I've found other compiler anomalies. In the 30 years I've been doing
>this I've found them in every compiler or assembler I've ever used. I simply
>adjust for them, or fix them if I am responsible for the build tools.
That's interesting. In my 10 years of writing different sorts of C
and assembly code I've never found a code generation bug in any of the
compilers or assemblers I've used -- I've heard they might exist, but
I've never personally found one. Most of the problems I've seen have
come from bad assumptions in code (mine and others'), and the rare
compiler bugs have fortunately caused builds to fail in obvious ways
instead of producing runnable code that fails in subtle ways.
The bug itself is interesting. Is InterlockedOr() supposed to serve
as a memory barrier and make no assumptions about the previous state
of it's variable? Is a workaround (but not the normal convention) to
declare the variable as volatile?
It is used by DDK examples so likely to be in MS and 3rd party drivers.
> I'll be sure to not use InterlockedOr
While an instance of incorrect code generation was discovered near this
function, it seems a bit reckless to draw conclusions as to the scope
of the problem. Unless there is an official statement to the contrary,
I have no choice but to assume code generation problems could exist
with other functions or even ordinary code. You'll never know how nasty
this problem was. Let's just say a very far away object failed because
of it. Because of the way it damaged the kernel stack, the cpu triple
faulted and there was absolutely nothing that could be done to recover.
Some of the code we generate can't be corrected after deployment and
absolutely must work so we take this problem seriously enough to have
discontinued all use of the ddk compiler. We actually discovered this
problem and made this decision some time ago, but for political reasons
it couldn't be disclosed up to now.
The operation of such a simple if() is something that can be tested
early in development, if not with external testing then by the
developer with some debug statements or a debugger, similar to
purposely sending invalid data through a sanity-check and observing
that the expected error path is followed. The device driver world
needs more anal retentiveness.
(not that a compiler bug isn't annoying)
Umm, you are judging a minimal example whittled down for the purposes
of illustration. It is furthermore noteworthy you go into the 'if'
normally--the first time. So don't try to draw conclusions about lack
of code coverage or testing as you have no idea of the code or the
chain reaction of events that had to take place. Closer to home, it is
interesting there are examples in the ddk that use InterlockedOr of 1
and rely on the result.
This was reported in ntdev by Charlie Suffin (aka Chazmo) from egenera
back in May.
http://www.osronline.com/showThread.cfm?link=92466
The bug appears to be real and appears to be a defect in the intrinsic
code emitted for the little used InterlockedOr function.
>> The operation of such a simple if() is something that can be tested
>
>Umm, you are judging a minimal example whittled down for the purposes
>of illustration. It is furthermore noteworthy you go into the 'if'
>normally--the first time. So don't try to draw conclusions about lack
The first time isn't noteworthy. It's the normal case. The normal
case is what works most of the time, what's the easiest to test and
debug.
>of code coverage or testing as you have no idea of the code or the
>chain reaction of events that had to take place. Closer to home, it is
>interesting there are examples in the ddk that use InterlockedOr of 1
>and rely on the result.
I'm sure you did all sorts of testing, and I'm sure there was a subtle
chain reaction. My point is only that whoever made InterlockedOr()
isn't completely to blame (maybe 95% to blame).
It's used in this manner in nearly all, if not all, of the
WRITE_REGISTER_XXXX functions for IO access. At no time do I see it's
returned value used or even saved away; it's ignored. The results of the
atomized OR is used, Mr. Muscle Car, but not the return value of the
function.
I still do not think it is the proverbial smoking gun that justifys the use
of anything but the DDK compiler for driver development. I also note a few
comments in DDK header files where it looks like a few issues with
interlocked functions were known and being addressed. I'm sure given close
enough scrutiny, any compiler du jour will have the same kind of errors. The
authors are, after all, just like you ... an error prone ten toed bi-ped.
--
The personal opinion of another one of those error prone ten toed bi-peds
Gary G. Little
<440...@email.com> wrote in message
news:1153848684.9...@m73g2000cwd.googlegroups.com...
--
-Eliyas
This posting is provided "AS IS" with no warranties, and confers no rights.
http://www.microsoft.com/whdc/driver/tips/default.mspx
"Gary G. Little" <gary.g...@seagate.com> wrote in message
news:44c66330$0$5426$8826...@news.teranews.com...
--
The personal opinion of
Gary G. Little
<BubbaGump> wrote in message
news:dulcc21pu60sq9jlk...@4ax.com...
Interlocked operations imply the necessary memory barriers:
----------
http://windowssdk.msdn.microsoft.com/en-us/library/ms686355.aspx
The following synchronization functions use the appropriate barriers to
ensure memory ordering:
- Functions that enter or leave critical sections
- Functions that signal synchronization objects
- Wait functions
- Interlocked functions
-------------
Why not just fix the compiler since it is not generating correct code?
Regards,
George.
"Eliyas Yakub [MSFT]" <eli...@online.microsoft.com> schrieb im Newsbeitrag
news:OfJlCeCs...@TK2MSFTNGP04.phx.gbl...
The next question is whether Microsoft takes illegal advantage of using
fixed tools internally while delivering broken tools to everyone else. I
bet we won't get an answer to that one.
"RossettoeCioccolato" <gmga...@newsgroup.nospam> wrote in message
news:O$HSVwGsG...@TK2MSFTNGP03.phx.gbl...
--
Ken Johnson (Skywing)
Windows SDK MVP
"RossettoeCioccolato" <gmga...@newsgroup.nospam> wrote in message
news:O$HSVwGsG...@TK2MSFTNGP03.phx.gbl...
--
Ken Johnson (Skywing)
Windows SDK MVP
"Norman Diamond" <ndia...@community.nospam> wrote in message
news:%23wj$7%23GsGH...@TK2MSFTNGP03.phx.gbl...
--
Ken Johnson (Skywing)
Windows SDK MVP
"Norman Diamond" <ndia...@community.nospam> wrote in message
news:%23wj$7%23GsGH...@TK2MSFTNGP03.phx.gbl...
Maybe you missed this in the toaster & pcidrv:
oldWakeState = InterlockedOr( (PULONG)&FdoData->WakeState, 1 );
> Mr. Muscle Car
Ha, thanks...someone knows their cars!
Does Microsoft recommend that customers should use the beta WDK instead of
the released 2003 DDK?
During more than 2 years that passed between Microsoft fixing this bug in
May 2003 and Microsoft making available the beta WDK, while customers
couldn't use the bug fix, did Microsoft also refrain from using the bug fix?
When Windows crashes due to drivers that Microsoft put in the Windows 2000
CD and Microsoft said that Microsoft provided and Microsoft signed with
Microsoft provisioning, was it wrong for customers to blame Microsoft?
When Windows crashes due to drivers that customers downloaded or received on
CDs from third parties and the BSODs say that those third party drivers are
to blame, maybe aren't there a few customers who blame those third party
drivers?
I misattributed blame to Microsoft one time because I thought I was only
using Microsoft built-in drivers for a floppy drive and I was unaware that
an anti-virus vendor had sneaked another driver into the stack. That is
outnumbered by the times that I and others had misattributed blame to other
vendors for bugs that turned out to be Microsoft's. On the other hand, that
is equal to the number of times that a customer blamed me for overwriting a
customer's file when Microsoft's Common Dialog Box lied about the file it
was selecting to be written.
"Skywing [MVP]" <skywing_...@valhallalegends.com> wrote in message
news:uMVaOLHs...@TK2MSFTNGP03.phx.gbl...
Then provide the knowledge. There was an if() with two possible
outcomes. Were both outcomes tested?
Yeah, I know, it's impossible to test every combination of every case.
I'm saying it's possible to test at least every case by itself (every
if(), every loop) within a module for which there is source, at least
once. It can be boring, I guess. It depends what entertains you.
Beverly
"Eliyas Yakub [MSFT]" <eli...@online.microsoft.com> wrote in message
news:OfJlCeCs...@TK2MSFTNGP04.phx.gbl...
>Is it just InterlockedOr or are the return values of all the interlocked
>functions suspect?
>
It seems to be only interlockedor.
=====================
Mark Roddy DDK MVP
Windows Vista/2003/XP/2000 Consulting
Device and Filesystem Drivers
Hollis Technology Solutions 603-321-1032
www.hollistech.com
--
Ken Johnson (Skywing)
Windows SDK MVP
"Mark Roddy" <ma...@hollistech.com> wrote in message
news:7o4kc2hc8e4mflo9i...@4ax.com...
It appears InterlockedXor generates incorrect code as well. Perhaps the
bug can appear anytime the compiler emits a cmpxchg instruction. Trying
to pinpoint all the possible things that can cause the DDK compiler to
generate bad code and wasting even more time rewriting source in
suboptimal ways and then dealing with the test hit is unnecessary,
error prone, and time consuming. Nor is it prudent to build code off
the WDK since it is beta. In our assessment, building off VS 2005 is in
the best interests of the customer until which time Microsoft offers a
patch to the DDK.
Using 2005 for driver development, simply because of two rather obscure
functions, is really not logical, at least to myself and my peers that
develop drivers for our organization. True, the 2005 compiler resolves those
two functions for you, but there are workarounds for the DDK compiler, and
using the 2005 compiler opens you up to a whole host of bugs that have not
even been found yet, and may or may not affect your driver. I think Don
pointed it out: the main reason for using the DDK compiler for development
is because it was used to develope the core components for the OS, and
therefore any driver you develope with it is in line with those components.
No one ever said the DDK compiler was bugless. Can you say the same for the
2005 compiler?
--
The personal opinion of
Gary G. Little
<440...@email.com> wrote in message
news:1154150164.4...@p79g2000cwp.googlegroups.com...
To rephrase this as a medical problem "Gee you have two warts, your solution
while it is known to eliminate the warts, has been know to induce cancer and
other fatal diseases". I don't think you would buy into the treatment
medically, you should not buy in VS2005 either.
--
Don Burn (MVP, Windows DDK)
Windows 2k/XP/2k3 Filesystem and Driver Consulting
http://www.windrvr.com
Remove StopSpam from the email to reply
To answer you question, yes, you can.
There will be many people here (some experts) who will tell you not to.
Do as you feel appropriate.
Set the Linker /SUBSYSTEM:NATIVE and make /ENTRY:DriverEntry
If you leave off /WDM and don't initialize pDriverObject->AddDevice
pointer (in DriverEntry) then your driver will be regarded as a
"legacy" non-PNP NT-type driver, loadable as a driver service using
StartService, etc.
There are other things like making sure you are #includ'ing the right
headers and linking in the right .LIBS, but that's the fun part.
I build my driver using VS 2005 IDE with no problems.
-Le Chaud Lapin-
> [Gary writes] I'll be sure to not use InterlockedOr, and
> use one of a dozen other ways to do the same thing.
>> [Gary later writes] Oh gosh, you have found 2, rarely
>> used functions where the DDK compiler has a glitch
You first assumed the problem was limited to a function it was found
around. I called this a reckless assumption and alas, it is now known
to affect more. Failing to see the error in forming your assessment,
you snap back that it impacts 2 functions yet we are already up to 3 in
this very thread. We don't know what all code will trigger it yet;
just a few examples so far so why jump to conclusions? As the impacted
functions grow, so does the exposure. We don't even know if the
interlocks are the only ones.
> [Gary writes] Serious? A showstopper? I think not. In 10 years of writing
> driver code, that is the first time I have ever seen InterlockedOr used.
>> [Gary later writes] Oh gosh, you have found 2, rarely used functions
>> where the DDK compiler has a glitch
You first downplay the severity because you had never seen
InterlockedOr used. It was pointed out it is used so commonly that it
is in a number of examples in the latest ddk. Correct me if I am wrong,
but this represents orders of magnitude more significance than your
experience. ANY functions used in the examples are VERY significant
because these samples are intended for and widely used for the basis of
almost every single new driver. And knowing this you still have the
gall to call them rarely used functions.
> [Gary writes] in all instances in the kits where it is used in code,
> the returned variable is NOT used.
>> [Gary later writes] the particualr functionality is normally not used;
>> e.g. the returned value in this case not normally not used
You repeatedly claim the examples don't use the result, but they do:
"if (WAKESTATE_ARMED == oldWakeState)". That sure looks like a used
result of the preceding line: oldWakeState = InterlockedOr(
(PULONG)&FdoData->WakeState, 1 );
> [Gary writes] Does the VS compiler have equivalent bugs?
> I am very sure it does.
>> [Gary later writes] using the 2005 compiler opens you up to a whole host
>> of bugs that have not even been found yet
You claim there are bugs in VS 2005 code generation that could result
in faulty drivers when you don't even know of a single one. If you do
find one, you better report it to the WDK beta feedback program since
our upcoming officially blessed WDK compiler is the same vintage as VS
2005. Otherwise if I trip over it I'll make a thread about bugs in the
WDK compiler forcing me to use VS 2007, LOL.
For the record, I believe there are *at least* 3 major problems, not 2
minor ones. And correct me if I am wrong, but the true scope of the
problem is an unknown. So far, people like you have been guessing
(incorrectly at that) based on how many functions you have seen so far
that exhibit it. If we don't know the extent of the problem we cannot
work around it. 3 years after fixing the compiler, MS hasn't so much as
put up a knowledgebase article acknowledging it exists and how to avoid
it let alone given a patch.
> VS2005 can cause serious bugs in drivers, been there done that
> chasing a bug because a customer use 2005 instead of the DDK!
So you have personal knowledge of not just one, but multiple serious
bugs that effect drivers. Care to post just one here (if they really
exists that is)? And "I recompiled with the DDK and it went away"
doesn't count since it proves nothing. If you are worth your salt as a
driver engineer you will pinpoint the exact bug in the compiler or
linker and post a code fragment that exhibits it. If in fact the
problem is setup related or incompliant code breaking due to the
improved optimizer, then I'd say that's your bug, not the compilers.
For my purposes, the DDK compiler generates bad drivers and VS 2005
generates good ones. But I'm happy for other people to share their
knowledge in this regard.
Care to share specific examples of those VC++ 2005 compiler bugs that
cause the buggy drivers you mentioned and are not in DDK's compiler?
I know there must be some (or evn lots of) bugs in both compilers, and I
think many of us like to know those know bugs in either of them.
Thank you,
Isaac Chen
I'm not surprised you found 3 bugs. I'm very sure there are others, but at
the same time I have absolutely no doubt but that 2005 has it's own set of
bugs. Cripes ... the 2005 IDE is rank with them so do you really want to
continue the logic that because it hasn't the three found in the DDK that it
is "bugfree", or even has fewer? I and the team I work with have 10 of
thousands of lines of code currently running, all compiled with the DDK
compiler. I highly doubt we'll be changing to any other until WDK reaches
RTM.
If history is repeated, the WDK will most likely be released with an updated
version of the 2005 compiler, and it will then be frozen and be the
recommended compiler for driver developement. Will it be bug free? Nope, of
that I am sure.
So as I said ... thanks for the heads up on the problems with the
Interlocked commands, but we won't be changing compilers.
--
The personal opinion of
Gary G. Little
I have a customer who has built their drivers with 2005, we had a number of
blue screens that were not reproducable with the DDK compiler. At least one
case, the optimizer removed incorrectly part of a pointer calculation,
causing a deref to non-existent memory. This does not appear to be the case
for the other faults (there were multiple) since we did not find the same
problem.
Please let us know which products you are building drivers for with 2005 so
we can all be sure not to buy them.
Not incriminating in itself. When migrating to a new compiler,
typically problems turn out to be code issues, not compiler bugs.
> At least one case, the optimizer removed incorrectly part of a
> pointer calculation,causing a deref to non-existent memory.
Please post a code snippet that exhibits the problem so it can be
verified--a lot of so called compiler bugs aren't. I could easily see
this being a coding mistake where someone simply overlooked a promotion
rule. If you don't care to post anything then I think a lot of people
will draw the obvious conclusion.
--
Don Burn (MVP, Windows DDK)
Windows 2k/XP/2k3 Filesystem and Driver Consulting
http://www.windrvr.com
Remove StopSpam from the email to reply
<440...@email.com> wrote in message
news:1154446520.6...@m79g2000cwm.googlegroups.com...
You may as well have said the dog ate it. You can't be bothered to
whittle the problem down to a few generic lines of code that illustrate
it? Being on standards committees somehow reduces the need for
scrutiny? Hardly. It is disappointing when someone barges in barking
baseless claims and when called into question is unable/unwilling to
provide any appropriate supporting facts and leaves yelping with their
tail between their legs. I'd wager most people have reached the obvious
conclusion to these claims, correctly or not.
Bottom line, the customer sent it to Microsoft with a bug report, but I
cannot share it with you. I have worked on compilers on and off for the
last 30 years. I have been a C language developer for 30 years, and yes I
was involved with the C++ standard.
If you don't want to trust me fine, but I don't want to trust your code, so
please identify yourself and your products so those of us who believe in
using the approved tools can avoid your code.
--
Don Burn (MVP, Windows DDK)
Windows 2k/XP/2k3 Filesystem and Driver Consulting
http://www.windrvr.com
Remove StopSpam from the email to reply
<440...@email.com> wrote in message
news:1154508907.1...@m73g2000cwd.googlegroups.com...
http://blogs.msdn.com/mlacey/archive/2006/08/01/686098.aspx
no need to cut off the nose to spite the face ;)
d
--
Please do not send e-mail directly to this alias. this alias is for
newsgroup purposes only.
This posting is provided "AS IS" with no warranties, and confers no rights.
<440...@email.com> wrote in message
news:1154150164.4...@p79g2000cwp.googlegroups.com...
--PA
"Doron Holan [MS]" <dor...@nospam.microsoft.com> wrote in message news:eeNTCZ2t...@TK2MSFTNGP05.phx.gbl...
> mark lacey,a dev on the compiler team, has blogged about this bug and the workarounds for it.
>
> http://blogs.msdn.com/mlacey/archive/2006/08/01/686098.aspx
>
> no need to cut off the nose to spite the face ;)
>
> d
>
> --
>
To directly answer your question ... if the next DDK, SP or otherwise,
includes an up rev'd compiler then that will be the recommended compiler for
driver development. Please note that I said RECOMMENDED, and not BEST.
--
The personal opinion of
Gary G. Little
"Pavel A." <pav...@NOwritemeNO.com> wrote in message
news:%23nOVjI%23tGH...@TK2MSFTNGP05.phx.gbl...
No need for us to cut off noses when that's Microsoft's job.
No problem that Mr. Lacey prohibits anonymous comments, I logged in in order
to post something approximating the following:
> When reported to us, these bugs are generally treated as must-fix
> right up until the last few weeks before shipping a product.
If you know that this bug is a "must fix" then you know that this bug
is a "must fix". If it's found during the last few weeks before
shipping then you MUST announce a hotfix and make the hotfix available
for downloads. If it's found after shipping then you MUST announce a
hotfix and make the hotfix available for downloads.
But even this opinion was too far out, either for the sensibilities of
Microsoft or of Mr. Lacey, I'm not sure which. Though actually we do know
the Microsoft's policy, since Microsoft made a fix but sat on it for years
instead of releasing it.
Must fix but must not make the fix available. Wow, what a useful fix.
What a contribution to the reliability of Microsoft's products and to
Microsoft's sometimes asserted goal of delighting customers.
--PA
"Norman Diamond" <ndia...@community.nospam> wrote in message news:%23JlH4np...@TK2MSFTNGP05.phx.gbl...
"440...@email.com" wrote:
> Hi guys, I ran into a compiler bug in the latest ddk that is very
> serious. I have a piece of code I want to execute exactly one time, but
> discovered it gets run every single time. It is written as follows:
>
>
> void func(void)
> {
> static LONG Init = 0;
>
> if (InterlockedOr(&Init, 1) == 0)
> {
> ...do one time stuff here...
> }
> }
>
>
> The problem is the inner code executes every single time the function
> is invoked. I traced it down to a bug in the compiler. I then
> recompiled the driver using the Visual Studio 2005 compiler and the
> problem was fixed. I have attached the code each generated below. I'm
> now concerned about shipping any future drivers compiled using the DDK
> build environment because I don't know what all functions besides this
> particular example would generate corrupt code.
>
>
>
> ;------------------------------------;
> ; Windows 2003 DDK SP1 (3790.1830)
> ;------------------------------------;
> mov ecx, 1
> mov edx, OFFSET FLAT:?Init
> mov eax, DWORD PTR [edx]
> $L14033:
> mov esi, eax
> or esi, ecx
> lock cmpxchg DWORD PTR [edx], esi
> jne $L14033
> jne $L14011
>
> ;------------------------------------
> ; Visual Studio 2005
> ;------------------------------------
> mov ecx, 1
> mov edx, OFFSET ?Init
> mov eax, DWORD PTR [edx]
> $LN4
> mov esi, eax
> or esi, ecx
> lock cmpxchg DWORD PTR [edx], esi
> jne SHORT $LN4
> test eax, eax ; <---------fixed
> jne SHORT $LN2
>
>
A global variable produces the same problem as a local static. However,
InterlockedCompareExchange (as well as InterlockedExchange) produced
good code under the same conditions. That is curious because the former
emits the cmpxchg instruction which has been the culprit up to now. So
still hard to define exactly what will trigger it.
Another interesting point is the problem is sensitive to the compiler
flags used. The flags set by the ddk build do trigger the problem
regardless if debug or release. So another hypothetical solution would
be to tweak the compiler flags though I haven't bothered to isolate
which one(s).
--PA
<440...@email.com> wrote in message news:1155876350....@p79g2000cwp.googlegroups.com...