ZwWriteFile/ZwReadFile and APCs
Dale Larson
Hi,
I'm having crash problems in my driver that have me scratching my head.
I have a system thread that is responsible for performing the IRP_MJ_READ /
WRITE operations for the driver using ZwWriteFile or ZwReadFile. When the
device dispatch routine gets an IRP which requests a read or write it queues
the IRP to a list managed by a routines ExInterlockedInsertTailList() and
ExInterlockedRemoveHeadList(). The Irps are linked together by the
Irp->Tail.Overlay.ListEntry field of the Irps. A semaphore is used to signal
the thread to process the Irp:
:
IoMarkIrpPending(Irp);
ExInterlockedInsertTailList(&pDevExt->IrpQueueListHead,
&Irp->Tail.Overlay.ListEntry, &pDevExt->IrpQueueSpinLock);
KeReleaseSemaphore(&pDevExt->IrpQueueSemaphore, 0, 1, FALSE);
return STATUS_PENDING;
The thread was created with the following call:
ntStatus = PsCreateSystemThread(&ThreadHandle, (ACCESS_MASK)0,
NULL, (HANDLE)0, NULL, FileDiskThreadMain, pDevExt);
When the thread starts it sets its priority to LOW_REALTIME_PRIORITY.
The thread removes the Irp using the following code:
:
KeWaitForSingleObject(&pDevExt->IrpQueueSemaphore, Executive, KernelMode,
FALSE, NULL);
//NOTE: CRASH OCCURS HERE WHEN ACCESSING pDevExt. CRASH CONDITIONS
// ARE DESCRIBED LATER IN THE POSTING.
ListEntry = ExInterlockedRemoveHeadList(&pDevExt->IrpQueueListHead,
&pDevExt->IrpQueueSpinLock);
Irp = CONTAINING_RECORD(ListEntry, IRP, Tail.Overlay.ListEntry);
:
I should note that these pieces of code where taken from the Art Baker's book:
"The Windows NT Device Driver Book".
After some messin' around I make a call to ZwWriteFile or ZwReadFile like so:
ntStatus = ZwReadFile(pDevExt->hDiskFile, NULL, XXRWApcRoutine, Irp, IoStatus,
pBfrAddr, pIrpStack->Parameters.Read.Length, &ByteOffset, NULL);
The APC routine is declared thusly:
VOID XXRWApcRoutine(IN PVOID ApcContext,
IN PIO_STATUS_BLOCK IoStatusBlock, IN ULONG Reserved)
{
PIRP Irp = (PIRP)ApcContext;
Irp->IoStatus = *IoStatusBlock; // Xfer status
if (NT_SUCCESS(IoStatusBlock->Status))
IoCompleteRequest(Irp, IO_DISK_INCREMENT);
else
IoCompleteRequest(Irp, IO_NO_INCREMENT);
}
Everything works just fine unless the "return STATUS_PENDING" occurs BEFORE
the APC runs. If this happens, my device extension pointer stored on my
thread's stack gets trashed. this causes an unhandle exception error
(0x0000001E).
I've created a trace of what's going on using DbgPrint's:
----------------------
:
:
TT ...... IRP is at: 0x8059B3A8
TT >>>>>> FDReadWrite: Call ZwReadFile(0x8059B3A8: 0x00000000, 0x00000800)
AA >>>>>> FileDiskRWApcRoutine: Irp=0x8059B3A8 Status=0x00000000,
Info=0x00000800)
TT <<<<<< FDReadWrite: ZwReadFile returned 0x00000000
TT ...... FileDiskThreadMain: Enter IRP wait.
TT ...... Device Extension is : 0x80592628
DD <-<-<- FDReadWrite: IRP Queued. Returning STATUS_PENDING.
DD ...... Unknown IOCTL = 7C008
DD ...... Current Process/Thread = 80591DE0/80591020
DD ->->-> FDReadWrite: Queue IRP_MJ_WRITE (0x804E34C8: 0x00000000, 0x00006800)
DD ..@@.. PreInsert: LISTDUMP: [0x80592680] -> EMPTY
DD ..@@.. PostInsert: LISTDUMP: [0x80592680] -> 0x804E3520.
TT ...... FileDiskThreadMain: Leave IRP wait.
TT ...... Device Extension is : 0x80592628
TT ...... FileDiskThreadMain: Removing IRP from Queue.
TT ...... Device Extension is : 0x80592628
TT ..@@.. PreRemove: LISTDUMP: [0x80592680] -> 0x804E3520.
TT ...... Device Extension is : 0x80592628
TT ..@@.. PostRemove: LISTDUMP: [0x80592680] -> EMPTY
TT ...... IRP is at: 0x804E34C8
TT >>>>>> FDReadWrite: Call ZwWriteFile(0x804E34C8: 0x00000000, 0x00006800)
TT <<<<<< FDReadWrite: ZwWriteFile returned 0x00000103
TT ...... FileDiskThreadMain: Enter IRP wait.
TT ...... Device Extension is : 0x80592628
DD <-<-<- FDReadWrite: IRP Queued. Returning STATUS_PENDING.
AA >>>>>> FileDiskRWApcRoutine: Irp=0x804E34C8 Status=0x00000000,
Info=0x00006800)
DD ...... Current Process/Thread = 80591DE0/80591020
DD ->->-> FDReadWrite: Queue IRP_MJ_READ (0x804E34C8: 0x00000000, 0x00006800)
DD ************* CONDITION TRIGGERED?
DD ..@@.. PreInsert: LISTDUMP: [0x80592680] -> EMPTY
DD ..@@.. PostInsert: LISTDUMP: [0x80592680] -> 0x804E3520.
TT ...... FileDiskThreadMain: Leave IRP wait.
PS: Unhandled Kernel Mode Exception Pointers = fc8cbba4
Code c0000005 Addr feeb587b Info0 0 Info1 67cc Info2 67cc Info3 805918a0
DbgPrint Legend: TT=Thread code, DD=Driver Dispatch code, AA=APC code.
----------------------
Please pardon my "creative" use of ">->->", etc. character strings I used to
get a quick eyeball scan of what's going on.
I've been sweating this problem for several days. I've hit a brick wall and
could use some expert opinions.
Dale Larson
Sterling Software
Gary G. Little
First, remember Art's book tain't perfect. It has several errors and I
would look at his code examples with a jaundiced eye. I've spent several
hours chasing code problems because his code example that I was using left
out some ... rather critical ... formal prarmeters in a function call.
Using his book as a second source to supplement the MSDN and DDK is good,
but don't depend on it, swear by it, or bet the farm on it. :)
I haven't created a thread, but I have chased down many c0000005
(STATUS_ACCESS_VIOLATION) errors in the DD's that I have developed.
When you say that the exception is thrown when pDevExt is accessed, was it
when the extension itself was accessed or when a structure it is pointing
to is accessed? The device extension should be created when the device
object is created. If access to the extension itself is throwing an
exception, then did you allow IoCreateDevice to create the extension or did
you use ExAllocXxx to create it? The extension when created by
IoCreateDevice is non-paged memory.
Check to make sure your extension allocation size is computed correctly.
This one bit me because I had sized it to a pointer to a struct (4 bytes),
and not to the structure (many more bytes than 4). Things would actually
work for a while, then explode at strange places.
What is the return value for KeWaitForSingleObject? Are you sure want a
timeout of NULL? Maybe you need to wait till the object is signalled. Or
maybe you need to set a timeout value. Also did you initialized
pDevExt->IrpQueueSemaphore (DriverEntry), and then force it to a known
state before waiting for the state to change (such as KeSetEvent or
KeResetEvent)?
Bracket the offending code with a try-except block, or try-except-finally
block. You then "try", but have an escape route incase a pointer is not
yet valid, handle the exception yourself and not get a BSOD.
Sorry if these sound trite, or duplicates of things you have done. But,
remember, it also keeps it in the thread and if I am TOTALLY wrong Jamie or
Paul will pop up, slap my hand, and may then give you insight into NT.
(Ulterior motives :))
(Are you in the Dallas office? Say hi to Mike Davis and Keith Brasch if
you are.)
Dale Larson wrote in article <33d0925b...@news.execpc.com>...
Dale Larson
Hi Gary,
>First, remember Art's book tain't perfect. It has several errors and I
>would look at his code examples with a jaundiced eye. I've spent several
>hours chasing code problems because his code example that I was using left
>out some ... rather critical ... formal prarmeters in a function call.
>Using his book as a second source to supplement the MSDN and DDK is good,
>but don't depend on it, swear by it, or bet the farm on it. :)
Point taken. ;)
>I haven't created a thread, but I have chased down many c0000005
>(STATUS_ACCESS_VIOLATION) errors in the DD's that I have developed.
>
>When you say that the exception is thrown when pDevExt is accessed, was it
>when the extension itself was accessed or when a structure it is pointing
>to is accessed? The device extension should be created when the device
>object is created. If access to the extension itself is throwing an
>exception, then did you allow IoCreateDevice to create the extension or did
>you use ExAllocXxx to create it? The extension when created by
>IoCreateDevice is non-paged memory.
It's the access to the device extension element itself that causes the
problem.
>Check to make sure your extension allocation size is computed correctly.
>This one bit me because I had sized it to a pointer to a struct (4 bytes),
>and not to the structure (many more bytes than 4). Things would actually
>work for a while, then explode at strange places.
Wow! Neat bug. Lets see....nope the sizeof() is for the actual struct. I will
remember you mentioning it.
>What is the return value for KeWaitForSingleObject? Are you sure want a
>timeout of NULL? Maybe you need to wait till the object is signalled. Or
>maybe you need to set a timeout value. Also did you initialized
>pDevExt->IrpQueueSemaphore (DriverEntry), and then force it to a known
>state before waiting for the state to change (such as KeSetEvent or
>KeResetEvent)?
Hmm.... I don't have a KeWaitForSingleObject. Once the ZwWriteFile/ZwReadFile
call is issued I loop back and check for the next IRP to process. The thread
waits on an Irp queue sychronized via a semaphore. The semophore was
initialized when the driver was created.
>Bracket the offending code with a try-except block, or try-except-finally
>block. You then "try", but have an escape route incase a pointer is not
>yet valid, handle the exception yourself and not get a BSOD.
Good tip.
>Sorry if these sound trite, or duplicates of things you have done. But,
>remember, it also keeps it in the thread and if I am TOTALLY wrong Jamie or
>Paul will pop up, slap my hand, and may then give you insight into NT.
>(Ulterior motives :))
Not at all. A guy's gotta slither before he crawls...<G>
>(Are you in the Dallas office? Say hi to Mike Davis and Keith Brasch if
>you are.)
Nope. I work for the Iowa office (formerly data access division..boy I wish
they'd quit reorganizing the company). I'm located in Wisconsin and work out
of my house.
Dale Larson
Sterling Software