problem with NdisReturnPackets ( )

[serge]

Hello, All!

I get BSOD on vmware 5.0 and some other networc cards.
It does not happen on other vmware versions and physical NICs.

The problem appears when I try to send previously queued packed. The same(?)
packets are sent before BSOD using the same functions, pools, etc without
problems.

On Windows 2000 Pro, mydriver rises D2 bugcheck somewhere in
ethFilterDprIndicateReceivePacket (instead of D1 one in XPsp2 system, see
below).

Does anybody know why this may happen?

According to
http://pcausa.com/support/KB05050101.htm
I use separate pools for send and receive packets.

Also, I tried to zero Packet->MacReserved field. No luck.

Any thoughts about this would be greatly appreciated.

Best regards,
Serge.

############################################################################
#########

*** Fatal System Error: 0x000000d1
(0x00000008,0x00000002,0x00000000,0xF9889848)

Break instruction exception - code 80000003 (first chance)

A fatal system error has occurred.
Debugger entered on first try; Bugcheck callbacks have not been invoked.

A fatal system error has occurred.

Connected to Windows XP 2600 x86 compatible target, ptr64 FALSE
Loading Kernel Symbols
...................................................................
Loading unloaded module list
...
Loading User Symbols
****************************************************************************
***
*
*
* Bugcheck Analysis
*
*
*
****************************************************************************
***

Use !analyze -v to get detailed debugging information.

BugCheck D1, {8, 2, 0, f9889848}

Probably caused by : mydriver.sys ( mydriver!MPReturnPacket+73 )

Followup: MachineOwner
---------

nt!RtlpBreakWithStatusInstruction:
804e3b25 cc int 3
kd> !analyze -v
****************************************************************************
***
*
*
* Bugcheck Analysis
*
*
*
****************************************************************************
***

DRIVER_IRQL_NOT_LESS_OR_EQUAL (d1)
An attempt was made to access a pageable (or completely invalid) address at
an
interrupt request level (IRQL) that is too high. This is usually
caused by drivers using improper addresses.
If kernel debugger is available get stack backtrace.
Arguments:
Arg1: 00000008, memory referenced
Arg2: 00000002, IRQL
Arg3: 00000000, value 0 = read operation, 1 = write operation
Arg4: f9889848, address which referenced memory

Debugging Details:
------------------

READ_ADDRESS: 00000008

CURRENT_IRQL: 2

FAULTING_IP:
NDIS!NdisReturnPackets+48
f9889848 8b7308 mov esi,[ebx+0x8]

DEFAULT_BUCKET_ID: DRIVER_FAULT

BUGCHECK_STR: 0xD1

LAST_CONTROL_TRANSFER: from f975f58f to f9889848

TRAP_FRAME: f9f028ec -- (.trap fffffffff9f028ec)
ErrCode = 00000000
eax=ffffffff ebx=00000000 ecx=00000002 edx=00000002 esi=81664518
edi=81669f30
eip=f9889848 esp=f9f02960 ebp=f9f02978 iopl=0 nv up ei pl zr na po
nc
cs=0008 ss=0010 ds=0023 es=0023 fs=0030 gs=0000
efl=00010246
NDIS!NdisReturnPackets+0x48:
f9889848 8b7308 mov esi,[ebx+0x8]
Resetting default scope

STACK_TEXT:
f9f02978 f975f58f f9f02998 00000001 817b1f30 NDIS!NdisReturnPackets+0x48
f9f029b4 f988987f 815ba3a0 817b1f30 81558280 mydriver!MPReturnPacket+0x73
[C:\Work\Driver\NDIS\miniport.c @ 598]
f9f029dc f9755061 f9f02a00 00000001 817b1f78 NDIS!NdisReturnPackets+0xe9
f9f029f4 f9896d09 815ac140 817b1f30 f975ab40 psched!MpReturnPacket+0x3b
f9f02a48 f975501d 00029d70 81685be0 00000001
NDIS!ethFilterDprIndicateReceivePacket+0x56d
f9f02a5c f97551b4 81558280 81685be0 00000001 psched!PsFlushReceiveQueue+0x15
f9f02a80 f97555f9 815ac148 00000000 81558280
psched!PsEnqueueReceivePacket+0xda
f9f02a98 f9896d40 815ac140 00000000 816b1120 psched!ClReceiveComplete+0x13
f9f02ae8 f9761fb4 00029d70 f9f02b08 00000001
NDIS!ethFilterDprIndicateReceivePacket+0x5a4
f9f02c60 f976ab31 815ba3a0 817b1f30 00000001
mydriver!PtReceivePacketEx+0x119 [C:\Work\Driver\NDIS\protocol.c @ 745]
f9f02ca0 f976ae93 815ba3a0 817b1f30 00000002
mydriver!Queue_Packet_ExecuteEx+0x85 [C:\Work\Driver\NDIS\queue_packet.c @
239]
f9f02cd8 f976b453 815ba3a0 81027720 00000000
mydriver!Queue_Packet_Execute_Item+0xef [C:\Work\Driver\NDIS\queue_packet.c
@ 270]
f9f02d30 f976b6fa 815ba3a0 81027720 00000000
mydriver!Queue_Packet_Execute+0x7b [C:\Work\Driver\NDIS\queue_packet.c @
331]
f9f02d74 f976bdd5 815ba3a0 00000000 816b1120
mydriver!Queue_Packet_Manage+0xa6 [C:\Work\Driver\NDIS\queue_packet.c @ 396]
f9f02dac 8057dfed 815ba3a0 00000000 00000000
mydriver!Queue_Packet_Thread+0xa5 [C:\Work\Driver\NDIS\queue_packet.c @ 505]
f9f02ddc 804fa477 f976bd30 815ba3a0 00000000 nt!PspSystemThreadStartup+0x34
00000000 00000000 00000000 00000000 00000000 nt!KiThreadStartup+0x16

FOLLOWUP_IP:
mydriver!MPReturnPacket+73 [C:\Work\Driver\NDIS\miniport.c @ 598]
f975f58f eb0d jmp mydriver!MPReturnPacket+0x82 (f975f59e)

SYMBOL_STACK_INDEX: 1

FOLLOWUP_NAME: MachineOwner

SYMBOL_NAME: mydriver!MPReturnPacket+73

MODULE_NAME: mydriver

IMAGE_NAME: mydriver.sys

DEBUG_FLR_IMAGE_TIMESTAMP: 42695faa

STACK_COMMAND: .trap fffffffff9f028ec ; kb

BUCKET_ID: 0xD1_mydriver!MPReturnPacket+73

Followup: MachineOwner
---------

############################################################################
#########

kd> !ndiskd.pkt 0x81669f30 4
NDIS_PACKET at 81669f30
NDIS_BUFFER at 81606120
Next 00000000
Size 20
MdlFlags c
Process 00000000
MappedSystemVa 814d3000
Start VA 814d3000
ByteCount 5ea
ByteOffset 0
MacReserved[]: 00000000 00000000 00000000 00000000
0. TcpIpChecksumPacketInfo = 00000000
1. IpSecPacketInfo = 00000000
2. TcpLargeSendPacketInfo = 00000000
3. ClassificationHandlePacketInfo = 00000000
4. NdisReserved = 0000000e
5. ScatterGatherListPacketInfo = 00000000
6. Ieee8021QInfo = 00000000
7. OriginalPacketInfo = 00000103
8. PacketCancelId = 00000000
9. OriginalNetBufferList = 00000000
10. CachedNetBufferList = 00000000
11. MaxPerPacketInfo = 00000000

Packet.Private
PhysicalCount 00000000 Total Length 00000000
Head 00000000 Tail 00000000
Pool 00000000 Count 00000000
Flags 00000000 ValidCounts 00
NdisPacketFlags 00000000 NdisPacketOobOffset 0000

Private.Flags : 00000000
Private.NdisPacketFlags: 80
fPACKET_ALLOCATED_BY_NDIS,

############################################################################
#########

kd> !ndiskd.pkt 0x81669f30 5
NDIS_PACKET at 81669f30
MDL = 81606120
StartVa ffffffff814d3000, ByteCount 0x5ea, ByteOffset 0x0, NB MdlOffset
0x0
814d3000: 00 0c 29 5e 75 ca 00 50 56 c0 00 01 08 06 00 01
814d3010: 08 00 06 04 00 02 00 50 56 c0 00 01 0a 01 00 01
814d3020: 00 0c 29 5e 75 ca 0a 01 00 07 00 00 00 00 00 00
814d3030: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
[zeros skipped]
814d35c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
814d35d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
814d35e0: 00 00 00 00 00 00 00 00 00 00

[msnews.microsoft.com]

Serge, you complain that your driver bugchecks when you send a previously
queued packet, but the stack you posted below is all about receive. Can you
explain what your driver is doing? is is an IM driver? do you "re-package"
the data in another packet before sending it (or indicating it) to the next
layer? Does the problem repro if you uninstall psched?

-ali

--
This posting is provided "AS IS" with no warranties, and confers no rights.

"serge" <pser...@ukr.net> wrote in message
news:d4bp8m$mm1$1...@news.dg.net.ua...

[Stephan Wolf [MVP]]

The address that you pass to the !ndiskd.pkt command does not seem to
appear on the stack anywhere. I can only guuess you took that value
from register EDI out of the trap frame.

However, the interesting part is the BX register here. And it is zero,
i.e. it contains a NULL pointer.

It would seem the address of the NDIS_PACKET is actually 0x817b1f30
(and the miniport context is 0x815ba3a0, or vice versa).

Please make sure the 'PacketsToReturn' you pass to NdisReturnPackets()
is actually "the address of the address" of the NDIS_PACKET. It seems
you actually pass some address of a stack variable to
NdisReturnPackets(), but make sure it is the address of the correct
NDIS_PACKET pointer.

Also, try and disable the packet scheduler protocol (psched) in your
network card's settings. Then see if the bugcheck still occurs.

Stephan
---

[Stephan Wolf [MVP]]

Just a note: The received packet in question seems to be an ARP reply
generated by VMware (both the source and destination MAC addresses
belong to VMware Inc.). Nothing comes to mind that would be special
about this kind of packet with respect to the bugcheck.

Stephan
---

[serge]

Hello, msnews.microsoft.com!
You wrote on Sun, 24 Apr 2005 01:47:15 -0700:

mmc> Serge, you complain that your driver bugchecks when you send a
mmc> previously queued packet, but the stack you posted below is all about
mmc> receive. Can you explain what your driver is doing? is is an IM
mmc> driver? do you "re-package" the data in another packet before sending
mmc> it (or indicating it) to the next layer? Does the problem repro if you
mmc> uninstall psched?

Sorry, that was received packet, of cause.
Yes, I re-allocate packet handle, and it's payload, queue it. Than I
indicate the packet, and call ndisReturnPacket for original one.

best regards,
Serge

[serge]

Hello, Stephan!

Thanks your your reply.

SWM> The address that you pass to the !ndiskd.pkt command does not seem to
SWM> appear on the stack anywhere. I can only guuess you took that value
SWM> from register EDI out of the trap frame.

Yes. But the packet is still not released as far as I understand.
BSOD happened at the beginning of execution of NDIS!NdisReturnPackets

SWM> However, the interesting part is the BX register here. And it is zero,
SWM> i.e. it contains a NULL pointer.

Actually, it is.
I can see "xor ebx,ebx" followed by "mov esi,[ebx+0x8]"
in the code, but its not clear for me why it may happen,
I am not assembler guru :)

f9889f54 83c9ff or ecx,0xffffffff
f9889f57 f00fc108 lock xadd [eax],ecx
f9889f5b e929f9ffff jmp NDIS!NdisReturnPackets+0xfc (f9889889)
f9889f60 ff15309e87f9 call dword ptr [NDIS!_imp_KfLowerIrql
(f9879e30)]
f9889f66 e943f9ffff jmp NDIS!NdisReturnPackets+0x152 (f98898ae)
f9889f6b 33db xor ebx,ebx <<==========
f9889f6d e9d6f8ffff jmp NDIS!NdisReturnPackets+0x48 (f9889848)
f9889f72 817a1412010100 cmp dword ptr [edx+0x14],0x10112
f9889f79 0f8562faffff jne NDIS!ndisMRequest+0x8e (f98899e1)
f9889f7f e9bb020000 jmp NDIS!ndisMRequest+0x5e (f988a23f)

[skipped]

f9889839 0f832c070000 jnb NDIS!NdisReturnPackets+0x46 (f9889f6b)
f988983f 2bc1 sub eax,ecx
f9889841 8d0440 lea eax,[eax+eax*2]
f9889844 8d5cc7f8 lea ebx,[edi+eax*8-0x8]
f9889848 8b7308 ===>> mov esi,[ebx+0x8] ds:0023:00000008=????????
f988984b 8d430c lea eax,[ebx+0xc]
f988984e 83c9ff or ecx,0xffffffff
f9889851 f00fc108 lock xadd [eax],ecx
f9889855 49 dec ecx

SWM> It would seem the address of the NDIS_PACKET is actually 0x817b1f30
SWM> (and the miniport context is 0x815ba3a0, or vice versa).

Exactly! Thanks for noticing it.

I modifyed PASSTHUGH driver from DDK, and the following function remained
the same in my driver.

VOID MPReturnPacket(IN NDIS_HANDLE MiniportAdapterContext, IN PNDIS_PACKET
hPacket)
{
PNDIS_PACKET MyPacket;
PRECV_RSVD RecvRsvd = (PRECV_RSVD)(hPacket->MiniportReserved);

MyPacket = RecvRsvd->OriginalPkt;
NdisFreePacket(hPacket);
NdisReturnPackets(&MyPacket, 1);
}

I can confirm MyPacket was passed to NdisReturnPackets.
But the packet handle you are talking about is hPacket.
It's a mistery... How hPacket came up there?...

SWM> Please make sure the 'PacketsToReturn' you pass to NdisReturnPackets()
SWM> is actually "the address of the address" of the NDIS_PACKET. It seems
SWM> you actually pass some address of a stack variable to
SWM> NdisReturnPackets(), but make sure it is the address of the correct
SWM> NDIS_PACKET pointer.

This is Ok. See MPReturnPacket() code above.

SWM> Also, try and disable the packet scheduler protocol (psched) in your
SWM> network card's settings. Then see if the bugcheck still occurs.

I disabled QoS, problem remains the same...

Best regards,
Serge.

[Stephan Wolf [MVP]]

serge wrote:
[..]

> I can see "xor ebx,ebx" followed by "mov esi,[ebx+0x8]"
> in the code, but its not clear for me why it may happen,
> I am not assembler guru :)

It seems the packet stacking has been scrambled somehow. Please try and
call NdisIMGetCurrentPacketStack() at various places in your code and
see whether the returned 'StacksRemaining' is TRUE.

I almost bet the bugcheck occurs when NdisIMGetCurrentPacketStack()
returns FALSE right before you call NdisReturnPackets().

Not sure as to *why* this happens, though. Are you using packet
stacking in your code?

Stephan

[serge]

Hello, Stephan!

Thanks for your reply.

SWM> serge wrote:
SWM> [..]

>> I can see "xor ebx,ebx" followed by "mov esi,[ebx+0x8]"
>> in the code, but its not clear for me why it may happen,
>> I am not assembler guru :)

SWM> It seems the packet stacking has been scrambled somehow. Please try
SWM> and call NdisIMGetCurrentPacketStack() at various places in your code
SWM> and see whether the returned 'StacksRemaining' is TRUE.

It seems to me that I have outdated DDK a little.
There is no any references to NdisIMGetCurrentPacketStack in .h and .lib
files.
I am not sure how to call this function from driver (are there analogs of
LoadLibrary/GetProcAddress?).

SWM> I almost bet the bugcheck occurs when NdisIMGetCurrentPacketStack()
SWM> returns FALSE right before you call NdisReturnPackets().

No, this happened at begining of NDIS!NdisReturnPackets (see below).

SWM> Not sure as to *why* this happens, though. Are you using packet
SWM> stacking in your code?

I do not use it directly.
My code is a little modified PASSTHRU sample from DDK.

Do you know what value is located at ebp+0xc and ebp+0x8 addresses?
I can see JBE and JNB dirrectives below.
Probably the code expects some values, but how can I know what values
exactly are compared?

NDIS!NdisReturnPackets:
f9889800 8bff mov edi,edi
f9889802 55 push ebp
f9889803 8bec mov ebp,esp
f9889805 83ec0c sub esp,0xc
f9889808 b102 mov cl,0x2
f988980a ff152c9e87f9 call dword ptr [NDIS!_imp_KfRaiseIrql
(f9879e2c)]
f9889810 8845fe mov [ebp-0x2],al
f9889813 33c0 xor eax,eax
f9889815 39450c cmp [ebp+0xc],eax
f9889818 8945f8 mov [ebp-0x8],eax
f988981b 0f8681000000 jbe NDIS!NdisReturnPackets+0x144 (f98898a2)
f9889821 53 push ebx
f9889822 56 push esi
f9889823 57 push edi
f9889824 8b4d08 mov ecx,[ebp+0x8]
f9889827 8b3c81 mov edi,[ecx+eax*4] ds:0023:f9f0e99c=816a6f30
f988982a 8b47fc mov eax,[edi-0x4] ds:0023:816a6f2c=ffffffff
f988982d 8b0d78a287f9 mov ecx,[NDIS!ndisPacketStackSize (f987a278)]
ds:0023:f987a278=00000002
f9889833 3bc1 cmp eax,ecx
f9889835 c645ff00 mov byte ptr [ebp-0x1],0x0
ss:0010:f9f0e97b=00

f9889839 0f832c070000 jnb NDIS!NdisReturnPackets+0x46 (f9889f6b)

[br=1]
f9889f6b 33db xor ebx,ebx

f9889f6d e9d6f8ffff jmp NDIS!NdisReturnPackets+0x48 (f9889848)

f9889848 8b7308 mov esi,[ebx+0x8] <== access violation
(ebx==0).

Thank you.

Best regards,
Serge.

[Steve]

If you queue the packets then you must alloc and copy the
payload. The passthru example just allocs and copies the
packet descriptor and sets the desriptor to point to the
payload in the original packet descriptor. My quess (and
I mean guess) is that the payload is being freed by the
NIC (receive) while the copied packet is still in your
queue.

...........
>Loading unloaded module list
>....
>Loading User Symbols
>*********************************************************

>.
>

[Stephan Wolf [MVP]]

Steve wrote:
> If you queue the packets then you must alloc and copy the
> payload.

This is not required. You can queue original packets until their
payload is no longer in use by the IM. You can also use the original
NDIS_PACKET descriptor if you use packet stacking.

Without packet stacking, a new NDIS_PACKET descriptor (allocated by the
IM) must be used. But its NDIS_BUFFER list can be the same as the one
used by the original NDIS_PACKET.

IIRC, PASSTHRU does exactly this, i.e. it does not copy the payload
*unless* it needs to *modify* the payload.

That is, the original payload must not, under no circumstances, be
modified. It is read-only to the IM. But you can have your own
NDIS_BUFFER list that points to your own payload, e.g. the MAC header.
Then use your own NDIS_BUFFER (or buffers) to point the original
payload. This way, parts of the original payload can be replaced
without having to copy the actual payload data.

> The passthru example just allocs and copies the
> packet descriptor and sets the desriptor to point to the
> payload in the original packet descriptor.

Yup.

> My quess (and
> I mean guess) is that the payload is being freed by the
> NIC (receive) while the copied packet is still in your
> queue.

How can the NIC free any NDIS_PACKET, NDIS_BUFFER, or payload before
the NDIS_PACKET is being returned to it? That would be a programming
fault and I seriously doubt it since the OP says he sees the problem in
more tha one configuration (ie.e with different miniport drivers).

Also, the NDIS Test tool (see my article at
http://www.microsoft.com/whdc/resources/mvp/SW-MVP.mspx) would for sure
detect such misbehaviour.

Stephan

[serge]

Hello, Steve!
You wrote on Mon, 25 Apr 2005 20:56:29 -0700:

S> If you queue the packets then you must alloc and copy the
S> payload.

Actually, I do reallocate NDIS_BUFFER, copy paypoad, and attach buffer to
packet.
Problem is still there even if I use original packet buffer descriptors
instead...

Best regards,
Serge

[serge]

Hello, Stephan!
You wrote on 25 Apr 2005 23:08:55 -0700:

SWM> Also, the NDIS Test tool (see my article at
SWM> http://www.microsoft.com/whdc/resources/mvp/SW-MVP.mspx) would for
sure
SWM> detect such misbehaviour.

It will not help, because I get BSOD on first packet received...

I found one interesting thing at the begin of NDIS!NdisReturnPackets:

f988982a mov eax,[edi-0x4] ds:0023:816a6f2c=ffffffff
f988982d mov ecx,[NDIS!ndisPacketStackSize (f987a278)]
ds:0023:f987a278=00000002
f9889833 cmp eax,ecx

EDI is a pointer to NDIS_PACKET structure passed to NdisReturnPackets.
Why the code evaluates 4 bytes prior NDIS_PACKET?
Seems like all my packets have 0xffffffff there, but driver expects 0x2
there probably.
Can anybody clarify this?

Thanks.

--
Best regards,
Serge

[Stephan Wolf [MVP]]

This is exactly the reason why I suggested you call
NdisIMGetCurrentPacketStack() and see whether it returns TRUE or FALSE.

Packet stacking is actually implemented such that it uses data
preceding the NDIS_PACKET. The NDIS_PACKET that you see is just a
member in a superior (surrounding) packet structure.

It is thus crucial that you allocate your NDIS_PACKET descriptors via
NdisAllocatePacket() because it takes care of allocating the required
additional space (before and behind each NDIS_PACKET structure).

Please make sure you follow this rule, i.e. do not allocate NDIS_PACKET
descriptors in any other way but via NdisAllocatePacket().

Also, it should be easy to instruct WinDbg to watch the DWORD right
before the NDIS_PACKET such that it will cause a breakpoint as soon as
its value drops below zero (see WinDbg Help for details).

Stephan
---
serge wrote:
[..]

[Pavel A.]

Serge,
are you debugging *your* driver or vmware?
Vmware can introduce it's own artifacts.
Just stay with Vmware 4 or use a *host* netcard that does not cause crash,
submit bug to Vmware and let them sort this out.
Then test your stuff on *real* machine.

Regards,
--PA

[Alireza Dabagh [MS]]

Serge,
I suspect the packet you are getting is coming to you with
NDIS_STATUS_RESOURCES which means you can not hold on to it. i.e. as soon as
you return back from the original call, NDIS assumes you are done with the
packet (and hence the -1 right before the packet pointer.) therefore, you
are not allowed to return this packet later on.
The reason you are seeing this with -some- network cards and vmware, could
be because those network drivers indicate the packet with status resources
or use old style IndicateReceive which at some point forces an IM driver in
between (such as psched) to indicate the packet with NDIS_STATUS_RESOURCES.
The same could be true for vmware.

Please check your code and make sure you handle packets indicated to you
with status set to NDIS_STATUS_RESOURCES properly. you can get the status of
a packet by using NDIS_GET_PACKET_STATUS macro.

Hope this helps.

-ali

--
This posting is provided "AS IS" with no warranties, and confers no rights.

"serge" <pser...@ukr.net> wrote in message

news:d4lbvo$nnn$2...@news.dg.net.ua...