NDIS Intermediate/Miniport Drivers - Threads/Synchronization
rohit sah
I am a newbie to NDIS and am developing an
Intermediate Driver.
I am basically trying to protect critical regions in
an IM/Miniport driver and create a separate system
worker thread that processes tasks dequeued from a
NDISList. I face the following problems while trying
this:
1. Can System Threads (PsSystemThreadCreate( )) be
created in Intermediate Drivers?
I am extending the passthru driver and it compiles
with the "NDIS_MINIPORT_DRIVER NDIS40_MINIPORT"
compile time flags defined. When these flags are
defined at compile time, thread creation doesn't seem
possible as the relevant functions
(PsSystemThreadCreate( )) fail to compile.
Is this understanding correct or am I missing
something?
2. Can Synchronization Primitives like Semaphores,
Mutexes be used in Intermediate/Miniport drivers?
3. Can the IRQ Level of an executing routine be
raised(KeRaiseIrql)/lowered(KeLowerIrql) in an
IM/Miniport driver?
4. I want to make the ProtocolPacketReceive and
MiniportSend functions uninterruptible by the other.
The problem I face is the following:
* Both ProtocolReceivePacket and MiniportSend run at
IRQL_DISPATCH.
* KeAcquireSpinLock( ) work by raising the IRQL of
the calling routine to IRQL_DISPATCH, thereby making
the currently executing routing uninterruptible by
routines that run at IRQL < DISPATCH.
* However, since both ProtocolReceivePacket and
MiniportSend run at IRQL_DISPATCH these functions
continue interrupting each other despite using a spin
lock.
5. Is it possible to create WorkItems and get these
processed by SystemWorker threads in an IM/Miniport
driver?
I tried this, but again the code fails to compile.
Thanks in advance.
Cheers,
Rohit
Stephan Wolf
(miniport) driver should, by definition, not use any non-NDIS
functions.
Use NDIS work items instead, i.e. NdisScheduleWorkItem().
You can, however, use *any* system functions you like in a separate C
file. It is however my personal opinion that making use of non-NDIS
functions in wahetver NDIS driver is not necessary at all and I have
managed to write perfect IM drivers this way.
2. Yes, see 1).
3. NDIS does not provide functions to directly raise or lower IRQL.
You can use NdisAcquireSpinLock() or NdisAcquireReadWriteLock() to
implicitly raise IRQL to DISPATCH_LEVEL. Relasing the lock will
restore the previous IRQL. 1) Also applies here.
4. Your understanding of DISPATCH_LEVEL seems incomplete. Any thread
running at DISPATCH_LEVEL will run to completion without any other
thread being able to get control *on the same CPU*. (Interrupt Service
Routines, however, can run because they run at DIRQL.)
But in a multi-processor system other CPUs can execute threads at any
IRQL no matter if your thread is currently executing at
DISPATCH_LEVEL.
Thus you *cannot* serialize the send and receive handlers of your (by
definition deserialized) IM via the IRQL mechanism. Instead, you could
for instance put any send or receive packets in a queue and then call
some handler that can only execute serialized. The handler then needs
to handle all outstanding sends an receives.
To "serialize" the handler use NdisInterlockedIncrement() and
NdisInterlockedDecrement() similar to how the MUX sample implements
mutexes:
SendReceiveHandler()
{
do {
if (NdisInterlockedIncrement(&MutexCounter) != 1) {
return; // Handler already active.
}
// Handle *all* queued sends and receives....
} while (NdisInterlockedDecrement(&MutexCounter) != 0)
}
[Obvious optimizations omitted here for simplicity...]
5. See NdisScheduleWorkItem().
Stephan
---
James Antognini
passthru sample with that routine in it.
2) Probably, but be careful of the IRQL where it is done.
3) Possibly safer to use NdisAcquireReadWriteLock(). This was discussed
recently.
4) You must be running on an SMP machine. IRQL affects the current CPU
only. Only one routine can hold a spin lock, but one routine on CPU 0
could hold it and another routine on CPU 1 could spin trying to get it.
That is what a spin lock is all about. Let me note, however, a
signficant caveat: If you're running a deserialized miniport driver,
MiniportSend can be entered at PASSIVE_LEVEL and so could be interrupted
by a ProtocolReceivePacket routine (always DISPATCH_LEVEL) or -- I
presume, as I've not tested this -- by another MiniportSend instance
running at DISPATCH_LEVEL, so long as the first MiniportSend instance is
running at less than DISPATCH_LEVEL. So check your logic.
5) I think this should work. I haven't tried it. As for compilation, see
above.
rohit sah wrote:
> (PsSystemThreadCreate( )) fail to compile
> 2. Can Synchronization Primitives like Semaphores,
> Mutexes be used in Intermediate/Miniport drivers?
>
> 3. Can the IRQ Level of an executing routine be
> raised(KeRaiseIrql)/lowered(KeLowerIrql) in an
> IM/Miniport driver?
>
> 4.
> * However, since both ProtocolReceivePacket and
> MiniportSend run at IRQL_DISPATCH these functions
> continue interrupting each other despite using a spin
> lock.
>
> 5. Is it possible to create WorkItems and get these
> processed by SystemWorker threads in an IM/Miniport
> driver?
--
If replying by e-mail, please remove "nospam." from the address.
James Antognini
Windows DDK MVP
rohit sah
Thanks for the detailed response.
I have another question based on observed behavior after making some
design changes:
> 4. Your understanding of DISPATCH_LEVEL seems incomplete. Any thread
> running at DISPATCH_LEVEL will run to completion without any other
> thread being able to get control *on the same CPU*. (Interrupt Service
> Routines, however, can run because they run at DIRQL.)
>
> But in a multi-processor system other CPUs can execute threads at any
> IRQL no matter if your thread is currently executing at
> DISPATCH_LEVEL.
>
> Thus you *cannot* serialize the send and receive handlers of your (by
> definition deserialized) IM via the IRQL mechanism. Instead, you could
> for instance put any send or receive packets in a queue and then call
> some handler that can only execute serialized. The handler then needs
> to handle all outstanding sends an receives.
I am using a Uniprocessor system. I have now implemented changes in
my design to use "NdisScheduleWorkItem( )". Am doing minimal
processing in the ProtocolReceivePackets and MiniportSend handlers and
queue packets using NdisInterlockedInsertTailList( ).
NDIS documentation mentions that MiniportSend runs at level <=
IRQL_DISPATCH and ProtocolReceivePacket runs at IRQL_DISPATCH. This
means that ProtocolReceivePacket should never be interrupted by
MiniportSend on a Uniprocessor system. However, on running my driver
after making these changes I see ProtocolReceivePacket being
interrupted by MiniportSend several times. Also, I noticed that
MiniportSend is never interrupted (Even though the DDK mentions that
it is MiniportSend that runs at IRQL <= IRQL_DISPATCH and that
MiniportSend can be interrupted). I find the documentation and the
observed behavior contradictory. Is my understanding of this incorrect
or is the NDIS documentation faulty?
Thanks in Advance.
Cheers,
Rohit
Stephan Wolf
ProtocolReceivePacket() -> NdisSend() -> MiniportSend[Packets]()
(ARP is an example)
So you're not actually "interrupted" and still your MiniportSend()
handler gets called.
Your approach still sounds like you will for sure run into severe
problems on MP systems.
Stephan
---
James Antognini
mistaken, as that would have profound implications for every NDIS driver's
design. So, a couple of things to check:
1) Is your ProtocolReceivePacket callback registered only as that callback
and not for anything else? Namely, something that would be called at less
than DISPATCH_LEVEL?
2) What is the call stack when ProtocolReceivePacket is interrupted by your
MiniportSend? It should show only system routines; I wonder whether some
routine of yours is calling ProtocolReceivePacket at less than
DISPATCH_LEVEL.
It's best to run Windbg and confirm that things are happening as you think.
rohit sah wrote:
> I see ProtocolReceivePacket being
> interrupted by MiniportSend several times.
--
Stephan Wolf
wrote:
> ProtocolReceivePacket() -> NdisSend() -> MiniportSend[Packets]()
I ment to say
-> ProtocolReceivePacket() [in your IM]
-> NdisMIndicateReceivePacket()
-> ProtocolReceivePacket() [in e.g. TCP/IP]
-> NdisSend()
-> MiniportSend[Packets]() [in your IM]
Just place a DbgBreakPoint() in your code where you detect you are
being "interrupted" and take a look at the call stack. You should see
something similar to the above.
Stephan