FltSendMessage, et. al
Dakota Ridge
etc. group of functions - or who has done so in the past. The most critical
piece of knowledge on these functions remains (to my knowledge) completely
undocumented by Microsoft. If you don't know the following, you should not
even be using them - they may work for you, but if they do, it's only because
you were lucky....
These functions are, as best I can understand, very damaging and confusing
wrappers for Io Completion Port functions. Not KNOWING how those functions
work didn't help.
I have 11 events that I communicate down to a DLL from my minifilter driver.
10 of them require a 40 byte buffer. The 11th requires an additional 260
bytes (MAX_PATH) of space to pass down a file name that the DLL looks up and
says yea or nay to track its activity. I'll spare you all the gory detalis,
and the literally scores of hours this lack of documentation has cost me, and
I'll get right to the point: various Power(less)Point stuff on the web says
these functions don't use queueing. Well, yes and no. These functions are
completely buffered. There is no real sense of "a message" within them. If
your user mode app is requesting 300 bytes (as mine was, unconditionally; my
faulted logic being that a message is a message and if I allocate 300 bytes
to receive a 40 byte message, the extra space just won't be used), then what
the filter manager will do is dump message after message from the kernel into
that 300 byte abyss until it's full, then it'll finally respond to the
waiting FilterGetMessage function. If the user mode app requests 200 bytes,
the filter manager will dump 5 or 30 or 50 message sent by the kernel into
that 200 byte buffer you provided until it's full. It will all show up via
a single FilterGetMessage call in the user app, and the user mode app will
think it just retrieved only one message - never knowing about the 4 or 29 or
49 that were lost.
Think "byte count," not "message." There is no sense of a message as an
entity within any of these functions. It's all about byte count. You
define a size for a buffer, that buffer is filled regardless of how many
FltSendMessage calls it takes to do that, and then it's dumped. A simple
solution - which probably most people do already, but only by default - is to
be absolutely sure that every message sent down from the driver using
FltSendMessage, and every message received in the user mode app with
FilterGetMessage, use EXACTLY the same size buffer. Never, never, never let
there be a deviation in ANY location - driver or app - or there will be no
error, but all "alignment" of your messages will be shot to hell and you'll
never know why; you'll never be told anything is wrong. Use what you need
out of any given buffer but always send down and request to receive EXACTLY
the same byte count or the filter manager will start breaking up and
redistributing your messages and you'll never be told it's happening.
FltSendMessage has no concept of "a message." All that call does is "dump x
number of bytes into the buffer." On the user app end, FilterGetMessage is
equally stupid about what a message is. It only knows "retrieve x number of
bytes from the buffer and don't come back (or DO timeout) until that buffer
HAS that many bytes." There is no concept of "a message" anywhere in the
process.
Don't become a statistic.
Daniel Terhell
news:D72B20B1-5A32-436A...@microsoft.com...
> If your user mode app is requesting 300 bytes
A user mode app does not request a number of bytes, instead it passes a
buffer and specifies the maximum size.
> if I allocate 300 bytes to receive a 40 byte message, the extra space
> just won't be used), then what
> the filter manager will do is dump message after message from the kernel
> into
> that 300 byte abyss until it's full, then it'll finally respond to the
> waiting FilterGetMessage function.
Inside you message notify callback, you need to set ReturnOutputBufferSize
to the number of bytes returned and your app should check this.
/Daniel
Dakota Ridge
buffer and specifies the maximum size." No, it doesn't. There is no
"maximum size." There is only "THE size," and FltSendMessage will use 2, or
5, or 50 messages to fill that "maximum size." You cannot have less, you
cannot have more than "the maximum size." When I changed the size of these
buffers, ALL my problems evaporated and the functions worked perfectly. I
specified a "maximum size" on the receiving end of 300 bytes (for that one
message out of eleven that needed 300 bytes instead of 40), and
FltSendMessage used 7.5 messages of 40 bytes each on the driver end to fill
that "maximum size." One FilterGetMessage call in the user mode app
retrieved 7.5 messages from the driver. I never knew this was occurring
because it's not documented, so I was losing 6.5 messages each time I called
FilterGetMessage; worse, the second round of 7.5 messages started off by
filling my message structure with the second half of a 40 byte message so
this was being tossed as indecipherable and I was trying to figure that out
was well. When I changed the send buffer size to 300 bytes, to match the
"maximum size" of 300 bytes on the receiving end, everything worked; all
problems were gone. I've verified this in ten different ways, there is no
question that this is what's happening.
See for yourself. Set your send buffer to one size; set your receive buffer
to twice that size and see if you don't get two complete messages on every
FilterGetMessage call in the user mode application - one on top of the other.
The second half of your "maximum size" will not be empty. Zero it out
before receiving the message and see what happens. I've created counters
that display on screen and on WinDbg to count messages sent and messages
received; they never matched up. I fixed the receive buffer size and they
lined up perfectly every time. I was never receiving critical messages I
needed; I could never figure out why. I verified a hundred times that they
were being sent without error from the driver but the user mode app just
couldn't receive them. They were being lost inside that extra "maximum
size." I fixed the sizes of the buffers and all my messages started coming
through as expected, there were no more problems. Etc. etc. There's no
question about this.
Daniel Terhell
what you are observing is a used buffer being overwritten. . But it is just
a guess, you should post some code or explain better what you are doing to
find out what you are doing wrong. First of all what type of 'messages' are
you sending around ? Do they have a FILTER_MESSAGE_HEADER ? So what is the
reply length and does it match the value returned by
ReturnOutputBufferLength in your message notify callback ?
/Daniel
"Dakota Ridge" <Dakot...@discussions.microsoft.com> wrote in message
news:E4434348-2CDF-4BC8...@microsoft.com...
Dakota Ridge
1. The driver calls FltSendMessage with a 40-byte message. This function
doesn't use a MessageHeader.
2. The user mode app calls FilterGetMessage. It knows there are 11
possible messages. 10 of them will be 40 bytes plus the size (8 bytes) of
MESSAGE_HEADER that is added on. However the 11th possible type of message
is 300 bytes plus 8, so it specifies a "maximum size" of 308 bytes, since it
doesn't know which message is in the buffer until it's already retrieved.
3. There is no response from FilterGetMessage because the "Maximum size" of
308 bytes has not yet been sent into the buffer by FltSendMessage.
4. Back in the driver, FltSendMessage sends a second message of 40 bytes
... and a third ... etc.
5. Back in the user mode app, FilterGetMessage still has not returned (it's
on infinite wait).
6. Eventually the driver pumps enough data into FltSendMessage, maybe with
7 or 8 separate calls to FltSendMessage, to exceed the 308 byte buffer that
FilterGetMessage passed, so then FilterGetMessage returns. The user mode app
has no idea it just lost 6 or 7 or however many messages, so it only
processes one. The "lost messages" are actually present inside its 308 byte
buffer, but it doesn't know to look for them. To code a program around this
kind of silliness is no really very responsible, now is it? :-) We
shouldn't have to adapt our code. Microsoft should document their functions;
that's a lot easier.
That's it.
Daniel Terhell
> <Sigh> It's just a lot simpler than all that.
>
> 1. The driver calls FltSendMessage with a 40-byte message. This function
> doesn't use a MessageHeader.
So when you call FltSendMessage, what are SenderBufferLength, ReplyBuffer
and ReplyBufferLength and do you actuallyset ReplyLength in the
FILTER_MESSAGE_HEADER ? Do you also check the return statuses ?
>2. The user mode app calls FilterGetMessage.
>3. There is no response from FilterGetMessage because the "Maximum size"
>of 308 bytes has not yet been sent into the buffer by FltSendMessage.
But FilterGetMessage comes out of its wait state (assuming you are not using
an overlapped structure), doesn't it ? So do you check the reply length and
the message id ? Why you are not calling FilterReplyMessage at this point ?
Things are more complicated because you communicate over a client port and
not over a server port. More often, a user mode app calls FilterSendMessage
and the minifilter gives replies in a message notify callback (and not by
calling FltSendMessage).
/Daniel
Dakota Ridge
0.
Send buffer length is 40 bytes.
FilterGetMessage sets up to receive sizeof ( FILTER_MESSAGE_HEADER )
followed by 300 bytes. The extra 260 bytes available for receive should not
be used. The message should fill the first 48 bytes ( FILTER_MESSAGE_HEADER
) plus the 40 bytes I sent. It doesn't work that way. FilterGetMessage
doesn't return from its first call until FltSendMessage has filled the buffer
with 308 bytes - across any number of FltSendMessage calls. It does not exit
its wait state until this occurs.
No reply length is involved. No replies are involved. I'm not asking for
any and I'm not sending any. I don't call FilterReplyMessage because the
driver doesn't need a reply. It would have no use for that data if it
received it.
The message notify callback is completely separate from this entire process.
The point is that one message sent by FltSendMessage will not be received by
FilterGetMessage - this is the case long before replies even have a chance to
come into the picture - until the entire FilterGetMessage buffer has been
filled, even if it takes 30 FltSendMessage calls to do that.
Much simpler than all the questions would be to simply write the code.
Create a FltSendMessage call sending a 40-byte buffer. Create a
FilterGetMessage call receiving a "maximum size" 88-byte buffer. Put a
counter on FltSendMessage. Put another counter on FilterGetMessage.
FltSendMessage will show two calls; FilterSendMessage will show 1. 2
messages of 40 bytes each, plus a message header, will be present in your 88
byte buffer. Just write the code and all questions will be answered. No
replies, no message notify callbacks are invovled in any way.
Daniel Terhell
out why instead of experimenting and drawing conclusions from further
observed behavior.
When you call FltSendMessage, make sure you specify the client port. This is
obtained from your connect notify callback and is not the serverport output
parameter which is returned by FltCreateCommunicationPort.
You do not want to post code which makes it difficult to help you just by
following your reasoning. Check out the scanner source code for an example
on how to do it.
/Daniel
"Dakota Ridge" <Dakot...@discussions.microsoft.com> wrote in message
news:44D0010B-CE0E-46CE...@microsoft.com...
Dakota Ridge
sent by FltSendMessage. There is no confusion here. Of course the client
port is being specified on FilterGetMessage, otherwise nothing would ever get
through. The user application never knows the server port so it's hard to
confuse things here. When I set the buffer sizes the same on FltSendMessage
and FilterSendMessage, it was perfect operation on both ends. Everything
worked just the way it should. All I did was change the buffer size on
FilterGetMessage to exactly match (then adding the FILTER_MESSAGE_HEADER
size) the buffer size used by FltSendMessage. That was the only change and
everything fell into place; perfect communication. Change the buffer size to
something larger on FilterGetMessage and I start losing messages for exactly
the reason I explained.
Scanner was far too convoluted, complex and inefficient for what it does to
be of any help. It was a joke, more or less; an explosion of inefficiency.
Definitely a VERY amateur job - messy. sloppy, and it takes 45 minutes to
track anything down because of runaway redefinitions - wrappers for wrappers
that are only wrappers for other wrappers. Bureaucracy run amok. I was not
impressed.
This terminates this conversation on my end; simply writing the code will
verify everything I've said, I've been very clear about it, there is no room
for debate; just write the code and it works the way it works. Client ports
and replies never come into the picture. It's a very simple concept. To
argue theory is just silly when one can just write the code and see for him /
herself. So I'm bowing out. I think everybody else who cares gets the point
very clearly.
Matt Garson [MSFT]
I work on the File System Filters team at Microsoft. I'd like to discuss the
problems you're having in further detail. I don't see an email addr for you
listed - would you mind sending me an email? My alias is the first letter of
my first name followed by my last name at microsoft.com. I think email will
be an easier means of communicating than newsgroup posts.
--
This posting is provided "AS IS" with no warranties, and confers no rights.