OnSendRawData ASAP
German
I have a serious problem. Maybe because I'm not understand ISAPI logic,
maybe because Microsoft don't understand what developer wants. Anyway. In
OnSendRawData I'm changing (using pRawData->pvInData pointer) content that
server is sending to user. Everything is fine if size of content still
preserved, but if after my changes size of content is improved, user will
not be able to see end of the document. As I understand it's because in
first call of OnSendRawData IIS write Content-Length in header, so later it
is useless to add some data to the outgoing content because user browser
will show only content not larger then in Content-Length. As I have analyzed
it's impossible change Content-Length header if once it have been sent.
So, that's my problem.
Again, what I'm trying to do:
I have old ASP page that is generating some content. I want to write some
ISAPI filter to change that content just a bit. I find en example in
Platform SDK. That example is changing letters in outgoing content from
lower case to upper case. But there content size is preserved. But if I
should change content size how to tell IIS that header should be changed. My
god, I'm going to be crazy. Help me PLEASE !!
Thank you so much.
--
Sincerely, German
"I think there is a world market for maybe five computers." --Thomas Watson,
Chairman of IBM, 1943
Wade A. Hilmo [MS]
Writing a raw data filter that changes the size of the response is quite
non-trivial. Since you are working with the raw data stream, it is up to
you to maintain the integrity of the response.
There was a recent thread on this newsgroup on this very subject. Near the
end of the thread, I made some suggestions for how a filter could buffer the
response for editing, update both the content-length and the entity in the
buffer, and then send the final result. Here is a link to the thread in
Google:
http://groups.google.com/groups?hl=en&lr=&ie=UTF-8&oe=UTF-8&threadm=Op9NWgUb
CHA.1884%40tkmsftngp12&rnum=1&prev=/groups%3Fhl%3Den%26lr%3D%26ie%3DUTF-8%26
oe%3DUTF-8%26selm%3DOp9NWgUbCHA.1884%2540tkmsftngp12
I hope this helps,
-Wade Hilmo,
-Microsoft
"German" <german.koninin@pwrgenerationDOTnet> wrote in message
news:erTJmEsfCHA.1816@tkmsftngp12...
German
gone?
I mean I have investigated that my ISAPI filter process two OnSendRawData
calls when user request the page. First call process HTTP headers and second
process body of the document. So, it's look like I should chnage
Content-Length in first call. It is possible and I do, but I don't know the
content-length that will be (I will only after second call, when content
will be coming) on the final version of document. So, in second call I can
change body of the document and count current size, but I'm processing
second call (HTTP headers in first call have gone already). Doyou understand
me?
I just can't understand why there's nothing info in SDK about this effect. I
investigate it myself. Reading SDK documentation it's look like everything
is fine and quite easy to do some filter. And SDK have some f%cking example
that shows how to change RawData. Fine, but what about changed data will be
different size? Not everybody wants simple change letters from lower case up
to the upper case. This is just stupid.
I have solved it this way: In first call I just multiple multiply
Content-Length on two. But there's some unwanted effect have came. Real
document size after chages of course smaller then just Content-Length*2, so
browser continue to show progress bar (look like docume is not downloaded
completely). But anyway this is working.
I just can't understand why they design this way. OnSendRawData should
process document in one time wioth headers and body!
PS: Thank you for your replyu, I take a look at cthat google link.
Wade A. Hilmo [MS]
First off, let me say that you must get the content-length correct. As
you've seen, just making it some number that is "big enough" will not work.
The client is depending on the correct number here, and it must be accurate.
One thing to keep in mind as you look at this problem is that this is *not*
an easy task. I don't believe that you'll find anything, anywhere that says
it is. When you alter the data in raw streams, you *must* understand
everything that you are doing. In the case of a raw data ISAPI filter, you
(and by extension, your code) must have a complete understanding of HTTP and
how it works. You will pretty much need to write a partial HTTP engine to
take apart and repackage the data you are dealing with. One of the reasons
that no sample exists that shows this is that it is very hard to do right.
Any such sample would either be quite complex, or would not handle all of
the scenarios that you'd need to deal with in production code. There are
commercial software packages out there that do all of this correctly, but
they do not typically release their source code to the public.
Now then, on to your main question.
You cannot change the content-length header once it's been sent to the
client. You must prevent it from reaching the client until you can
accurately calculate it.
So how do you prevent it from reaching the client?
Basically, you need to grab the data that's going to the client, store it in
your own buffer for later user, and then tell IIS to send no data for this
notification. In the HTTP_FILTER_RAW_DATA structure that gets passed to the
SF_NOTIFY_SEND_RAW_DATA notification, there are two members that you'll need
to use. You'll need to look at the cbInData member to determine how much
data is currently going to the client. You'll need to allocate a buffer of
at least that many bytes so that you can copy it from pvInData for later
use. Then, if you set cbInData to 0, IIS will send no data to the client
for that notification.
How should you manage the data?
It's really an open ended question as to you how should manage the data. I
will make a recommendation for logic that would be relatively straight
forward. I suggest that you allocate two buffers, one for the HTTP data
(the version, status and response headers), and one for the entity. These
buffers should be smart enough that you can grow them as necessary as you
add data to them.
What this means is that, when you see the first SEND_RAW_DATA notification
for the request, you will parse it to find the end of the response headers.
There are 3 possible scenarios here (and even though one of them is probably
the most common, all of then can and do happen).
First, it's possible that you get lucky, and the data associated with the
notification is contains all of the HTTP data and no response entity. In
this case, you just copy all of the notification data to your HTTP data
buffer and set cbInData to zero. All subsequent data in further
notifications then goes into your entity buffer.
Second, it's possible that the first notification contains all of the HTTP
data and also some entity. In this case, you'll want to copy the HTTP data
into your HTTP data buffer, and the entity data into your entity buffer and
set cbInData to zero. All subsequent data in further notifications then
goes into your entity buffer.
Finally, you might see a case where the first notification contains part of
the HTTP data, but not all of it (ie. you don't get the "\r\n\r\n" that
marks the end of the response headers). In this case, you'll need to copy
all of the notification data into your HTTP data buffer and set cbInData to
zero. When the next notification occurs, you'll need to do this check again
(keeping in mind that the "\r\n\r\n" might be broken up into more than one
notification). As before, HTTP data goes into the HTTP data buffer, and
when you find the start of the entity, start putting stuff into the entity
buffer. Keep zeroing out cbInData all the time.
Keep doing this until SF_NOTIFY_END_OF_REQUEST occurs.
So now, where are we?
When SF_NOTIFY_END_OF_REQUEST occurs, you should have two buffers, and no
data has been sent to the client. One of the buffers contains the HTTP data
(including content-length if it was provided), and one of them contains all
of the entity data.
From here, you can parse the entity data and make the changes you need.
Then, after you've made the changes, you can calculate the correct
content-length and tweak the HTTP data buffer. Now, you have two buffers
that represent the complete and correct response after your modifications.
So how do you get the correct response to the client?
To send the response, you now just need to call WriteClient once with the
contents of the HTTP data buffer, and once with the contents of the entity
buffer. The trick here is that SF_NOTIFY_SEND_RAW_DATA will fire once for
each WriteClient call that you do here. To prevent your raw data logic from
trying to do its buffer thing, you'll need to use pFilterContext to mark the
state in some way so that the send raw logic knows that you really want this
one to get to the client (note that you will have to write the logic into
the send raw notification code so that it knows to check pFilterContext for
this state and do the right thing).
So what are the "gotchas"?
As I stated up front, the biggest "gotcha" is that there are lots of ways a
response can use - or not use - content-length in its response.
If a request wants to, it can omit the content-length alogether. At first
thought, you might be tempted to think "A-ha! My filter can just remove the
content-length from all responses." Unfortunately, this will not work.
According to HTTP, if you don't send a content length in some form, then you
must close the connection at the end of the response (how else would the
client know that you are done if you don't send a length?) The problem here
is that, as a raw data filter, you are seeing the response just before it
goes out on the wire. IIS has already made its decision as to whether or
not to close the connection. If the response originally had a
content-length, IIS will keep the connection open if the client requested it
(which pretty much always does). Since IIS doesn't process the data after
your filter, IIS doesn't know that you've changed it and will still keep the
connection open no matter what you do. If you remove the content-length,
and IIS doesn't close the connection, the client thinks that there will be
more data coming and will wait for a very, very long time for it.
If a request wants to, it can send a content-length header. This is the
common scenario that you are seeing. In this case, your filter will need to
work as above and rebuild the entire response with a correct content-length.
If a request wants to, it can use chunked transfer encoding (see the RFC
reference in the other thread I pointed you to). Chunked transfer encoding
is just a variation on the content-length idea, where each "chunk" of the
response will have size information in it. Note that chunks may or may not
correspond to the data in a given notification. You should not make any
assumptions - plan for the notification data to be not tied to chunks. In
this case, you'll still need to buffer the response as above. The
difference is that you'll need to either modify the chunk size information
as appropriate, or convert the entire response from chunk transfer encoded
to a basic content-length. Since you are buffering the response anyway, I
would just remove the "transfer-encoding" response header and all of the
chunk size parts in the entity and just add a valid content-length. If you
wanted to get really advanced, you could optimize your buffering so that you
never have to buffer more than one chunk.
I hope that this information is helpful to you,
-Wade Hilmo,
-Microsoft
"German" <german.koninin@pwrgenerationDOTnet> wrote in message
news:#VO#vFBgCHA.1256@tkmsftngp10...
German
This morning I have done exactly the same things that you talking about.
Right now I'm feeling that I'm not idiot as well. Because I find a way
myself. This trick with changing cbInData is cool, I agree. It's just bad
that there's really poor description of this staff in SDK. "...Commercial
software can do it and a little bit more...". Yep, I agree. but somebody
should right that software and that somebody should use some documentation
do it. To encapsulate that logic in some other application. But! You (MS),
guys have super cool thing named newsgroup. This is powerful. Ok, thank you
for your help.
==\ The End /==
"Wade A. Hilmo [MS]" <wa...@microsoft.com> wrote in message
news:Od2zMlDgCHA.1148@tkmsftngp11...
runawa...@yahoo.com
However, not I am doing this in .Net.
Where should the buffer be stored?
Should this be a member to the class? A static global page variable?
Any direction on this would be helpful.
Thank you.
Wade A. Hilmo [MS]
You have trimmed so much information from the thread that there is no way to
know what you are talking about.
If you can repost and include enough relevant information so that we can
tell what you are really asking, then maybe we can help.
Thank you,
-Wade A. Hilmo,
-Microsoft
<runawa...@yahoo.com> wrote in message
news:1106600175.9...@f14g2000cwb.googlegroups.com...
runawa...@yahoo.com
wasn't going to show up).
It isn't that I was trying to trim anything off. Most of what was said
here I already have working. (Changing the data byte length and the
content header after adding/subtracting data.)
What I am trying to do is buffer the html and headers. To do this, you
need to save both the state information and a buffer (be it a char[] or
some other data structure).
In .Net, I have access to pCtxt->m_pFC->pFilterContext, but this is
only one place to store something. As mentioned in other places, state
would be stored here. (The state of first responses or changing to
write back to the client.)
What I am looking for is how to go about storing the buffer (the actual
char[] or more advanced struct). Is it all going to be stored here via
a pointer to some more complex struct/class with buffer, state
information and all?