Wininet Traffic

[brok21k]

Im writing a mod downloader for a game, Im able to download mods from
my server which works fine. But I want it to download from a popular
file hosting site. I've got this to work, but it only seems to
download the file while fiddler is running (While its capuring the
Wininet traffic!).

Im using the HttpWebRequest and HttpWebResponse in VB.NET (instead of
the Wininet class because of their ease)

I've turned off all of the options in fiddler, and then downloaded my
file and it still works. The only time the download dosen;t work is
when I disable the request.proxy = nothing. So Im guessing its a
proxy issue, or theres somthing fiddler is automaticly doing( which I
am not in my vb.net program).

Note: This is actually a redirection, my first response gave me the
full url and file name (the first response also contained Vary: cache)
but as I said it downloads when fiddler is captures traffic.

Heres the server response:

REQUEST=
GET /filename.txt HTTP/1.1
User-Agent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0; Trident/
4.0; GTB6; SLCC1; .NET CLR 2.0.50727; Media Center PC 5.0; .NET CLR
1.1.4322; .NET CLR 3.5.21022; .NET CLR 3.5.30729; InfoPath.1; .NET CLR
3.0.30729; Creative AutoUpdate v1.10.10)
Accept-Language: en-gb
Host: host.com
Cookie: ukey=lllyamyvelaldcdyycwexylayl2avmll
Connection: Keep-Alive


RESPONSE=
HTTP/1.1 200 OK
Server: LRBD-2.0.1
Date: Thu, 26 Dec 2009 17:36:28 GMT
Connection: close
Accept-Ranges: bytes
Content-transfer-encoding: binary
Content-Length: 356445
Content-Disposition: attachment; filename="Filename.txt"
Content-Type: application/x-rar-compressed

[EricLaw]

What *specifically* happens in the failure case? What exceptions or
errors are seen?

You could try grabbing a low-level capture using NetMon to see what
exactly is different about the request when Fiddler isn't running.

[brok21k]

I found the problem!

Here:
=========================================================
Due to increased security requirements, .NET 1.1 SP1 will now only
accept HTTP responses that are 100% compliant with
the HTTP RFC specification (see RFC 2616). The servers listed below
return almost, but not quite compliant HTTP
responses. For example, when I tried your code against www.amazon.com,
the response does not appear to contain a description of the status
code, which is probably a protocol violation. These violations will
generate server protocol violation exceptions.

As a work around, you can configure your application to accept headers
that are not perfectly formed:

- Open the *.exe.config for your application, or the configuration
file for your entire .NET system.
- Look for the following XML tags in the configuration:

<system.net>
<settings>
<httpWebRequest useUnsafeHeaderParsing = "true" />
</settings>
</system.net>

- If the system.net tag is not there, then add it.
- Add the setting to allow unsafe headers as shown above.
==============================================

So fiddler was making my requests 100% compliant with the HTTP.

> > Content-Type: application/x-rar-compressed- Hide quoted text -
>
> - Show quoted text -

[EricLaw]

What exactly did the invalid traffic look like? Do you have a byte-
level capture?

Fiddler should complain about anything that is invalid; I wonder
if .NET is overzealous in its whining.

> > - Show quoted text -- Hide quoted text -