Multiple HTTP requests merging together

[Chris]

Hello,

I have a situation in which I see multiple HTTP requests merging
together into a single confusing result. This happens ONLY when
Fiddler is running, and only with Firefox.

It looks like 5 - 10 requests go through ok (resulting in status codes
of 200 or 304). Then a request goes through to a different server.
The request headers LOOK ok in fidder, but the response body is rather
mangled. Fiddler reports a violation of the HTTP status protocol
because the server set a "Content-Length" header of XXXX but actually
sent YYYY bytes. Looking at "raw" view, the response actually
contains several sets of headers and bodies: it looks like the
previous 8 requests or so were sent to the different server all at
once (each resulting in a 404), and the responses were all
concatenated together into a single response.

After this happens, most (but not all) http requests return unexpected
results, as though they are out of alignment. For instance, a request
for "default.htm" will give me "default.htm", but the very next
request for "layout.css" will give me the contents of "default.htm"
again.

If I reissue the "weird" request in fiddler, it goes through just
fine, no problems. However, this does not affect the problem with
subsequent requests being out of alignment.

If I close fidder and reload a couple of times, the problem with
requests being out of alignment resolves itself. If I re-open
fiddler, everything is fine until a new instance of the same problem
crops up again.

I can reproduce this, but only on one machine so far. I can generate
SAZ files that illustrate this behavior.

Any ideas?

[EricLaw]

Inside Firefox, type "about:config" in the address line.
In the search box, type "pipelining"

What are the values for the settings?

[Chris]

Ah ha!

network.http.pipelining: true
network.http.pipelining.maxrequests: 10
network.http.pipelining.ssl: false
network.http.proxy.pipelining: true

Setting these back to the default values fixed the problem.

Interestingly, the problem seemed to be specific to a single site
(possibly to a specific server). Other sites continued to work
normally.

Let me know if you'd like any more information about this. Otherwise,
thanks for your help!

- Chris

[EricLaw]

Thanks, Chris!

Yes, unfortunately many proxies and network devices (including
Fiddler, I'm sad to say) do not support HTTP pipelining, which is why
it's currently disabled by default in all browsers except Opera.
(Opera is a special case where they have tons of code to try to detect
non-pipelining-aware servers/proxies and recover).

At some point in the future, I want to add pipelining support to
Fiddler (if nothing else, it would be an interesting exercise), but I
haven't had the chance to do this yet.

-Eric

> > > Any ideas?- Hide quoted text -
>
> - Show quoted text -

[Kendall]

Hi Eric,

Do you have an idea of why I was experiencing this but another machine
which also used the same settings was not? I don't know if it makes a
difference but he's running Vista while I'm running XP. Both using the
same version of Firefox and we're both on the same network. While I
could reproduce it easily by going to a particular site and refreshing
several times, I was unable to duplicate the problem on his machine
going to that same site and refreshing.

[EricLaw]

Are you saying both Firefox instances had the option
network.http.proxy.pipelining set to true?

I'd imagine that Firefox takes a lot of different factors into account
when deciding whether or not a given request should be pipelined;
those factors could mean that even if pipelining is permitted in the
configuration, it may not be used in the cases you tested.

> > -Eric- Hide quoted text -

[Kendall]

Yes both machines had it set to true but your explanation makes sense.
That was something I proposed could be but was not certain but it was
the only answer that made sense.

Thanks.