That annoying POST bug people are seeing - I need help reproducing it

[Adrian Chadd]

Hi guys,

So now that university is over for another semester, I'd like to get
back into the swing of active development.

The #1 thing I'd like to identify and fix right now is this busted
POST behaviour that people are seeing on some websites. I know people
have talked with me about it but I can't seem to find all the right
details.

So, can someone please help me identify the problem? Even if it's just
a test case, that'll be helpful. :-)

Thanks,

Adrian

[Roelf Diedericks]

Have had some very vague intonations from customers as well that there
is still some broken
post behaviour with things like Yahoo, or Gmail, even post my "HTTP
1.1 100 Continue" fix.

Have not yet been able to get any solid tcpdumps or reproducibility on
it, and it
only seems to happen under some vague circumstances.

Will keep an eye out for some solid data.

[Adrian Chadd]

> --
> You received this message because you are subscribed to the Google Groups "lusca-users" group.
> To post to this group, send email to lusca...@googlegroups.com.
> To unsubscribe from this group, send email to lusca-users...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/lusca-users?hl=en.
>
>

Thanks. I think the real solution moving forward is implementing full
and proper HTTP/1.1 support.

I'll see if I can wrap up my IPv6 DNS related changes this week in my
branch and get that ready for committing to -HEAD. I'd really like to
finish off this pass of IPv6 work before I start thinking some more on
how to do HTTP/1.1 properly. There'd need to be a few things:

* it'd have to become a "pinned" connection, but then it's a POST and
likely should be;
* it'd need a method of communicating between client<->server sides,
so they can handle flow control themselves;
* it needs to be not (too) dirty. :-)

Adrian

[Roelf Diedericks]

OK, after some more investigation the "Yahoo" problems that were
reported to me was
related to a sem-transparent setup.

Yahoo considers a large amount of requests from a single IP address to
be "bot" behaviour
and they then return an HTTP 999 status code, along with an
explanatory body message indicating
you must talk to your ISP etc etc.

From what I could see Lusca won't pass along the body to the client
with an HTTP 999 status code
so clients report "blank pages" and random behaviour.

The recent StatusLine patch will probably address the passing along of
the body error message which
makes debugging the true issue a bit simpler. I have not however tried
that in production yet.

Hopefully this at least narrows the problem domain a little bit.

Bottom line, Yahoo doesn't like lots of HTTP requests from a single
IP, and returns a 999 status code
along with an informative message as to the cause.

On Dec 13, 2:19 am, Adrian Chadd <adrian.ch...@gmail.com> wrote:

> On 13 December 2010 05:27, Roelf Diedericks <roelf.diederi...@gmail.com> wrote:
>
>
>
>
>
>
>
>
>
> > Have had some very vague intonations from customers as well that there
> > is still some broken
> > post behaviour with things like Yahoo, or Gmail, even post my "HTTP
> > 1.1 100 Continue" fix.
>
> > Have not yet been able to get any solid tcpdumps or reproducibility on
> > it, and it
> > only seems to happen under some vague circumstances.
>
> > Will keep an eye out for some solid data.>
> > --
> > You received this message because you are subscribed to the Google Groups "lusca-users" group.
> > To post to this group, send email to lusca...@googlegroups.com.
> > To unsubscribe from this group, send email to lusca-users...@googlegroups.com.

> > For more options, visit this group athttp://groups.google.com/group/lusca-users?hl=en.

[Adrian Chadd]

Yeah, I don't like that 999 status thing. But it exposes a bug - Lusca
should be returning an error, not just hanging.

So my change to bump the "invalid" codes fixes the symptom but not the
underlying problem. That still needs to be identified and fixed. :/

Adrian

[Roelf Diedericks]

Did you rebuild lusca cleanly ? I applied the same patch and don't get
a connection that hangs anymore.

Seem my comments at http://code.google.com/p/lusca-cache/issues/detail?id=134

There's a piece of code I quoted that seems suspect.

On Dec 14, 2:52 am, Adrian Chadd <adrian.ch...@gmail.com> wrote:
> Yeah, I don't like that 999 status thing. But it exposes a bug - Lusca
> should be returning an error, not just hanging.
>
> So my change to bump the "invalid" codes fixes the symptom but not the
> underlying problem. That still needs to be identified and fixed. :/
>
> Adrian
>

[Adrian Chadd]

Yeah, and I replied to that. :-)

The comment in the code is mine. The problem is that the connection
isn't "completed" somehow. See what the rest of the routine does in
terms of either aborting a connection (which finishes it) or begins
transferring data (where "finishing it" happens later.) This routine
simply returns that some data was consumed, but doesn't at all
terminate the connection with an error.

Adrian