Re: compressed response problem with gitit+happstack
Gwern Branwen
>
> +++ Anton van Straaten [Apr 11 09 23:16 ]:
>>
>> On #darcs, twb mentioned a problem with Gitit when using clients like
>> curl, w3m, wget, elinks, and lynx: all requests fail, typically with
>> HTTP 404 and an error haiku, which comes from Happstack.
>>
>> gwern guessed what the problem was and I confirmed it: the problem
>> happens when the client doesn't specify that it supports compressed
>> responses. Clients like curl will work if you specify that compressed
>> responses should be accepted.
>>
>> This bug has apparently been fixed in the Happstack dev version, as of
>> March 28:
>>
>> http://groups.google.com.gi/group/HAppS/browse_thread/thread/419a012cdc60cd47
>>
>> However, if you build a recent version of Gitit that uses Happstack's
>> compressedResponseFilter, against a version of Happstack without the
>> above fix, it'll be broken in this way.
>>
>> I haven't verified that the Happstack fix actually resolves this. I'll
>> probably do that tomorrow and report back. There's a new Happstack
>> release scheduled for April 20, so it would be good to make sure
>> everything works with that.
>
> I'd noticed the problem with curl. Thanks for the explanation.
> It seems that this will go away once the new Happstack is out.
> Has anyone tried compiling gitit against the latest darcs version
> of happstack?
>
> John
I've pulled from patch-tag and cleaned and rebuilt happstack-util, happstack-data, happstack-server, and gitit, and made sure that the patch mentioned in that thread, 'compressedResponseFilter: don't mzero out if no "Accept-Encoding" header in the request' is present in my repo.
It doesn't seem to make a difference to either curl or elinks.
--
gwern
Gregory Collins
> I've pulled from patch-tag and cleaned and rebuilt happstack-util,
> happstack-data, happstack-server, and gitit, and made sure that the
> patch mentioned in that thread, 'compressedResponseFilter: don't mzero
> out if no "Accept-Encoding" header in the request' is present in my
> repo.
>
> It doesn't seem to make a difference to either curl or elinks.
"Works for me," for what it's worth, with wget, curl, "curl
--compressed", and web browsers.
GHC can do pretty aggressive cross-module inlining, so it may be that
you have old code inlined in some other library. (Usually you get a
linker error when that happens though.)
I know you said you did this already, but just to make sure it would be
a good idea to do a full clean & install of all libraries; in happstack
there are some scripts in the bin/ directory that will take care of that
for you. Clean and rebuild any libraries which depend on happstack also
(like happstack-helpers and gitit.)
If it's still busted maybe you can strap a tcpdump across the server
connection and send the output. On my OSX box the command to do this
looks like:
$ sudo tcpdump -i lo0 -A -s 0 port 5050 > tcp.dump
G.
--
Gregory Collins <gr...@gregorycollins.net>
Gwern Branwen
<gr...@gregorycollins.net> wrote:
> tcpdump -i lo0 -A -s 0 port 5050
After discussions with Eric, it seems that my problem stemmed from
gitit.cabal declaring an upper-bound on happstack-server - so -server
was being installed but not used.
Kowey updated http://test.darcs.net to use darcs-everything; currently
curl now works on it, but w3m bombs and elinks displays a HTTP 406
error (attached). So I wonder if there is another bug?
--
gwern
Gregory Collins
> On Sun, Apr 12, 2009 at 2:10 PM, Gregory Collins
> <gr...@gregorycollins.net> wrote:
>> tcpdump -i lo0 -A -s 0 port 5050
>
> Kowey updated http://test.darcs.net to use darcs-everything; currently
> curl now works on it, but w3m bombs and elinks displays a HTTP 406
> error (attached). So I wonder if there is another bug?
I'd like to see the request headers. Could you provide a dump?
According to the spec, if the client sends an "Accept-Encoding" header
we don't understand, 406 is the correct response. I'm of the opinion
that we should be a little laxer about this.
G
--
Gregory Collins <gr...@gregorycollins.net>
Gwern Branwen
[11:16 AM] 302.3Mb$ sudo tcpdump -i lo -A -s 0 port 5001 -vv
tcpdump: listening on lo, link-type EN10MB (Ethernet), capture size 65535 bytes
11:16:15.720953 IP (tos 0x0, ttl 64, id 23376, offset 0, flags [DF], proto TCP (6), length 60) localhost.35368 > localhost.5001: S, cksum 0xd293 (correct), 2017571871:2017571871(0) win 32792 <mss 16396,sackOK,timestamp 60354383 0,nop,wscale 6>
E..<[P@.@..i.........(..xA................@....
...O........
11:16:15.720968 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 60) localhost.5001 > localhost.35368: S, cksum 0x0f79 (correct), 2022856615:2022856615(0) ack 2017571872 win 32768 <mss 16396,sackOK,timestamp 60354383 60354383,nop,wscale 6>
E..<..@.@.<............(x.W.xA. .....y....@....
...O...O....
11:16:15.720981 IP (tos 0x0, ttl 64, id 23377, offset 0, flags [DF], proto TCP (6), length 52) localhost.35368 > localhost.5001: ., cksum 0xf69b (correct), 1:1(0) ack 1 win 513 <nop,nop,timestamp 60354383 60354383>
E..4[Q@.@..p.........(..xA. x.W............
...O...O
11:16:15.721247 IP (tos 0x0, ttl 64, id 23378, offset 0, flags [DF], proto TCP (6), length 292) localhost.35368 > localhost.5001: P, cksum 0xff18 (incorrect (-> 0xf225), 1:241(240) ack 1 win 513 <nop,nop,timestamp 60354383 60354383>
E..$[R@.@............(..xA. x.W............
...O...OGET / HTTP/1.1
Host: 127.0.0.1:5001
User-Agent: ELinks/0.12~pre2.dfsg0-1ubuntu1 (textmode; Debian; Linux 2.6.28-8-generic i686; 116x42-1)
Accept: */*
Accept-Encoding: bzip2, deflate, gzip
Accept-Language: en
Connection: Keep-Alive
11:16:15.721258 IP (tos 0x0, ttl 64, id 47335, offset 0, flags [DF], proto TCP (6), length 52) localhost.5001 > localhost.35368: ., cksum 0xf59b (correct), 1:1(0) ack 241 win 529 <nop,nop,timestamp 60354383 60354383>
E..4..@.@..............(x.W.xA.............
...O...O
11:16:15.728996 IP (tos 0x0, ttl 64, id 47336, offset 0, flags [DF], proto TCP (6), length 212) localhost.5001 > localhost.35368: P, cksum 0xfec8 (incorrect (-> 0x1212), 1:161(160) ack 241 win 529 <nop,nop,timestamp 60354385 60354383>
E.....@.@..9...........(x.W.xA.............
...Q...OHTTP/1.1 406 Not Acceptable
Connection: Keep-Alive
Content-Length: 0
Content-Type: text/plain
Date: Wed, 15 Apr 2009 15:16:15 GMT
Server: Happstack/0.3
11:16:15.729008 IP (tos 0x0, ttl 64, id 23379, offset 0, flags [DF], proto TCP (6), length 52) localhost.35368 > localhost.5001: ., cksum 0xf4f6 (correct), 241:241(0) ack 161 win 530 <nop,nop,timestamp 60354385 60354385>
E..4[S@.@..n.........(..xA..x.XH...........
...Q...Q
11:16:22.359042 IP (tos 0x0, ttl 64, id 23380, offset 0, flags [DF], proto TCP (6), length 52) localhost.35368 > localhost.5001: F, cksum 0xee7c (correct), 241:241(0) ack 161 win 530 <nop,nop,timestamp 60356042 60354385>
E..4[T@.@..m.........(..xA..x.XH.....|.....
.......Q
11:16:22.359105 IP (tos 0x0, ttl 64, id 47337, offset 0, flags [DF], proto TCP (6), length 52) localhost.5001 > localhost.35368: F, cksum 0xe803 (correct), 161:161(0) ack 242 win 529 <nop,nop,timestamp 60356042 60356042>
E..4..@.@..............(x.XHxA.............
........
11:16:22.359123 IP (tos 0x0, ttl 64, id 23381, offset 0, flags [DF], proto TCP (6), length 52) localhost.35368 > localhost.5001: ., cksum 0xe802 (correct), 242:242(0) ack 162 win 530 <nop,nop,timestamp 60356042 60356042>
E..4[U@.@..l.........(..xA..x.XI...........
........
At a guess, -server isn't handling 'bzip2' well.
--
gwern
Gregory Collins
> On Wed, Apr 15, 2009 at 10:57 AM, Gregory Collins <gr...@gregorycollins.net> wrote:
>
> After some tweaking, this is what I got for a single Elinks request:
The parser didn't like the "2" in "bzip2". Attached a fix.
Gwern Branwen
<gr...@gregorycollins.net> wrote:
>> On Wed, Apr 15, 2009 at 10:57 AM, Gregory Collins <gr...@gregorycollins.net> wrote:
>>
>> After some tweaking, this is what I got for a single Elinks request:
>
> The parser didn't like the "2" in "bzip2". Attached a fix.
With that patch, elinks is now working with Gitit. Great!
--
gwern
Matthew Elder
The parser didn't like the "2" in "bzip2". Attached a fix.