Connection: close and POST requests
fidian
marvelously. There is one problem that we found - node will not
properly forward requests that are both using POST and with the
"Connection: close" header. If I use GET or if I omit the Connection:
close header, then things do work properly. With both at the same
time, I am unable to send a response. Here's my JS that illustrates
the problem. If I were to not set the timeout, there is no issue. If
I include the delay, I will not get "Hello World" sent back to the
client. It looks like res is closed too early.
require('http').createServer(function (req, res) {
setTimeout(function () {
res.writeHead(200, {'Content-Type': 'text/plain'});
res.end("Hello World\n");
}, 500);
}).listen(8001);
I've searched for help on the developers list and now seek anyone who
can solve this problem for me on this list. If there's some way I can
help in solving this, let me know.
fidian
On May 30, 10:41 am, fidian <minimifidian...@gmail.com> wrote:
> I am using Node to act as a proxy for work and it does the job
> marvelously. There is one problem that we found - node will not
Ben Noordhuis
Do you have a repeatable test case? (By the way, top-posting is a mortal sin).
fidian
> Do you have a repeatable test case? (By the way, top-posting is a mortal sin).
require('http').createServer(function (req, res) {
setTimeout(function () {
res.writeHead(200, {'Content-Type': 'text/plain'});
res.end("Hello World\n");
}, 500);
}).listen(8001);
localhost 8001" and paste this in and watch how you don't get the
response. Don't forget that you'll need a couple returns after
"Sample". If you remove "Connection: close", then it works just
great.
POST /post HTTP/1.1
Content-Type: application/x-www-form-urlencoded; charset=utf-8
Content-Length: 6
Connection: close
Host: localhost
Sample
Ben Noordhuis
I suspect your request body has a trailing CRLF. Your example works if
I bump the Content-Length to 8.
$ nc -Cv4 127.0.0.1 8001
Connection to 127.0.0.1 8001 port [tcp/*] succeeded!
POST /post HTTP/1.1
Content-Type: application/x-www-form-urlencoded; charset=utf-8
Content-Length: 8
Connection: close
Host: localhost
Sample
HTTP/1.1 200 OK
Content-Type: text/plain
Connection: close
Transfer-Encoding: chunked
c
Hello World
0
fidian
> I suspect your request body has a trailing CRLF. Your example works if
> I bump the Content-Length to 8.
My POST data does not contain CRLF and the Content-Length should be
6. I generated the traffic with some PHP code (file_get_contents()
with a custom stream). I've captured the traffic with
tcpwatch_httpproxy and here's the hex dump of my request. It does
send extra CR/LF characters at the end, but why would that kill the
properly formatted request at the top?
00000000 50 4f 53 54 20 2f 70 6f 73 74 20 48 54 54 50 2f |POST /
post HTTP/|
00000010 31 2e 31 0d 0a 43 6f 6e 74 65 6e 74 2d 54 79 70 |
1.1..Content-Typ|
00000020 65 3a 20 61 70 70 6c 69 63 61 74 69 6f 6e 2f 78 |e:
application/x|
00000030 2d 77 77 77 2d 66 6f 72 6d 2d 75 72 6c 65 6e 63 |-www-form-
urlenc|
00000040 6f 64 65 64 3b 20 63 68 61 72 73 65 74 3d 75 74 |oded;
charset=ut|
00000050 66 2d 38 0d 0a 43 6f 6e 74 65 6e 74 2d 4c 65 6e |
f-8..Content-Len|
00000060 67 74 68 3a 20 36 0d 0a 43 6f 6e 6e 65 63 74 69 |gth:
6..Connecti|
00000070 6f 6e 3a 20 63 6c 6f 73 65 0d 0a 48 6f 73 74 3a |on:
close..Host:|
00000080 20 6c 6f 63 61 6c 68 6f 73 74 0d 0a 0d 0a 53 61 |
localhost....Sa|
00000090 6d 70 6c 65 0d 0a 0d 0a |mple....|
I have three files. (1) a copy of the request from PHP with the extra
CR LF CR LF at the end (seen above), (2) the request with the last 4
characters removed, and (3) a GET request that looks great. 2 and 3
both will get a response from the server. Why is the response from 1
not working?
16:34 clove:~/node_bug$ nc -v -q 10 localhost 8001 < request.dump
Connection to localhost 8001 port [tcp/*] succeeded!
16:34 clove:~/node_bug$
Is the HTTP module reading more from the request and deciding it is
invalid? I don't disagree with that behavior, but it looks like it
closes the socket before the response for the first request is
served. If I pegged that correctly, shouldn't the first request get
fully processed before the second one is started? I wouldn't want
previous requests to get abandoned because the following request is
formatted poorly.
fidian
> with a custom stream). I've captured the traffic with
> tcpwatch_httpproxy and here's the hex dump of my request. It does
> send extra CR/LF characters at the end, but why would that kill the
> properly formatted request at the top?
the web if you want to download them.
> Is the HTTP module reading more from the request and deciding it is
> invalid? I don't disagree with that behavior, but it looks like it
> closes the socket before the response for the first request is
> served.
"Connection: close" header, everything works fine even though it has
the extra CR/LF CR/LF at the end. Why would the socket get closed
early with the "Connection: close" header and not closed early without
it?
Ben Noordhuis
> I have three files. (1) a copy of the request from PHP with the extra
> CR LF CR LF at the end (seen above), (2) the request with the last 4
> characters removed, and (3) a GET request that looks great. 2 and 3
> both will get a response from the server. Why is the response from 1
> not working?
>
> 16:34 clove:~/node_bug$ nc -v -q 10 localhost 8001 < request.dump
> Connection to localhost 8001 port [tcp/*] succeeded!
> 16:34 clove:~/node_bug$
Note that `nc` is tricky in that it won't print to stdout if stdin
isn't a tty. I've no idea why it does that - it's annoying.
> Is the HTTP module reading more from the request and deciding it is
> invalid? I don't disagree with that behavior, but it looks like it
> closes the socket before the response for the first request is
> served. If I pegged that correctly, shouldn't the first request get
> fully processed before the second one is started? I wouldn't want
> previous requests to get abandoned because the following request is
> formatted poorly.
On Tue, Jun 7, 2011 at 23:50, fidian <minimif...@gmail.com> wrote:
> Wow, the hex dump of my file got mangled. I can put the files up on
> the web if you want to download them.
Yes, please. It's rather hard to parse this way. If request #1 is okay
and request #2 is malformed, #1 should still be served.
> The other thing I forgot to mention was that if you remove the
> "Connection: close" header, everything works fine even though it has
> the extra CR/LF CR/LF at the end. Why would the socket get closed
> early with the "Connection: close" header and not closed early without
> it?
Because the HTTP parser thinks that the extraneous junk is the
beginning of a new request (if "Connection: close" isn't set).
Trailing CRLFs are fairly common, it seems, so the parser has a
special case built in that deals with them.
fidian
>
> Note that `nc` is tricky in that it won't print to stdout if stdin
> isn't a tty. I've no idea why it does that - it's annoying.
the Connection: close header and it also works fine if I chop off the
last 4 bytes.
I have put my files on the net: http://rumkin.com/media/support_files.tar.gz
I've filed an official "issue" on github for this problem. It has
some more information about where I think the problem lies, but I'm
still really new to this. https://github.com/joyent/node/issues/1165
Ben Noordhuis
>> > 16:34 clove:~/node_bug$ nc -v -q 10 localhost 8001 < request.dump
>> >Connectionto localhost 8001 port [tcp/*] succeeded!
>> > 16:34 clove:~/node_bug$
>>
>> Note that `nc` is tricky in that it won't print to stdout if stdin
>> isn't a tty. I've no idea why it does that - it's annoying.
>
> nc will print out if I use the same format with a request that omits
> the Connection: close header and it also works fine if I chop off the
> last 4 bytes.
Maybe it's the version of `nc` that ships with Ubuntu Lucid.
> I have put my files on the net: http://rumkin.com/media/support_files.tar.gz
>
> I've filed an official "issue" on github for this problem. It has
> some more information about where I think the problem lies, but I'm
> still really new to this. https://github.com/joyent/node/issues/1165
Using this[1] and this[2] and the example requests from your tarball,
I get back this:
$ cat tmp/request.noclose tmp/request.nocrlf | ./netcat.py localhost 8001
POST /post HTTP/1.1
Content-Type: application/x-www-form-urlencoded; charset=utf-8
Content-Length: 6
Host: localhost
Sample
POST /post HTTP/1.1
Content-Type: application/x-www-form-urlencoded; charset=utf-8
Content-Length: 6
Connection: close
Host: localhost
SampleHTTP/1.1 200 OK
Content-Type: text/plain
Content-Length: 12
Connection: keep-alive
Hello World
HTTP/1.1 200 OK
Content-Type: text/plain
Content-Length: 12
Connection: close
Hello World
Here is what server.js logs:
Request { 'content-type': 'application/x-www-form-urlencoded;
charset=utf-8',
'content-length': '6',
host: 'localhost' }
Received chunk <Buffer 53 61 6d 70 6c 65> Sample
Request { 'content-type': 'application/x-www-form-urlencoded;
charset=utf-8',
'content-length': '6',
connection: 'close',
host: 'localhost' }
Received chunk <Buffer 53 61 6d 70 6c 65> Sample
About what you would expect, right? This is with the master branch,
commit 7a5977b (June 7).
[1] https://gist.github.com/1015643 (netcat.py)
[2] https://gist.github.com/1015649 (server.js)
fidian
> Using this[1] and this[2] and the example requests from your tarball,
> I get back this:
>
> $ cat tmp/request.noclose tmp/request.nocrlf | ./netcat.py localhost 8001
Yes, that is exactly what I would expect. If I use your programs and
the "request.dump" file from my tarball, I get this:
$ cat request.dump | ./netcat.py localhost 8001
Content-Type: application/x-www-form-urlencoded; charset=utf-8
Content-Length: 6
Host: localhost
Sample
$ node server2.js
charset=utf-8',
'content-length': '6',
connection: 'close',
host: 'localhost' }
Received chunk <Buffer 53 61 6d 70 6c 65> Sample
> About what you would expect, right? This is with the master branch,
> commit 7a5977b (June 7).
My gripe is that I should still see a response when I send
"request.dump" to a node server.