HTTP client request headers (lowercase) issue

[Eran Hammer-Lahav]

A recent update changed how client request headers are sent to force all headers to lowercase. This is causing problems with some well-established (and non-compliant) servers which look for specific case-sensitive formatting. My application is now failing on both the 'Authorization' and 'Cookie' headers with some large providers.

I see the value in changing incoming headers to lowercase, but it would be better to leave outbound headers alone and allow applications to set the headers to the exact format expected by the server in case of a misbehaving provider.

Can this be changed back?

EHL

[Dean Landolt]

On Tue, Mar 8, 2011 at 7:43 PM, Eran Hammer-Lahav <er...@hueniverse.com> wrote:

A recent update changed how client request headers are sent to force all headers to lowercase. This is causing problems with some well-established (and non-compliant) servers which look for specific case-sensitive formatting. My application is now failing on both the 'Authorization' and 'Cookie' headers with some large providers.

Out of curiosity can you tell us who they are? Perhaps a little name and shame can help -- after all, this is a pretty egregious oversight.

 

I see the value in changing incoming headers to lowercase, but it would be better to leave outbound headers alone and allow applications to set the headers to the exact format expected by the server in case of a misbehaving provider.

Can this be changed back?

EHL

--
You received this message because you are subscribed to the Google Groups "nodejs" group.
To post to this group, send email to nod...@googlegroups.com.
To unsubscribe from this group, send email to nodejs+un...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/nodejs?hl=en.

[Tim Caswell]

Interesting, I made very sure in my patch to the headers to preserve the header casing of outgoing headers for this exact reason.  

[Eran Hammer-Lahav]

For a second (actually an hour or two), I was sure I was losing my mind. But then I was able to reproduce this with:

---

var http = require('http');

function request() {

    var options = { host: 'yahoo.com',

                    port: 80,

                    path: '/',

                    method: 'GET',

                    headers: { Test: 'value' } };

    req = http.request(options);

    req.end();

}

request();

var connect = require('connect');

request();

---

Which generates (via tcpflow):

---

010.000.020.120.38274-072.030.002.043.00080: GET / HTTP/1.1

Test: value

Host: yahoo.com

Connection: close

010.000.020.120.38275-072.030.002.043.00080: GET / HTTP/1.1

test: value

host: yahoo.com

Connection: close

---

For some reason (too tired to look deeper right now), just loading connect does something odd.

EHL

[Eran Hammer-Lahav]

Looks like the issue is in connect/lib/patch.js. Not sure why that code is there but it changes all outgoing HTTP messages.

[tjholowaychuk]

my bad, oversight since the http client lower-cases I didnt catch
that. fixing

[vision media [ Tj Holowaychuk ]]

ps. stack traces are your friend

On Wed, Mar 9, 2011 at 2:41 PM, Eran Hammer-Lahav <er...@hueniverse.com> wrote:

Looks like the issue is in connect/lib/patch.js. Not sure why that code is there but it changes all outgoing HTTP messages.

--
Tj Holowaychuk
Vision Media
President & Creative Lead

[Dan Milon]

AFAIK outgoing headers are not lowercased. Do you have a test case?

danmilon.

On Sat, Dec 1, 2012 at 10:01 PM, Cristian Bullokles
<cristian....@gmail.com> wrote:
> Looks like this change stills in v0.8.15 of node.js, I'm trying to
> authenticate to an OAuth provider and I've the same problem. My provider is
> not able to recognize the authorization header.
> Do we have a fix for that issue?

> --
> Job Board: http://jobs.nodejs.org/
> Posting guidelines:
> https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines

> You received this message because you are subscribed to the Google
> Groups "nodejs" group.
> To post to this group, send email to nod...@googlegroups.com

> To unsubscribe from this group, send email to
> nodejs+un...@googlegroups.com

> For more options, visit this group at

> http://groups.google.com/group/nodejs?hl=en?hl=en

[Mikeal Rogers]

by default node.js sends headers in lowercase. by spec, HTTP headers are caseless. if someone does not treat them that way you email them and ask them why they suck at HTTP.

[Ben Noordhuis]

On Sun, Dec 2, 2012 at 2:07 AM, Mikeal Rogers <mikeal...@gmail.com> wrote:
> by default node.js sends headers in lowercase.

The core http library sends headers as-is, it's only incoming headers
that are lowercased.

Maybe some frameworks force lowercase everywhere but node.js itself does not.

[Dan Milon]

I just tested this, and it seems it wont touch them. [1]
I recall having this issue with S3 not recognizing lowercased
content-length header. And explicitly setting the header key as
Content-Length fixed it.

[1] https://gist.github.com/4186458

danmilon.

[Mikeal Rogers]

headers core sets, like transfer-encoding and host will be lowercase.

[Ben Noordhuis]

On Sun, Dec 2, 2012 at 3:53 AM, Mikeal Rogers <mikeal...@gmail.com> wrote:
> headers core sets, like transfer-encoding and host will be lowercase.

Wrong again. Auto-added headers like Authorization, Date,
Content-Length, Transfer-Encoding, etc. are all properly capitalized.

[Cristian Bullokles]

Debugging my code with node-inspector I found the code below. 

The problem is in line:

  var key = name.toLowerCase();

What is the value added of lowercase headers in outgoing messages?

Regards

Cristian

File: http.js

Line 639 to 653

OutgoingMessage.prototype.setHeader = function(name, value) {

  if (arguments.length < 2) {

    throw new Error('`name` and `value` are required for setHeader().');

  }

  if (this._header) {

    throw new Error('Can\'t set headers after they are sent.');

  }

  var key = name.toLowerCase();

  this._headers = this._headers || {};

  this._headerNames = this._headerNames || {};

  this._headers[key] = value;

  this._headerNames[key] = name;

};

[Isaac Schlueter]

So, in that bit of code, if you do setHeader('x-FoO', 'baR'), then:

this._headers['x-foo'] === 'baR'
this._headerNames['x-foo'] === 'x-FoO'

Later, when it actually writes the header lines, it writes:

this._headerNames['x-foo'] + ': ' + this._headers['x-foo'] + '\r\n'

or in other words:

'x-FoO: baR\r\n'

just like you asked it to.

This code is about the outgoing headers that *your app* sends out.
(Ie, the HTTP response in a Server, or the HTTP request in a Client.)


Without theories about *why* it is, can you you clarify what you're
actually having problems with?

1. My node http server is sending lowercased headers in the response.
2. My node http server is receiving lowercased headers in the request.
3. My node http client is sending lowercased headers in the request.
4. My node http client is receiving lowercased headers in the response.

[Cristian Bullokles]

3. My node http client is sending lowercased headers in the request.

I'm sending this headers:

Authorization: Bearer <token>

and the http.request client is sending:

authorization: Bearer <token>

I hope that helps.

-CB

[mscdex]

On Dec 2, 10:06 am, Cristian Bullokles <cristian.bullok...@gmail.com>
wrote:

> 3. My node http client is sending lowercased headers in the request.
>
> I'm sending this headers:
>
> Authorization: Bearer <token>
>
> and the http.request client is sending:
>
> authorization: Bearer <token>

How are you verifying the case of the header name? If you're testing
with a node server and doing `console.dir(req.headers);` it will show
a lowercase key in the object. However, if you use tcpdump/Wireshark/
etc., you will see the node http client _does_ send the header name as
"Authorization."

[Isaac Schlueter]

Yeah, show me the failing test, please.

[Cristian Bullokles]

Will, try... but now I'm guessing that I saw the debugguer but the reality is that it is sending the right headers.

Let me check with wireshark.

[Dave Winer]

I seem to be having this problem, not sure what the solution or workaround is.

I'm writing the head as follows:

response.writeHead (200, {"Content-Type": "text/html", "Access-Control-Allow-Origin": "fargo.io"});

When it shows up on the client, the header names are lowercased.

Happens in two clients, so I'm fairly sure it's the server doing this.

It doesn't look like this was resolved in 2012.

Help! :-)

Dave

[Michael Hart]

What node version are you using? Works fine for me on v0.10.25:

$ cat test.js

require('http').createServer(function(req, res) {

  res.writeHead(200, {Something: 'yep'})

  res.end()

}).listen(3000)

$ node test.js &

[1] 85457

$ curl -v localhost:3000

* Adding handle: conn: 0x7fdc09004000

* Adding handle: send: 0

* Adding handle: recv: 0

* Curl_addHandleToPipeline: length: 1

* - Conn 0 (0x7fdc09004000) send_pipe: 1, recv_pipe: 0

* About to connect() to localhost port 3000 (#0)

*   Trying ::1...

*   Trying 127.0.0.1...

* Connected to localhost (127.0.0.1) port 3000 (#0)

> GET / HTTP/1.1

> User-Agent: curl/7.30.0

> Host: localhost:3000

> Accept: */*

>

< HTTP/1.1 200 OK

< Something: yep

< Date: Fri, 24 Jan 2014 06:59:58 GMT

< Connection: keep-alive

< Transfer-Encoding: chunked

<

* Connection #0 to host localhost left intact