SSL request times out

[Chris Scribner]

I'm having problems with node.js being able to call a particular URL over SSL. I can curl the URL with no problems, so I think it must be an incompatibility between node.js and something on the remote host's network.

CURL:

curl https://oldbb.unmc.edu

(receives response)

Node.js:

var https = require('https');

var options = {

  hostname: 'oldbb.unmc.edu',

  port: 443,

  path: '/',

  method: 'GET',

  headers: { //Add headers used by curl

      Accept: '*/*',

      'User-Agent': 'curl/7.24.0 (x86_64-apple-darwin12.0) libcurl/7.24.0 OpenSSL/0.9.8r zlib/1.2.5'

  },

  agent: false //Tried with and without this, doesn't seem to matter

};

var req = https.request(options, function(res) {

  console.log("statusCode: ", res.statusCode);

  console.log("headers: ", res.headers);

  res.on('data', function(d) {

    process.stdout.write(d);

  });

});

req.end();

req.on('error', function(e) {

  console.error(e);

});

Running the node.js version just hangs. I've inspected the web requests from both curl and node and verified that they are sending the same headers.

Requesting the "http" version of the same URL does work with node. I feel like there's some incompatibility between node's SSL and the remote host's SSL (it is using IIS 7.5).

The same request also doesn't work using the request library, and I also tried the setting strictSSL: false with it.

Anyone have any ideas?

Thanks,

Chris

[Ben Noordhuis]

You didn't mention the version of node.js but it's possible IIS (or
the SSL terminator that's sitting in front of it) is getting confused
by the SSL/TLS versions, extensions or cipher suites that the TLS
client announces support for. Try different 'secureProtocol'[1]
values.

[1] http://nodejs.org/docs/latest/api/tls.html#tls_tls_connect_options_callback

[Chris Scribner]

I'm using node v0.10.15. 

Thanks for the tip - I'll play around with that.

[Chris Scribner]

I verified using wireshark that both node and curl are using TLS 1.0 (packet payload starts with 0x16 0x03 0x01). Must be some other aspect of the client hello that's throwing it off - they do vary in size considerably. Node's is 409 bytes, and curl's is 186 bytes.

[Ben Noordhuis]

Try { secureProtocol: 'SSLv3_method' } or { ciphers:
'RC4:HIGH:!MD5:!aNULL:!EDH' } (TLS 1.0 ciphers).

It's possible that the server is tripping on one of the advertised
extensions (servername, next proto.)

That last one you can override by passing { NPNProtocols: [] }, i.e.
an empty array.

The first one probably isn't that easy to override from the https
agent but it's the { servername: null } option to tls.connect().

[Chris Scribner]

Setting secureProtocol: 'SSLv3_method' worked! Thanks!

[Chris Scribner]

The thing that's not really clear to me is why curl works, and node doesn't. Both were initially sending TLS 1.0 in the Client Hello packet. Is it possible curl was also sending something along that indicated SSLv3 was also supported, but node wasn't?

[Chris Scribner]

Also, do you know if specifying SSLv3_method is always safe, or whether it might cause it not to work with older SSL terminators?

[Ben Noordhuis]

No, that's not really how SSL/TLS works.

It's possible there's a defective SSL terminator sitting in front of
the web server. I've seen cases where a TLS header > 256 bytes or an
extensions field > 64 bytes made the terminator silently drop the
connection.

[Chris Scribner]

That's probably what's going on in this case. Setting SSLv3_method reduces the packet size from 409 bytes to 218 bytes. The terminator isn't throwing an error - it's just silently dropping the connection.

I'm going to play around with options and see if I can tweak anything else to get the packet size down. We would prefer not to downgrade communication with all our clients to SSLv3.

[Chris Scribner]

Setting NPNProtocols: [] didn't seem to affect packet size, and it looks like node doesn't pass through the servername parameter to TLS. I'm not seeing any other options to play with.

[Matt]

FWIW we saw this in production and had to turn off SNI to make it work. The problem is a bug in F5s that has been fixed but your endpoint obviously hasn't been patched yet. We saw this at Amex but they have since patched their F5s at our request.

There's some info on this page: http://support.f5.com/kb/en-us/solutions/public/13000/600/sol13687.html

On Mon, Jul 29, 2013 at 6:19 PM, Chris Scribner <scr...@gmail.com> wrote:

Setting NPNProtocols: [] didn't seem to affect packet size, and it looks like node doesn't pass through the servername parameter to TLS. I'm not seeing any other options to play with.

--
--
Job Board: http://jobs.nodejs.org/
Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nod...@googlegroups.com
To unsubscribe from this group, send email to
nodejs+un...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en
 
---
You received this message because you are subscribed to the Google Groups "nodejs" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nodejs+un...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Chris Scribner]

Would you mind providing details about how you disabled SNI?

[Matt]

First we tried the (undocumented):

    process.features.tls_sni = false;

Then we settled on modifying the http client request object directly:

    r.servername = 'x'; // shorten TLS server_name extension

That gave us enough headroom to sneak under 256 bytes.

On Tue, Jul 30, 2013 at 10:56 AM, Chris Scribner <scr...@gmail.com> wrote:

Would you mind providing details about how you disabled SNI?

--

[Chris Scribner]

Hmm, I tried both process.features.tls_sn = false, and specifying req.servername = 'x' directly after calling https.request, but neither reduced the Client Hello packet size.

I wonder if you're using an older version of node and that workaround broke?

I noticed that if I manually set a "Host" header in the request, it shortens the packet a little bit (but not enough). I got the idea from http://blog.nodejs.org/2012/08/02/node-v0-8-5-stable/: "https: Use host header as effective servername (isaacs)".

[Chris Scribner]

If someone, some day, runs across this post, specifying this cipher list explicitly is another workaround. Using 'AES128-SHA:AES256-SHA:DES-CBC3-SHA:RC4-MD5:RC4-SHA' seems to work. This list keeps the packet size down, and has good universal support (this is the default cipher list from Amazon EC2 load balancers).

[Matt]

Yeah this was on Node v0.6. Since Amex upgraded their F5s we have disabled the hack (and the one we had to put in PhantomJS to do the same).

On Tue, Jul 30, 2013 at 2:24 PM, Chris Scribner <scr...@gmail.com> wrote:

If someone, some day, runs across this post, specifying this cipher list explicitly is another workaround. Using 'AES128-SHA:AES256-SHA:DES-CBC3-SHA:RC4-MD5:RC4-SHA' seems to work. This list keeps the packet size down, and has good universal support (this is the default cipher list from Amazon EC2 load balancers).

--

[Robert Soden]

Just wanted to drop a note here, this bit about setting the cipher list fixed the very same issue for me. Been struggling with this for DAYS. Trying to make an SSL request to a Tomcat machine, curl and everything else worked fine, no luck specifying only the secureProtocol option. However, dropped in { ciphers: 'RC4:HIGH:!MD5:!aNULL:!EDH' } worked like a charm.

THANK YOU!

[Liang Sun]

This issue cost me several days. This works for me too. Thank you!