Increase of parallel requests form node.js to external system takes more time to respond

[Kishore Kumar Yekkanti]

I have a simple case where I'm requesting a different upstream proxy server from my node.js server. With the increase in load I see the request takes lot of time to execute(though time taken to respond from my upstream proxy server is constant across the requests). To demonstrate the issue i've written a sample program as below. If you execute the below program, the first request takes 118ms to execute and the last one takes 10970ms. If you observe i'm using async to parallelize my requests.

The question is, what is the reason node.js takes this much time to execute a request when run in parallel. To give some more context on the infra settings(centos 6.5) I have opened up my port range from 1024 to 65535, change the fin_timeout to 15 seconds and enable tw_reuse =1 for sockets in sysctl.conf

var http = require('http');
var uuid = require('node-uuid');
var async = require('async');

function callExternalUrl(){
    var uniqueId = uuid.v4();
    console.time(uniqueId);
    var options = {
        host: 'google.com',
        port: '80',
        path: '/',
        method: 'GET'
    };
    var req = http.request(options, function(res) {
        var msg = '';
        res.setEncoding('utf8');
        res.on('data', function(chunk) {
            msg += chunk;
            console.timeEnd(uniqueId);
        });
        res.on('end', function() {
        });
    });
    req.end();
}

function iterateAsync(callback){
    var iter = [];
    for(var i=0; i<1000; i++){
        iter[i] = i;
    }
    async.each(iter,
        function(item, callback) {
            callExternalUrl();
        },
        function(err) {
            callback(err);
        }
    );
}

iterateAsync(function(){console.log('done');});

[Ryan Graham]

Are you using a version of node that defaults to only 5 sockets for the global agent?

~Ryan

--
Job board: http://jobs.nodejs.org/
New group rules: https://gist.github.com/othiym23/9886289#file-moderation-policy-md
Old group rules: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
---
You received this message because you are subscribed to the Google Groups "nodejs" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nodejs+un...@googlegroups.com.
To post to this group, send email to nod...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/nodejs/e45d0fac-f0d2-45b9-9eeb-4dcc0a4fe650%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Kishore Kumar Yekkanti]

Yes, Ryan. I'm using node 0.10 which restricts it to 5 sockets. I tried two variants. node 0.10 with bumping up the sockets to 999 and node 0.12(which by default sets the max sockets to Infinity. Both yielded with same results.

[kodamatic]

Missing call callback?

[Marcos Mendez]

I think so to..

function(item, callback) {
            callExternalUrl();
        },

callback never gets called.

On Tuesday, April 7, 2015 at 11:43:36 PM UTC-4, kodamatic wrote:

Missing call callback?

[Kishore Kumar Yekkanti]

In my original code i had a callback, missed it in this sample codebase. That said the issue was not with the callback at least in my case. Its because of the maxSockets of http node library. I'm using node 0.10 and as part of the spec its mentioned that max of 5 socket connections can be connection pooled. i bumped up this number to 100 and i can see faster response times.  

But one question still remains, when i bump up this number to something extremely higher like 999999, it fallsback to the default behaviour and the response time increases. I guess it could be my hardware limit also. Nevertheless i'll look into the http codebase to understand this behaviour in specific. 

[Jimb Esser]

There are all sorts of things that could cause this behavior.  A couple significant ones:  If you have an extremely high number of max sockets, that means every request makes a socket, and no connection pooling is going on.  If your connections are https, that's going to add a humongous amount of overhead dwarfing anything else (whereas if you're using only 100 connections and pooling them you pay that overhead once, and then get a lot of quick requests sent over the same connection).  Also, if your actual test is using https/ssl, just the local CPU usage of opening thousands of connections (which, I suspect only happens in one thread per node process) could explain some of it.

If you're actually testing this on a 3rd party server (like google.com in your example), then their load balancing and DDoS configuration of their network comes into play.  For example, if they have a limit of how many active connections per source IP get through some layer of their incredibly complicated and heavily layered network topology, even if your computer tries to open 1000 TCP connections at once, all but the first handful will just have to wait on the other end.

As for the specifics of your examples, your numbers sound just as expected - with default of 5 max sockets, parallelism is 5, with a request time of around 100ms, 1000 requests will take about 20000ms.  On the box I'm currently on (OSX, node 0.10.25), if I increase http.globalAgent.maxSockets to 1000 (each one gets its own socket, which I can verify in netstat, if I'm quick), I see it takes about 60ms just to issue the 1000 requests (timing before and after calling async.each), and then the first request to come back has a total response time of about 170ms, the last one comes back in about 730ms, so I'm not seeing the slowdown you're seeing, which sounds like it may just be something on your end throttling requests (could be OS issue, router issue, ISP issue, or any number of things between your and www.google.com).

Also of note, running node with "time", only about 20% of the time is idle (e.g. most of the time is spent in node on my own local machine, presumably the overhead of creating new connections, etc, which is done in serial, not waiting for responses to come back, which is what happens in parallel).

Also of note, in the time I ran these tests and wrote this email, I think Google's network has now flagged my IP as a possible DoS attack, and now it looks like I'm getting throttled and everything is taking quite a bit longer than it was when I first ran the test.  If you truly want to test node, you need to be hitting a server which you control and know the performance characteristics of, otherwise you may not be testing what you think you're testing.

  Jimb Esser