Interrupting long-running HTTP requests or infinite loops

[David]

I am trying to figure out a way to recover (as gracefully as possible)
from HTTP requests handled by node as an HTTP server from taking too
long.

That is, if the callback provided to http.createServer() doesn't
complete in some amount of time, I'd like to be able to take some
action rather than just let things spin.

process.Timer() doesn't seem to help since the libev event loop is not
engaged if V8 is being greedy and not giving up any control. E.g. in
the worst-case scenario of:

==
var sys = require('sys');
var http = require('http');

var handler = function(req, res) {
var timeoutId = setTimeout(function() { sys.puts
('interrupted'); }, 500);
res.sendHeader(200, { 'Content-Type': 'text/html' });
res.sendBody("Hello World\n");

while(true) { /* Ye olde infinite loop */ }

res.finish();
clearTimeout(timeoutId);
};

http.createServer(handler).listen(8000);
sys.puts("Server running on port 8000");
==

the client never receives a reply.

I played around with adding an exposed-to-JS function that uses
setitimer() to cause a SIGALRM to be delivered to a signal handler
which calls V8::TerminateExecution(); This indeed stops execution but
it is of course for the entire process, not just the iloop.

FWIW, the uncaughtException event that Felix Geisendörfer added in
response to Ryan's task proposal nicely catches the "uncatchable"
exception that V8::TerminateExecution() uses to terminate things. But
it seems the engine is not in a state to fully recover at this point
so that my uncaughtException callback's attempts to call res.finish()
fail. (In the code below, The process.alarm.* functions are the
wrappers around setitimer() that I added locally.)

==
var sys = require('sys');
var http = require('http');


var handler = function(req, res) {
var finisher = function(exception) {
res.finish();
};
process.addListener("uncaughtException", finisher);
process.alarm.set();
res.sendHeader(200, { 'Content-Type': 'text/html' });
res.sendBody("Hello World\n");
while(true) {
}
res.finish();
process.alarm.clear();
process.removeListener(finisher);
};

http.createServer(handler).listen(8000);
sys.puts("Server running on port 8000");
==

Any thoughts or suggestions on how to handle this sort of situation
gracefully?

Thanks,
David

[Malte Ubl]

I think that this is a really interesting question, but the answer
should be: There is no way to interrupt continuous execution flow of
JavaScript, ever. This is the classic example of cooperative
multitasking where a process ceases to be cooperative. In
V8-JavaScript-flavor your only way might be to call "shouldIYield()"
in the infinite loop which would throw an exception after a certain
timeout expired (On spidermonkey you could use coroutines).

Cheers
Malte

> --
>
> You received this message because you are subscribed to the Google Groups "nodejs" group.
> To post to this group, send email to nod...@googlegroups.com.
> To unsubscribe from this group, send email to nodejs+un...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/nodejs?hl=en.
>
>
>



--
http://twitter.com/cramforce
http://nonblocking.io

[ry]

On Dec 4, 11:15 pm, David <david.sk...@gmail.com> wrote:
> I am trying to figure out a way to recover (as gracefully as possible)
> from HTTP requests handled by node as an HTTP server from taking too
> long.
>
> That is, if the callback provided to http.createServer() doesn't
> complete in some amount of time, I'd like to be able to take some
> action rather than just let things spin.
>
> process.Timer() doesn't seem to help since the libev event loop is not
> engaged if V8 is being greedy and not giving up any control. E.g. in
> the worst-case scenario of:

Right, if you enter an infinite loop there is no recovering. If you're
performing I/O in the request event handler then a timer will be
sufficient to do a timeout. However in the case that you're doing a
lot of processing, that won't work. I suggest moving the heavy
calculations into a different process and send work to it via IPC. In
that situation you can still do timeouts.

I probably won't be providing a way to interrupt long running
calculations; it would be a lot of machinery and, I think in most
cases, the more robust design is to use multiple processes.

[weepy]

Is it possible to IPC into Node ?

[Connor Dunn]

2009/12/5 ry <coldre...@gmail.com>:

I'm not sure how good an idea this is, but I think promise.wait()
allows this functionality already. If we define a sleep function as:
sleep = function (time) {
var promise = new process.Promise();
setTimeout(function () { promise.emitSuccess(true) }, time);
promise.wait();
}
You can use sleep(0) to allow anything waiting in the event queue to
run. For example

while (true) {
...
sleep(0);
}

You could then do a timeout using
timeout = true;
setTimeout(function() { timeout = false }, 1000);
while(timeout) {
...
sleep(0);
}

[Ryan Dahl]

On Sat, Dec 5, 2009 at 9:41 PM, weepy <jona...@gmail.com> wrote:
> Is it possible to IPC into Node ?

Through the stdio of the child process. I'll be adding unix sockets
and pipes later.

[David]

Thanks for the thoughts and info. I am proceeding down the use-child-
processes path (via process.createChildProcess() and then
communication over stdio). If/when I have some useful data, I'll post
some benchmarking numbers on the overhead of bouncing things through
the parent process.

[Erik Corry]

2009/12/5 Malte Ubl <malt...@gmail.com>:

> I think that this is a really interesting question, but the answer
> should be: There is no way to interrupt continuous execution flow of
> JavaScript, ever. This is the classic example of cooperative

V8 actually has preemption so you can stop JS execution. However it's
not clear what to do at that point. You can start a new thread and
run JS there, but that fits reeealy badly with the way node works. Or
you could throw an uncatchable exception, I think. Mads Ager did some
work recently to enable that. The uncatchable exception is only
uncatchable in JS. In the C++ level you can and must catch it using
the methods in the V8 API.

[Malte Ubl]

Hey,

did you see my recent post regarding the Worker API. That might be a
good starting point.

Cheers
Malte

[David]

On Dec 8, 3:45 pm, Malte Ubl <malte....@gmail.com> wrote:

> On Mon, Dec 7, 2009 at 4:47 PM, David <david.sk...@gmail.com> wrote:
> > Thanks for the thoughts and info. I am proceeding down the use-child-
> > processes path (via process.createChildProcess() and then
> > communication over stdio). If/when I have some useful data, I'll post
> > some benchmarking numbers on the overhead of bouncing things through
> > the parent process.
>
> Hey,
>
> did you see my recent post regarding the Worker API. That might be a
> good starting point.
>
> Cheers
> Malte

Hi, Malte. Indeed I did (and it is a good starting point.)

Here are some initial and not excruciatingly scientific benchmarks:

Starting with http://github.com/cramforce/node/blob/662e03b3861942cf0b8a890c7d9775b2ae6917ca/lib/worker.js,
I:
- commented out the calls to sys.error()
- added a few semicolons to make Emacs js2-mode happy
- used the following for my child.js:

===
//child.js
var worker = require('./worker').worker;

worker.onmessage = function (msg) {
worker.postMessage({
call: msg.call,
data: "Hello, World"
});
};
===

- Used server.js as posted at http://pastebin.com/f7aa16f50 (except I
commented out the sys.puts() call inside the handler function passed
to createServer().

With this ultra-simple "Hello World" child, and 20 Worker processes,
"ab -n 500 -c 20 'http://127.0.0.1:8000/'" (20 concurrent requests,
500 requests total) gives me about 195 requests/second, or a mean
request time of 102.670 millis.

This is on a Mac Pro with 2 dual-core 2.66GHz Xeon's running OS X
10.5.8 / Kernel 9.8.0.

In comparison, the following hello-world.js server:

===

var http = require('http');
var sys = require('sys');

http.createServer(function(req,res) {
res.sendHeader(200, {'Content-Type': 'text/plain'});
res.sendBody("Hello, World");
res.finish();
}).listen(8001);

sys.puts('Server running at http://127.0.0.1:8001/');
===

does about 5416 requests/second with a mean request time of 3.692
millis. (using the same ApacheBench command.)


I got suspicious that the difference was about 100ms, the same as the
setTimeout() call in the WorkerCild to send the extra message
splitter.

So then I modified worker.js to use a 10ms timeout in postMessage for
WorkerChild and WorkerProcess and the ab numbers go to 1653 req/sec
with a mean of 12.093 millis per request.

Changing that timeout to 5ms brings it to 2195.51 req/sec and mean
9.110 millis per request. So, solving that flushing issue will
definitely help perf and bring down the overhead of using the Web
Worker API infrastructure.


David

[Malte Ubl]

Hey,

Thanks for the testing. I asked in another thread whether there is a
clean solution for the flushing problem.
Which system are you using. On OSX reducing the timeout does not yield
reliable flushing so I kept it high for the prototype.

Overall:
The single node solution should always be an order of magnitude faster
until you actually need the extra CPUs on your computer.

[David]

On Dec 10, 12:04 pm, Malte Ubl <malte....@gmail.com> wrote:
>
> Thanks for the testing. I asked in another thread whether there is a
> clean solution for the flushing problem.
> Which system are you using. On OSX reducing the timeout does not yield
> reliable flushing so I kept it high for the prototype.
>
> Overall:
> The single node solution should always be an order of magnitude faster
> until you actually need the extra CPUs on your computer.

I am using OS X; granted, the flushing reliability is critical but the
numbers I got after reducing the timeout indicate that the penalty for
using the webworker API is way less than an order of magnitude, and
this is on an artificially cheap response (just "Hello World"). On a
"real" response that does something, the overhead would be an even
smaller proportion.