Introducing trycatch: Async try catch / scoped error handler / handle http.serverRequest 500s

[Adam Crabtree]

Wish you could wrap an asynchronous call stack in just one try-catch and have it just work?

Wish you could get the error associated with a given long stack trace?

Wish you could send proper 500s and not have an uncaught exception take out all other concurrent requests?

Now you can. Introducing trycatch():

trycatch() is an asynchronous try catch / scoped error handler:

trycatch(function() {

  // try code

}, function(err) {

  // catch code

});

Or, for the http.serverResponse 500 case:

var http = require('http');

var trycatch = require('trycatch');

http.createServer(function(req, res) {

trycatch(function() {

setTimeout(function() {

throw new Error('Baloney!');

}, 1000);

}, function(err) {

res.writeHead(500);

res.end(err.stack);

});

}).listen(8000);

https://github.com/crabdude/trycatch

Thanks to Tim Caswell and Tom Robinson for indirectly contributing the majority of the code to make this work.

Cheers,

Adam Crabtree

--
Better a little with righteousness
       than much gain with injustice.
Proverbs 16:8

[Mark Hahn]

That looks awesome.  I'm definitely gonna give it a test drive.

Is there any way to get a feel for how it does what it does without reading the code?  I'm really curious.

P.S. It is nice to see something help async problems that isn't a sync-simulation framework. :-)

[Marco Rogers]

I remember seeing Tim's post about this on Twitter. It's actually not so complicated. It just takes all the low level places where async calls happen and wraps them so they can be tracked.

https://github.com/CrabDude/trycatch/blob/master/lib/hook.js

:Marco

[Mark Hahn]

Aha.  So only low-level node async ops create new stacks (and fibers) so there isn't too much to wrap.  Quite cool.  This seems like something that should be in core node.

--
Job Board: http://jobs.nodejs.org/
Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nod...@googlegroups.com
To unsubscribe from this group, send email to
nodejs+un...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en

[Dominic Tarr]

basically, it hot patches event emitter, setTimeout, setInterval, and nextTick. so that the functions passed to them get wrapped in functions that have try/catch around them.

[Martín Ciparelli]

Smart and useful work. Been tired of seeing sync frameworks. This is the way if you're gonna code a JS library. Give the guy the credits he deserves for sth that is definitely going to be used by many developers instead of discussing how complicated is to code something like that, meaning that it actually might be sort of difficult if noone else provided such a library since node released.

[Adam Crabtree]

Glad you like it.

When I wrote my control flow library, Sexy, http://github.com/Crabdude/sexy, one of the features I was surprised not to find in other libraries (at least I didn't see any at the time) was error coalescing in the form of one error handler per series (whatever you want to call it). In production, I found myself writing ALOT of redundant dode in the form of:

doSomethingAsync(function(err) {

  if (err) return myErrorHandler(err);

  try {

    // do something

  } catch (e) {

    myErrorHandler(err);

  }

});

Given enough async calls, this can lead to an increase in code bloat on the order of 25% and certainly violates DRY. Control flow libraries make this a little better by wrapping each new callback in a try catch, but I still found myself tediously writing "if (err) return onError(err);" over and OVER again. Having to write 2 lines of code for every 1 line of asynchronous code (100% increase in code bloat per async call) was getting old fast. While I like Sexy, there're some gaps in the API, so I don't use it in production and usually use Step instead for its simplicity, but Step doesn't have Sexy's "sexy" error coalescing. So I sought to add it. Since Tim and I work on the same team and across from each other, I told him about what I was doing that day, and instead he suggested going down the route Tom Robinson had done with long-stack-traces by wrapping node built-ins modules. Tim adapted Tom's long-stack-trace technique to build hook.js and a prototype of the concept, I took the two and finished it up, and trycatch was born. =)

Basically, what long-stack-traces does (though I'd encourage you to read the code) is shim all the built-in module functions that create a new event-source. When you call setTimeout, you're calling a shim which then creates a callback shim and ties the event-source to the hidden callback shim so the long stack trace can be generated. By combining that with the flow-control concept of auto-wrapping with try catch and error coalescing, we're able to now deal with asynchronous uncaught exceptions in a MUCH MORE SANE manner than try-catches EVERYWHERE or process.onUncaughtException. Also, you can't wrap 3rd party code in a try catch, but you can wrap it in a trycatch. =)

In production, the concept of a single request being able to crash the server was mind-boggling to some new to node and generated some not unreasonable push-back. Adding process.onUncaughtException conflated the issue by suppressing some errors we wanted to crash the server, and we found ourselves unable to deal with request-based errors.

At any rate, I'd encourage you to take a look at the internals, as it's a very fun hack to make this work. Basically it uses the V8 JavaScriptStackTraceAPI's Error.prepareStackTrace(error, structuredStackTrace) method to create a custom stack trace and tie callbacks to their event-source. When error.stack is accessed, instead of returning a string, we return an object that returns a long stack trace string and the event-source object so we can traverse up the event-source chain till we find the root trycatch callback.

http://code.google.com/p/v8/wiki/JavaScriptStackTraceApi

I'll write a post up on all this when I get the chance, but I definitely think something along these lines belong in core. I've been advocating for a long time the need to be able to detect if a given function is an asynchronous event-source for situations like this, and even more so for better control-flow libraries. Unfortunately, when I asked the panel at NodeConf if they'd ever considered it, I lacked to the proper vocabulary and as a result they didn't quite understand my question and it fell flat. At any rate, THIS (trycatch) does not need to be in node core so much as the ability to detect new event-sources. See the following for reference on the issue:

http://nodejs.org/illuminati0.pdf

Manually wrapping built-ins is brittle and guaranteed to miss something if not now, eventually. 

Cheers,

Adam Crabtree

[Mark Hahn]

How does trycatch help an fsWrite callback(err, data)?  Don't I still have to do "if(err) {lots of junk}" on each callback?  Those are the ones that bloat my code.

I guess writing "if(err) throw err;" is less bloat.

[Adam Crabtree]

I don't follow. Do you mean fs.readFile since you have (err, data)? That's actually where this idea originated from. Ideally, we'd be able to remove all errors from callbacks that are being scoped. So it'd look something like this:

trycatch(function() {

  fs.readFile(__filename, function(data) {

    fs.writeFile(__dirname + '/copy', data, function() {

      console.log('Success!');

    });

  });

}, function(err) {

  console.log(err.stack);

});

but that would change the function signature, possibly causing issues. We could detect that, but I don't like static analysis since it's brittle. Also, the low-level shim would only catch low-level callback errors. We could shim callbacks for higher-level node built-in async functions, but 3rd party code might still generate errors which wouldn't properly coalesce. We could also try to do that dynamically, but again where do you draw the line beyond handling every function call and checking if the first argument is an error? For developer purposes, there's no important distinction between higher-level (fs.readFile?) node built-in async functions and 3rd party async functions, but from a library perspective all the high-level async functions are known and can be manually shim'd, but 3rd party code, not so much.

Ultimately, we need some hook or convention to make this possible, something like:

// in ThirdParty.js

exports._async = [exports.readFile, exports.writeFile, ... ];

so that flow control libraries can properly wrap them. Otherwise, nothing comes to mind regarding how to make this functionality work robustly (3rd party). We could make it work for node built-ins though. 

Thoughts?

Cheers,

Adam Crabtree

On Thu, Sep 29, 2011 at 11:19 AM, Mark Hahn <ma...@boutiquing.com> wrote:

How does trycatch help an fsWrite callback(err, data)?  Don't I still have to do "if(err) {lots of junk}" on each callback?  Those are the ones that bloat my code.

I guess writing "if(err) throw err;" is less bloat.

--
Job Board: http://jobs.nodejs.org/
Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nod...@googlegroups.com
To unsubscribe from this group, send email to
nodejs+un...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en

--

[Bruno Jouhier]

Yes, problem is that async is contagious and you now have a mix of
functions that propagate errors via callback(err) and functions that
have been wrapped with your shim and that propagate errors via your
new trycatch mechanism. It's going to be really hard to make the two
coexist and cooperate smoothly, especially with third-party libraries
that rely on the standard mechanism and thus will miss the errors
propagated by your new mechanism. Looks like you've open Pandora's
box!

This is where the preprocessor approach comes to the rescue. You can
solve the problem with a try/catch transformation pattern that is
fully compatible with node's callback(err, result) standard callback
signature. Of course, you must be ready to introduce a preprocessor in
your tool chain, or go with fibers. Unfortunately, there is no free
lunch!

Bruno

[Tim Smart]

Personally a pre-processor or fibers isn't ideal.

Just define a generic error handler instead of using try catch - it only adds 1 line per callback.

    function onError (error) { ... }

    fs.readFile('myfile.txt', function (error, data) {

      if (error) return onError(error)

      fs.writeFile('filecopy.txt', fileWritten)

    }

    function fileWritten (error) {

      if (error) return onError(error)

    }

Tim.

[Floby]

Nice work Adam. This is actually the first "control-flow" library that
I'm looking forward to play with.

[Adam Crabtree]

Thanks Floby.

@Tim

Yes, it's not horrible, but when you find yourself writing lots of redundant code or heavy I/O, say for example when implementing a VFS API (actual use case) that interfaces closely with a 3rd party API like ftp-js, you find almost the majority of your code being boilerplate, and it adds a lot of noise. Javascript as a language being so flexible, usually means the problem can be solved one way or the other, which has always lead me to assume I can hack or fix any problem I come across, maybe that's why it's so frustrating to be unable to easily address this. =)

Honestly though, the issue seems to be feature parity. If implementing a stable vanilla server in node requires a lot of manual legwork that you get for free in other stacks, it's going to seem a little silly. Obviously there are huge payoffs to be gained from asynchronous I/O, but having to re-wrap EVERY new event-source just b/c try catch is synchronous and there's no asynchronous equivalent, just sounds a little broken to me.

Cheers,

Adam Crabtree

--
Job Board: http://jobs.nodejs.org/
Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nod...@googlegroups.com
To unsubscribe from this group, send email to
nodejs+un...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en