Callback hell and how to get around it?

[Ingwie Phoenix]

Hai, everyone.

So I just figured out that Connect’s static file servers dont work well with symlinks…so I had tow rite myself a little workaround, a fallback router, per-se. But whilst I did so, I had a nice meeting with the callback hell. Here’s my code:

CDN.use(config.CDN.baseUrl, function(req, res, next){
if(err) { next(); throw err; }
fs.lstat(config.base+"/cdn"+req.url, function(err,stats){
if(stats.isSymbolicLink()) {
fs.readlink(config.base+"/cdn"+req.url, function(err, linkstr){
if(err) throw err;
log.info("Redirecting request \""+req.url+"\" to \""+linkstr);
fs.readFile(linkstr, function(err, data){
if(err) throw err;
res.writeHead(200,{"Content-type":"text/html"});
res.end(data);
});
});
}
});
});
});

Okay…thats lots. And how does one get around this and into a bit of nicer code? It almost looks like a little pyramid there…

Kind regards, Ingwie.

[// ravi]

On Sep 15, 2014, at 2:57 AM, Ingwie Phoenix <ingwi...@googlemail.com> wrote:

Hai, everyone.

So I just figured out that Connect's static file servers dont work well with symlinks...so I had tow rite myself a little workaround, a fallback router, per-se. But whilst I did so, I had a nice meeting with the callback hell. Here's my code:

<…>

Okay...thats lots. And how does one get around this and into a bit of nicer code? It almost looks like a little pyramid there…

Have you taken a look at Promises? Bluebird is widely considered the fastest implementation. I found Q easier to understand and a bit better documented.

https://github.com/kriskowal/q

https://github.com/petkaantonov/bluebird

—ravi

[// ravi]

On Sep 15, 2014, at 12:50 PM, // ravi <ravi-...@g8o.net> wrote:

Have you taken a look at Promises? Bluebird is widely considered the fastest implementation. I found Q easier to understand and a bit better documented.

https://github.com/kriskowal/q

https://github.com/petkaantonov/bluebird

And if Promises are too heavyweight, try FlowJS: https://github.com/willconant/flow-js.

—ravi

[Alex Kocharin]

 

Promises just wrap callbacks. So you had callback hell, now you have wrapped callback hell. Nice change, huh. Only thing promises are good for is handling errors, but I don't know if it's worth it.

 

If you want to turn async code to sync-like code, I would suggest to use generators and `co` module instead.

 

 

15.09.2014, 20:51, "// ravi" <ravi-...@g8o.net>:

--
Job board: http://jobs.nodejs.org/
New group rules: https://gist.github.com/othiym23/9886289#file-moderation-policy-md
Old group rules: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
---
You received this message because you are subscribed to the Google Groups "nodejs" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nodejs+un...@googlegroups.com.
To post to this group, send email to nod...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/nodejs/F7E5827B-8287-4C6B-9975-DD0C8CA1059E%40g8o.net.
For more options, visit https://groups.google.com/d/optout.

[Alex Kocharin]

If you don't like nested callbacks, you can always flatten it with named functions:

```js
CDN.use(config.CDN.baseUrl, cdn_next)

function cdn_next(req, res, next){
if(err) { next(); throw err; }
fs.lstat(config.base+"/cdn"+req.url, fs_lstat_next)
}

function fs_lstat_next(err,stats){
if(stats.isSymbolicLink()) {
fs.readlink(config.base+"/cdn"+req.url, fs_readlink_next)
}
}

function fs_readlink_next(err, linkstr){
// and so on
}
```


15.09.2014, 20:27, "Ingwie Phoenix" <ingwi...@googlemail.com>:
> Hai, everyone.
>
> So I just figured out that Connect's static file servers dont work well with symlinks...so I had tow rite myself a little workaround, a fallback router, per-se. But whilst I did so, I had a nice meeting with the callback hell. Here's my code:

>
>             CDN.use(config.CDN.baseUrl, function(req, res, next){
>                 if(err) { next(); throw err; }
>                 fs.lstat(config.base+"/cdn"+req.url, function(err,stats){
>                     if(stats.isSymbolicLink()) {
>                         fs.readlink(config.base+"/cdn"+req.url, function(err, linkstr){
>                             if(err) throw err;
>                             log.info("Redirecting request \""+req.url+"\" to \""+linkstr);
>                             fs.readFile(linkstr, function(err, data){
>                                 if(err) throw err;
>                                 res.writeHead(200,{"Content-type":"text/html"});
>                                 res.end(data);
>                             });
>                         });
>                     }
>                 });
>             });
>         });
>

> Okay...thats lots. And how does one get around this and into a bit of nicer code? It almost looks like a little pyramid there...
>
> Kind regards, Ingwie.

>
> --
> Job board: http://jobs.nodejs.org/
> New group rules: https://gist.github.com/othiym23/9886289#file-moderation-policy-md
> Old group rules: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
> ---
> You received this message because you are subscribed to the Google Groups "nodejs" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to nodejs+un...@googlegroups.com.
> To post to this group, send email to nod...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/nodejs/AA9F0C8A-93DF-4797-9270-941CFDA759CE%40googlemail.com.

[// ravi]

On Sep 15, 2014, at 1:01 PM, Alex Kocharin <al...@kocharin.ru> wrote:

 

Promises just wrap callbacks. So you had callback hell, now you have wrapped callback hell. Nice change, huh.

Actually yes, I think it is. Readable code is worth it, in my view.

If you want to turn async code to sync-like code, I would suggest to use generators and `co` module instead.

I am waiting for the right time to start recommending generators+X to folks asking these sort of questions. I am not comfortable yet doing so, but I agree that anyone interested should take a look.

—ravi

[Aria Stewart]

On Monday, September 15, 2014 9:28:28 AM UTC-7, Ingwie Phoenix wrote:

Hai, everyone.

So I just figured out that Connect’s static file servers dont work well with symlinks…so I had tow rite myself a little workaround, a fallback router, per-se. But whilst I did so, I had a nice meeting with the callback hell. Here’s my code:

            CDN.use(config.CDN.baseUrl, function(req, res, next){
                if(err) { next(); throw err; }
                fs.lstat(config.base+"/cdn"+req.url, function(err,stats){
                    if(stats.isSymbolicLink()) {
                        fs.readlink(config.base+"/cdn"+req.url, function(err, linkstr){
                            if(err) throw err;
                            log.info("Redirecting request \""+req.url+"\" to \""+linkstr);
                            fs.readFile(linkstr, function(err, data){
                                if(err) throw err;
                                res.writeHead(200,{"Content-type":"text/html"});
                                res.end(data);
                            });
                        });
                    }
                });
            });
        });

Best way to not write callback hell: STOP WRITING IT.

CDN.use(config.CDN.baseUrl, require('cdn-middleware'));

cdn-middleware:

module.exports = function(req, res, next){

    var path = config.base + "/cdn" + req.url;

    fs.lstat(path, function(err,stats){ 

        if(stats.isSymbolicLink()) {

            serveLink();

        } else {

            return next();

        }

    });

    function serveLink() {

        fs.readlink(path, handleError(function(linkstr){

            log.info('Redirecting request "' + req.url + '" to "' + linkstr + '"'); 

            // This could probably be:

            //   res.statusCode(200); 

            //   fs.createReadStream(linkstr).pipe(res);

            fs.readFile(linkstr, handleError(data) {

                res.writeHead(200,{"Content-type":"text/html"}); 
                res.end(data); 
            }); 
        });

    }

    

    function handleError(fn) {

        return function (err, data) {

            if (err) {

                return next(err);

            } else {

                return fn(data);

            }

        }

    }
}); 

 

Okay…thats lots. And how does one get around this and into a bit of nicer code? It almost looks like a little pyramid there…

Name your functions. Move them as far left as you can (depending on how many variables they share with the context).

Now that said, not playing nice with symlinks sounds like a bug: have you opened an issue?

Aria

[Aria Stewart]

On Sep 15, 2014, at 10:09 AM, // ravi <ravi-...@g8o.net> wrote:

On Sep 15, 2014, at 1:01 PM, Alex Kocharin <al...@kocharin.ru> wrote:

 

Promises just wrap callbacks. So you had callback hell, now you have wrapped callback hell. Nice change, huh.

Actually yes, I think it is. Readable code is worth it, in my view.

What you really get with promises: Abstraction over exact order. You're no longer describing what to do next after a task, but instead, describing what depends on what inputs. For simple problems, the result is the same and promises are just expensive, complicated callbacks. For a complex, one, however, it lets you write functions that stay named in the problem domain rather than solution domain more easily.

Promise.all returns a promise that gives all the values of the promises you pass in. And remember a value without .then can be substituted for a promise -- so you now have an interface that can interpret "later values" and "already available" values identically. That's where the real power comes in: Promises that join and collect results of promises, functions that accept promises (or plain values) and give a promise result.

Aria

[Alex Kocharin]

 

 

15.09.2014, 21:13, "// ravi" <ravi-...@g8o.net>:

On Sep 15, 2014, at 1:01 PM, Alex Kocharin <al...@kocharin.ru> wrote:

 

Promises just wrap callbacks. So you had callback hell, now you have wrapped callback hell. Nice change, huh.

 

 

Actually yes, I think it is. Readable code is worth it, in my view.

 

 

Promises are really useful for a couple of things like error handling, but readability isn't one of them.

 

Quite the opposite, that ".then(...)" noise is even worse than callbacks. Try to rewrite that example with promises, I'm sure it won't be more readable.

 

I think generators are the only clean answer here. I know they aren't even in stable node, but still, they're worth recommending. Well there are fibers also, but they seem to be forgotten already.

 

 

 

 

If you want to turn async code to sync-like code, I would suggest to use generators and `co` module instead.

 

 

I am waiting for the right time to start recommending generators+X to folks asking these sort of questions. I am not comfortable yet doing so, but I agree that anyone interested should take a look.

 

—ravi

 

 

 

--

Job board: http://jobs.nodejs.org/
New group rules: https://gist.github.com/othiym23/9886289#file-moderation-policy-md
Old group rules: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
---
You received this message because you are subscribed to the Google Groups "nodejs" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nodejs+un...@googlegroups.com.
To post to this group, send email to nod...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/nodejs/6BCCC31B-7FCD-4B20-A045-7FFAC897A53A%40g8o.net.

[Bruno Jouhier]

Plenty of tools have been developed to solve this problem. See http://spion.github.io/posts/analysis-generators-and-other-async-patterns-node.html for a comparison.

Here is how I write your example (handling the else clause of your if test BTW):

CDN.use(config.CDN.baseUrl, function(req, res, next, _) {

    var path = config.base + "/cdn" + req.url;

    if (fs.lstat(path, _).isSymbolicLink()) path = fs.readlink(path, _); 

    res.writeHead(200, { "Content-type" : "text/html" }); 

    res.end( fs.readFile(path, _));

});

Bruno

[Buschini Edouard]

Hi,

I'm more onto events with EventEmitter() when I need to do a lot of async/sync stuff.
It's built in nodejs, and easy to use out of the box, npm is not required.

Example with sync stuff :

function (data, callback) {
  var Events = require("events");
  var event = new Events.EventEmitter();

  event.on("job1", function (data) {
    // do stuff with data
    event.emit("job2", err, data);
  });

  event.on("job2", function (err, data) {
    // do stuff with err and data
    event.emit("done", err, data);
  });

  event.on("done", function (err, data) {
    // do stuff with err and data
    // delete event and remove listeners
    callback(err, data);
  });
 
  event.emit("job1", data);
}

You get the idea.
For async just call emit after a function is done and keep an array updated so the "done" handle know when all the functions are executed.

Regards.

Edouard Buschini

To view this discussion on the web visit https://groups.google.com/d/msgid/nodejs/500601410802189%40web21h.yandex.ru.

[// ravi]

On Sep 15, 2014, at 1:26 PM, Aria Stewart <ared...@nbtsc.org> wrote:

On Sep 15, 2014, at 10:09 AM, // ravi <ravi-...@g8o.net> wrote:

On Sep 15, 2014, at 1:01 PM, Alex Kocharin <al...@kocharin.ru> wrote:

 

Promises just wrap callbacks. So you had callback hell, now you have wrapped callback hell. Nice change, huh.

Actually yes, I think it is. Readable code is worth it, in my view.

<snip happens>

Promise.all returns a promise that gives all the values of the promises you pass in. And remember a value without .then can be substituted for a promise -- so you now have an interface that can interpret "later values" and "already available" values identically. That's where the real power comes in: Promises that join and collect results of promises, functions that accept promises (or plain values) and give a promise result.

I am not unaware of these advantages, but the OP was speaking specifically of callback hell with an example that indicated the issue is the familiar one of readable (and manageable) code flow.

—ravi

[Alexey Petrushin]

I wrote short article http://jslang.info/blog/2014/escape-from-callback-hell

about simple approach that turn this

    var printCommentsForUser = function(login, password, cb){

      authenticateUser(login, password, function(err, user){

        if(err) return cb(err)

        getComments(user.id, function(err, comments){

          if(err) return cb(err)

          renderComments(user, comments, cb)

        })

      })

    }

    

Into this

    var printCommentsForUserWithFork = function(login, password, ecb, cb){

      authenticateUser(login, password, ecb, function(user){    

        getComments(user.id, ecb, function(comments){

          renderComments(user, comments, ecb, cb)

        })

      })

    }

    

Or this

    var printCommentsForUserWithTwoCallbacks = function(login, password, ecb, cb){

      authenticateUser(login, password, ecb, function(user){    

        getComments(user.id, ecb, function(comments){

          renderComments(user, comments, ecb, cb)

        })

      })

[greelgorke]

promises, generators, events aside (wonder, nobody mentioned async module) the so called callback-hell appears when your functions share some kind of state, in this case via the closure scope. the obvious way out of it is to reduce shared state, flatten as suggested already, and push the needed state far out of your code. you can use partial application on your callbacks, if they need operate on some implicit params (closure-scoped variables are just that: implicit parameters) or remain stateless by taking all params explicitly.

[q2dg2b]

What about Async?
https://github.com/caolan/async

[Alex Kocharin]

 

I think you just reinvented node.js core function named `domain.intercept`. :)

 

 

16.09.2014, 01:34, "Alexey Petrushin" <alexey.p...@gmail.com>:

--

Job board: http://jobs.nodejs.org/
New group rules: https://gist.github.com/othiym23/9886289#file-moderation-policy-md
Old group rules: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
---
You received this message because you are subscribed to the Google Groups "nodejs" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nodejs+un...@googlegroups.com.
To post to this group, send email to nod...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/nodejs/32603590-e433-43a9-860a-8c12fdbac3f9%40googlegroups.com.

[Tim Smart]

Instead of nesting functions within function, use a state object and named
functions:

function printCommentsForUser(login, password, done) {
var s = {};

authenticateUser(login, password, authenticated);

function authenticated(err, user) {
if (err) { return done(err); }
s.user = user;
getComments(s.user.id, gotComments);
}

function gotComments(err, comments) {
if (err) { return done(err); }
renderComments(s.user, comments, done);
s = null;
}
}

Reads much nicer even if it is more lines of code.

Tim

[Floby]

Hello.

This is a problem newcomers face so often that there is a website for it http://callbackhell.com/

This website show that "callback hell" is really just about how you structure your code.
Any construction that increase your cyclomatic complexity is bad (callbacks, promises, generators and whatnot included).

The most effective method I found for handling this is reading callbackhell.com and Clean Code [1]

[1] http://www.amazon.com/Clean-Code-Handbook-Software-Craftsmanship/dp/0132350882

[Tom Boutell]

The async module is a simple and widely used solution that doesn't require a big shift in your thinking like promises. 

https://www.npmjs.org/package/async

Check out the async.series method, which does what you want, allowing you to unroll consecutive steps into consecutive functions, much like traditional synchronous code.

async.eachSeries is also highly useful.

  

[Seth Pollack]

There are many choices out there. Here is a writeup i did a while back on why we choose Streamline as the solution to callback hell (and related issues) for our product: http://blog.rivaliq.com/develop-double-time-node-plus-streamline/

Seth

[Ken]

+1 for this approach with the note that theres probably one or more EventEmitters already in scope that you could leverage.  Creating new emitters and or registering event handlers within callbacks or other event handlers can lead to unexpected leaks.   Worth remembering that the global "process" is an EventEmitter, so for simple programs it's perfectly acceptable to write

process.on("whatever", function (error, data) { 

    ... 

});

I wouldn't do this from within a module since it can cause non-obvious behavior, but in the main program file it's fine.  In a web server context you generally have access an http.Server which is EventEmitter as well, as well as request and response objects.

[Alexey Petrushin]

> I think you just reinvented node.js core function named `domain.intercept`. :)

Domains are still experimental (and doesn't have support in browser). Also, I believe that if you decide to use the dark side of the Force - it's better to use it at the full potential with the Fibers.

> Reads much nicer even if it is more lines of code. 

Nicer? :) Don't know, for me the nice version is the synchronous one (with generators, code generators, yields, fibers etc.), but I believe the version with fork is still shorter and simpler.

[Tom Boutell]

Is the original poster's code really made up of little pieces each of
which someone would be excited to have as a reusable module? Is dicing
it up into more files really going to help? I don't think so in this
case. It's very task-specific.

Which is why, right on the callbackhell.com website, the author
advocates using the async module for complex cases. (:

> --
> Job board: http://jobs.nodejs.org/
> New group rules:
> https://gist.github.com/othiym23/9886289#file-moderation-policy-md
> Old group rules:
> https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
> ---

> You received this message because you are subscribed to a topic in the
> Google Groups "nodejs" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/nodejs/wBAJhYOzjqQ/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to

> nodejs+un...@googlegroups.com.
> To post to this group, send email to nod...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/nodejs/6af70f61-43a5-4bb7-8f77-f0c163f48dcf%40googlegroups.com.

>
> For more options, visit https://groups.google.com/d/optout.

--


THOMAS BOUTELL, DEV & OPS
P'UNK AVENUE | (215) 755-1330 | punkave.com

[Ahmed Eldosoukey]

+1 for the event emitter solution, really great

[Axel Kittenberger]

As you can see this has always been and is still a controversial issue.

The gospel is, callbacks are fine and you are wrong. Somethings are still going to be fixed with domains etc.

I think the misunderstandings is due to the fact on the level of complication of stuff being handled. Some people just push some streams through and need high effectiveness others do stuff that is already complicated enough by itself.

My first larger project I did with node was scientific analysis of data collected in a mongodb database -- controlled by a user via a web interface. Thus I personally never bought into that gospel. Things were complicated enough without having to wrap my head around callback issues, and being able to request several items at the same time from database could speed up stuff, it didn't really matter that much. I changed the project on the fly to Bruno's streamline which rescued it (otherwise I'd had to redo it with something completely different, it wasn't manageable anymore)

I also know of friends whom I tried to convince how awesome node is of which I know that turned it down due to callbacks.

Node has recently lost one of its main contributors due to callback issues. And no, promises and flow control libraries didn't cut it. His main point I understood was, bugs caused by misbehaving libraries that fail to call a callback or even worse, call it twice, are extremely hard to debug.

Right now I'm using suspend,

https://github.com/jmar777/suspend

which takes advantage of harmony generators.

I like it, since it is the most reliable on getting backtraces in case of an error. Bruno's stuff should generate backtraces too, but I had issues in case of rethrowing catched errors. On the other hand, suspend and generators are little more try on when to put a * and when not, and result into hangups if you do it wrongly, while Brunos preprocessor proofed to be very reliable in pointing out a variety of coding errors.

My advice is, if you do a library that is to be used by others, you ought to use simple callbacks, as this is still the lingua franca in node world.

What you do inside is up to you and if you do an application or anything that is not a library to be used by others, there is no definitive answer, do not listen to the gospel, try things out, look what works best for you.

[Tom Boutell]

The async module is used pretty heavily; I would suggest that if you
understand plain old callbacks, you won't have any trouble reading
code written with the async module. It's just a way to do the common
async patterns more consistently and not get lost.

Promises is a much bigger cognitive shift than the async module.

More importantly though, if your module successfully implements its
advertised API, it doesn't matter what you use internally to implement
it.

The API you advertise to the rest of the world is a more important
question. Generally you should be exporting methods like:

doSomething(arg1, arg2, ..., callback)

Where the callback expects to receive:

err, ...

And if err is falsey, nothing is wrong and the other arguments (if
any) can be safely examined.

That much at least is extremely consistent in node. The only common
exception is streams, which are a viable option if you'll be inputting
or outputting a lot of data gradually over time.

> --
> Job board: http://jobs.nodejs.org/
> New group rules:
> https://gist.github.com/othiym23/9886289#file-moderation-policy-md
> Old group rules:
> https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "nodejs" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/nodejs/wBAJhYOzjqQ/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> nodejs+un...@googlegroups.com.
> To post to this group, send email to nod...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/nodejs/CABg07fvC_5vKW3vNdJ86FW%2BBu4AXX%3DxzJ%2B%2BmaQOVO%2BRa2NAW5w%40mail.gmail.com.

[Tom Boutell]

(I don't disagree that "use callbacks to do five simple things
consecutively" is a lot to ask and can be offputting and bug-prone. It
absolutely is. I use sublimetext snippets to stay out of trouble,
which means I'm effectively coding in a higher-level language with a
really, really crude compiler (: )

[Alexey Petrushin]

The problem with sublime snippets is that most of the time you reading code or thinking and sadly there's no sublime snippets that helps you effectively read callback-heavy code :)

[Alexey Petrushin]

> if you understand plain old callbacks, you won't have any trouble reading 

> code written with the async module.

Ha-ha, yea, "you don't like callbacks because you don't understand how to work with it!". 

You may want callbacks in network layer, but definitely not in the business logic.

The funny thing is - in most web project 90% of code base is business logic when you don't need callbacks at all :)

What even more funny - it seems that node.js doesn't give you significant speedup compared to Ruby on Rails in classical Web App (classical request/response apps, without realtime or 10k connections).  So, you get extra complexities with callbacks in return for nothing (a very little performance boost).

P.S. I'm not against the node, I'm against accepting bad thing as if it's a good thing. Callback is a bad thing (for most of the projects, you don't write network optimised stuff everyday, usually you work more or less with plain old classical webapps), sadly, there's no better ways currently, and we have to use it, but we shouldn't fool ourselves into believing that it's a good thing.

On Friday, 19 September 2014 00:22:46 UTC+4, Tom Boutell wrote:

[Tom Boutell]

I think the async module results in readable code. It's just too verbose to type. (:

To view this discussion on the web visit https://groups.google.com/d/msgid/nodejs/dd788333-b998-4e32-b047-b67ea4add34b%40googlegroups.com.

[Peter Petrov]

After tasting the magic of ES6 generators (combined with promises or thunks), I never want to look again at the async module (or any other callback-based constructs).

[Bruno Jouhier]

As usual, when this topic comes to the mailing list, there is a lot of talking and very little facts. Let's come back to the OP's code (and fix it a bit so that it also works when the path is *not* a symlink.

Here is how I implement it with streamline.js and express-streamline:

  CDN.use(config.CDN.baseUrl, function(req, res, next, _) {

      var path = config.base + "/cdn" + req.url;

      if (fs.lstat(path, _).isSymbolicLink()) path = fs.readlink(path, _); 

      res.writeHead(200, { "Content-type" : "text/html" }); 

      res.end( fs.readFile(path, _));

  });

Here is the "rule" behind it (in plain English)

If the request is a CDN URL do

  if the path corresponds to a symbolic link resolve the symbolic link

  write a 200 status and a text.html content-type

  write the file to the response

How do you implement it with async? with promises? with event emitters? 

How close it your code to the plain English rule? 

Does it matter? for the person who writes the code? for the person who will read this code? for the person who will modify the code 6 months later? 

How does it perform? 

How robust is it? What response (if any) will you get if the file does not exist, if someone makes a change and introduces a bug like a null dereference? 

...

[Ahmed Eldosoukey]

​I dislike your comment about comparison between Ruby on Rails and Node.JS, as Ruby is the core and Rails is the framework. You can use Node.JS as the core and SailsJS as the framework.

One more thing, callbacks is not an issue by itself, it becomes an issue when developers mixes their code into spaghetti style, where no modularity used.

My opinion is, if you're going to use a framework/technology/engine you need to more understand how it is serving your idea, not to come up with other things ideas and ask; why it is not so clear here as in others?!

My Regards,

Ahmed

To view this discussion on the web visit https://groups.google.com/d/msgid/nodejs/16d1698f-8b38-4953-aaa9-5973fc856c78%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Kind Regards,

Ahmed Eldosoukey

[Alexey Petrushin]

> I dislike your comment about comparison between Ruby on Rails and Node.JS, as Ruby is the core and Rails is the 

> framework. You can use Node.JS as the core and SailsJS as the framework.

The point in comparison with RoR was about the performance for classical Web Apps (not streaming or 10k). 

I doubt that if you use mount SailsJS atop of node it would improve its performance. 

So, the performance comparison was valid, no matter if I use bare node.js or higher-level construct like sails.js.

> One more thing, callbacks is not an issue by itself, it becomes an issue when developers mixes their code into

> spaghetti style, where no modularity used.

I'm not talking about if its an issue or not, that's highly subjective matter. I'm talking about the concrete

code snippets you can look at and obviously see if there's a difference. 

Async code is more complicated than sync. Could you write manageable programs using async code? Definitely, it is possible.

The question is - why can't you admit that it's more complicated than the sync code? Write async code with

some conditions and loops, proper error handling (also try to handle sync errors properly). Now - rewrite

this code in sync style. Compare these two snippets - does it looks the same?

> My opinion is, if you're going to use a framework/technology/engine you need to more understand how it is serving your 

> idea, not to come up with other things ideas and ask; why it is not so clear here as in others?!

These tools used by many as a web frameworks for creating Web Apps. So, why can't I compare

it in terms of performance, developer productivity, simplicity etc?

[Matt]

On Sat, Sep 20, 2014 at 9:17 AM, Bruno Jouhier <bjou...@gmail.com> wrote:

How do you implement it with async?

This is a good question, so here's an example translation (not the only way to do it):

    CDN.use(config.CDN.baseUrl, function(req, res, next) {

        if(err) return next(err);

        async.waterfall([

            function (cb) {

                fs.lstat(config.base+"/cdn"+req.url, cb);

            },

            function (stats, cb) {

                if(stats.isSymbolicLink()) {

                    async.waterfall([

                        function (cb) {

                            fs.readlink(config.base+"/cdn"+req.url, cb);

                        },

                        function (linkstr, cb) {

                            log.info("Redirecting request \""+req.url+"\" to \""+linkstr);

                            fs.readFile(linkstr, cb);

                        },

                    ], cb);

                }

            }

        ], function (err, data) {

            if (err) return next(err);

            res.writeHead(200,{"Content-type":"text/html"});

            res.end(data);

        });

    });

It's more code, but it improves the error checking. If it didn't have that "if" statement in there it would be a lot simpler - async code doesn't deal well with "if" statements I've found (there's no way to short-circuit).

Matt.

[Tom Boutell]

Your "if" needs an else clause to ensure the callback is eventually invoked.

> --
> Job board: http://jobs.nodejs.org/
> New group rules:
> https://gist.github.com/othiym23/9886289#file-moderation-policy-md
> Old group rules:
> https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "nodejs" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/nodejs/wBAJhYOzjqQ/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> nodejs+un...@googlegroups.com.
> To post to this group, send email to nod...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/nodejs/CAPJ5V2Z8yeYWX_HXaJqvbid5K%2B1T%3DNz--VF0GtXnKr%2BsyZsQrA%40mail.gmail.com.

[Bruno Jouhier]

Matt,

Your remark is spot on:

If it didn't have that "if" statement in there it would be a lot simpler - async code doesn't deal well with "if" statements I've found (there's no way to short-circuit

But what about:

• composition of async calls: asyncA(asyncB(asyncC(_), _), _)
  
• chaining of async calls: asyncA(_).asyncB(_).asyncC(_)
  
• async calls in expressions: asyncA(_) + asyncB(_), asyncA(_) && asyncB(_)
  
• async calls in array and object literals: [asyncA(_), asyncB(_)], { a: asyncA(_) }
  
• async calls in loop conditions: while (asyncA(_)) ...
  
• try/catch/finally: try { asyncA(_); } catch (ex) { asyncB(_); } finally { asyncC(_); }
  
• async calls in switch (condition or branches).
  

Libraries like async help you with sequences of statements and with loops but they don't help you at all with any of these constructs. So you have to handle them by "disassembling" your code. Let's take chaining for example. First you disassemble asyncA(_).asyncB(_).asyncC(_) into:

var a = asyncA(_);

var b = a.asyncB(_);

var c = b.asyncC(_);

// more ...

and then rewrite with callbacks (I'm omitting error handling for clarity):

asyncA(function(err, a) { 

a.asyncB(function(err, b) { 

b.asyncC(function(err, c) {

// more ...

});

});

});

When I see asyncA(_).asyncB(_).asyncC(_) in a piece of code, I IMMEDIATELY understand what it means. Things are very different with the callback version above. I have to trace the parameters (a, b, c) to understand (verify) that this is actually chaining.

Note: the "use named functions" approach goes even one step further in the disassembly process. It feels like wiring the steps with gosub instructions. And it forces you to give names to things that don't deserve a name (because they are so specific).

There is nothing really "difficult" in the callback version above but it is definitely more tedious to write, more difficult to read and more error prone (especially once you add the error handling). 

And typical application code is full of simple logic that you would naturally write with these language constructs (if/else, composition, chaining, ...). Async libraries don't help you with these; you have no choice but disassemble. 

There is a pattern to handle each of these language constructs (even try/catch/finally!) so you could write some kind of super-async library that would cover the entire language. But such a library would be hard to learn and cumbersome to use. This is what I realized before writing streamline. This is why I abandoned the idea of a library and went with a preprocessor. 

(*) coroutines can solve this problem too (fibers, ES6 generators).

One last word about your error handling comment: the streamline version does the error checking. You don't see it but it does it. Errors are funneled to the magic `_` callback and express-steamline forwards them to next. Streamline lets you handle errors with try/catch so you don't need to check them in every call; you can catch globally.

Bruno

[klrumpf]

I've been tasting it too the last couple of months in combination with koa and could not agree more, about half the lines and clean error handling ... 

You received this message because you are subscribed to the Google Groups "nodejs" group.

To unsubscribe from this group and stop receiving emails from it, send an email to nodejs+un...@googlegroups.com.

To post to this group, send email to nod...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/nodejs/50de4bdb-7214-412e-a9e2-e844011bccd4%40googlegroups.com.

[Floby]

I would probably write something like this

var baseUrl = config.CDN.baseUrl;

CDN.use(baseUrl, checkFileSanity, getRealPath, serveFile);

function checkFileSanity (req, res, next) {
  var filepath = path.join(config.base, '/cdn', req.url);
  fs.lstat(filepath, function (err, stat) {
    if(err) return next(err);
    stat.path = filepath;
    req.fileStats = stat;
    next();
  });
}

function getRealPath(req, res, next) {
  if(req.fileStats.isSymbolicLink()) {
    fs.readlink(req.fileStats.path, function (err, realpath) {
      if(err) return next(err);
      req.realpath = realpath;
      next();
    });
  }
  else {
    res.status(404).end();
  }
}

function serveFile (req, res, next) {
  fs.createReadStream(req.realpath).pipe(res);
}

On Monday, 15 September 2014 18:28:28 UTC+2, Ingwie Phoenix wrote:

Hai, everyone.

So I just figured out that Connect’s static file servers dont work well with symlinks…so I had tow rite myself a little workaround, a fallback router, per-se. But whilst I did so, I had a nice meeting with the callback hell. Here’s my code:

            CDN.use(config.CDN.baseUrl, function(req, res, next){

                if(err) { next(); throw err; }
                fs.lstat(config.base+"/cdn"+req.url, function(err,stats){
                    if(stats.isSymbolicLink()) {
                        fs.readlink(config.base+"/cdn"+req.url, function(err, linkstr){
                            if(err) throw err;

                            log.info("Redirecting request \""+req.url+"\" to \""+linkstr);

                            fs.readFile(linkstr, function(err, data){
                                if(err) throw err;

                                res.writeHead(200,{"Content-type":"text/html"});
                                res.end(data);
                            });
                        });
                    }

[Matt]

On Sun, Sep 21, 2014 at 10:12 PM, Tom Boutell <t...@punkave.com> wrote:

Your "if" needs an else clause to ensure the callback is eventually invoked.

Yes I know - the original didn't either. I'm assuming because it wasn't the entire actual code, more of an example.

[Tom Boutell]

There's an important difference though. With a regular "if" statement
in synchronous code, if there is no else clause, that doesn't prevent
the next line *after* the if from executing. With an "if" statement in
synchronous code, every possible branch must eventually result in
callback invocation, or the "next line" unrelated to the "if"
condition will never execute.

> --
> Job board: http://jobs.nodejs.org/
> New group rules:
> https://gist.github.com/othiym23/9886289#file-moderation-policy-md
> Old group rules:
> https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "nodejs" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/nodejs/wBAJhYOzjqQ/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> nodejs+un...@googlegroups.com.
> To post to this group, send email to nod...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/nodejs/CAPJ5V2Z656%3DPvqT%3DOzdC2A_cOi%2B4EeKQkVeik7yyO2PGxAi7zw%40mail.gmail.com.

[Alexey Petrushin]

Ha-ha, very good example, compare this async version it with the sync 

version below, and tell me that there's no problem with 

callbacks """if you know how to deal with it""" :D

Not saying that in even such a simple code it's extremely

easy to make small errors like missing `else` statements etc.

Also it doesn't preserve the call stack - so when you see error in console

you would have no context and no idea what happened...

And it still doesn't catch the sync errors...

And some of callbacks can be fired twice...

And with streams there are lots of events fired and should be properly handled...

And still many insist that it is "not a problem".

    CDN.use(config.CDN.baseUrl, function(req, res, next) {

        if(err) return next(err);

        async.waterfall([

            function (cb) {

                fs.lstat(config.base+"/cdn"+req.url, cb);

            },

            function (stats, cb) {

                if(stats.isSymbolicLink()) {

                    async.waterfall([

                        function (cb) {

                            fs.readlink(config.base+"/cdn"+req.url, cb);

                        },

                        function (linkstr, cb) {

                            log.info("Redirecting request \""+req.url+"\" to \""+linkstr);

                            fs.readFile(linkstr, cb);

                        },

                    ], cb);

                }

            }

        ], function (err, data) {

            if (err) return next(err);

            res.writeHead(200,{"Content-type":"text/html"});

            res.end(data);

        });

    });

    

Sync version.

    CDN.use(config.CDN.baseUrl, function(req, res, next) {

      var stats = fs.lstat(config.base+"/cdn"+req.url)

      if(stats.isSymbolicLink()) {

        var linkstr = fs.readlink(config.base+"/cdn"+req.url);

        log.info("Redirecting request \""+req.url+"\" to \""+linkstr);

        var data = fs.readFile(linkstr);

        res.writeHead(200,{"Content-type":"text/html"});

        res.end(data);

      }

    });

[Matt]

On Mon, Sep 22, 2014 at 9:37 PM, Tom Boutell <t...@punkave.com> wrote:

There's an important difference though. With a regular "if" statement
in synchronous code, if there is no else clause, that doesn't prevent
the next line *after* the if from executing. With an "if" statement in
synchronous code, every possible branch must eventually result in
callback invocation, or the "next line" unrelated to the "if"
condition will never execute.

Absolutely - I don't disagree with that at all. Like I said, async code doesn't deal with if statements very well. Maybe streamline does - I've yet to check it out. Another option might be to use Smalltalk-style class based logic, so that you don't even have if-statements.

Matt.

[Tom Boutell]

You won't catch me suggesting streamline isn't a good idea, since the
sublimetext snippets I'm using can be thought of as a crude
implementation of streamline. What I prefer about them is that I don't
have to integrate any preprocessors into my workflow for front or
back-end code.

> --
> Job board: http://jobs.nodejs.org/
> New group rules:
> https://gist.github.com/othiym23/9886289#file-moderation-policy-md
> Old group rules:
> https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "nodejs" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/nodejs/wBAJhYOzjqQ/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> nodejs+un...@googlegroups.com.
> To post to this group, send email to nod...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/nodejs/36905e32-1293-4ea3-b2d8-7015509afa31%40googlegroups.com.

[Matt]

On Tue, Sep 23, 2014 at 7:29 AM, Alexey Petrushin <alexey.p...@gmail.com> wrote:

Ha-ha, very good example, compare this async version it with the sync 

version below, and tell me that there's no problem with 

callbacks """if you know how to deal with it""" :D

Not saying that in even such a simple code it's extremely

easy to make small errors like missing `else` statements etc.

Also it doesn't preserve the call stack - so when you see error in console

you would have no context and no idea what happened...

And it still doesn't catch the sync errors...

And some of callbacks can be fired twice...

And with streams there are lots of events fired and should be properly handled...

And still many insist that it is "not a problem".

I don't insist it's "not a problem", but programming is always going to present challenges. I've programmed in both sync and async code, and neither is perfect. Node basically has trade-offs - I get better performance [1] at the expense of a slightly cumbersome programming model. I could go even more cumbersome for even more performance if I switched to C++ or Java. But I'm OK with where Node takes me for now.

The flip side with sync code of course is that sync programming is often a lot harder than people make it out to be. Your sync version doesn't handle errors at all, for example. Error handling is very language/environment specific and often an area of huge neglect. One thing Node got right was to always use the "error first" callback approach, that makes things like Streamline and Async.js possible. It's not perfect but all programming is trade-offs.

Matt.

[1] Whoever was arguing that Ruby and/or Rails has equal performance obviously has never ported a large project from Rails - I have and the performance difference is staggering

[Alexey Petrushin]

> I've programmed in both sync and async code, and neither is perfect. Node basically 

> has trade-offs - I get better performance [1] at the expense of a slightly cumbersome programming model. 

I compared Node.js with "slow" Ruby on Rails and the result is that node.js is only 1.3 times faster.

(maybe there's an error in my benchmark, but I can't find it so far).

http://alex-craft.com/blog/2014/ruby-on-rails-vs-nodejs

sources https://github.com/alexeypetrushin/web-frameworks-benchmarks

> Your sync version doesn't handle errors at all

I knew that someone would point to it. But actually it does handle errors. The whole request thread wrapped with try/catch (implicitly, inside of web framework code) - and it does handle errors.

In node.js it's impossible to handle errors in such a way implicitly because there's 

no context there. You don't know which error belongs to which request and has to do it manually.

[Matt]

On Tue, Sep 23, 2014 at 3:14 PM, Alexey Petrushin <alexey.p...@gmail.com> wrote:

> I've programmed in both sync and async code, and neither is perfect. Node basically 

> has trade-offs - I get better performance [1] at the expense of a slightly cumbersome programming model. 

I compared Node.js with "slow" Ruby on Rails and the result is that node.js is only 1.3 times faster.

(maybe there's an error in my benchmark, but I can't find it so far).

http://alex-craft.com/blog/2014/ruby-on-rails-vs-nodejs

sources https://github.com/alexeypetrushin/web-frameworks-benchmarks

Yes, like all benchmarks this one is horribly flawed. The simplest reason being your Rails server (using Puma) uses all cores. Your Node version is using one core. But that's beside the point - minor benchmarks like this show absolutely nothing of value.

Try porting a real application. I did.

 

> Your sync version doesn't handle errors at all

I knew that someone would point to it. But actually it does handle errors. The whole request thread wrapped with try/catch (implicitly, inside of web framework code) - and it does handle errors.

http://www.ckwop.me.uk/Why-Exceptions-Suck.html

Besides that, not all ruby functions/methods throw exceptions on errors (thankfully - in fact I'd venture to say that most don't), so you have to handle them locally/manually anyway, same as you do in Node.

Matt.

[Tom Boutell]

Alexey, that's an interesting comparison. Your benchmark does almost
no work in the node/ruby code itself. Since you used a modern Ruby
webserver that is based on lightweight threads rather than processes,
it's not too surprising that node is "only" 1.3 times faster under an
appreciable but not outrageous load. (:

So you're basically measuring how close you can come to delivering the
performance you'd get with async by using modern threads, provided
things aren't scaled up too far.

Not surprisingly it beats the pants off old school
one-connection-per-process model that is still favored in PHP
development. It's still slower than a direct async implementation.

I bet that if you used Cramp - an async ruby environment that I
randomly found with Google, there may be a better one - the
performance would be closer to equality.

But what you have not really tested is the impact of how much faster
v8 is than Ruby, because your code does so little actual work. If you
were doing some nontrivial template rendering here, anything that took
a decent amount of data crunching, I suspect the results would be
different.

Thanks, though, for bringing me up to speed on what the threaded model
is like in Ruby web development these days.

> --
> Job board: http://jobs.nodejs.org/
> New group rules:
> https://gist.github.com/othiym23/9886289#file-moderation-policy-md
> Old group rules:
> https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "nodejs" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/nodejs/wBAJhYOzjqQ/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> nodejs+un...@googlegroups.com.
> To post to this group, send email to nod...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/nodejs/ef189a34-c7ae-49ab-97c9-48035cea8bd0%40googlegroups.com.

[Bruno Jouhier]

http://www.ckwop.me.uk/Why-Exceptions-Suck.html

"Checked exceptions are certainly better than unchecked exceptions" !!!

Java had checked exception but its successors (C#) removed this feature. The designers of Java libraries have moved away from checked exceptions too (the presence of checked exceptions is a good way to know how old a library is). So I would not take this blog article as an authoritative source on exceptions.

Structured exception handling makes it possible to build very robust systems with very little error handling code. But you need a methodology. Mine is based on "software contracts" (a lvariant of B Meyer's original formulation, without retry). Behind every function that you write there is some kind of contract (usually tacit but you can make it explicit with pre/post conditions). If your function fullfils its contract it should return normally; if it cannot fullful it (for whatever reason), it should throw. If you follow this simple rule (the difficulty is not in the rule but in clarifying what your contracts are) you'll design robust systems with very little exception handling code. You'll throw a lot and you'll just have a few try/catch in strategic places. And try/finally will be your best friend: it will allow you to release resources and maintain invariants.

This works well when your code is written in sync style, and it generalizes well to future/promises (exceptions, if any, are thrown where you wait on futures). But it is not manageable with callbacks everywhere.

Bruno

[Matt]

On Tue, Sep 23, 2014 at 5:04 PM, Bruno Jouhier <bjou...@gmail.com> wrote:

So I would not take this blog article as an authoritative source on exceptions.

I don't. But it's a good read on some of the pitfalls exceptions can lead you towards.

[Alexey Petrushin]

> http://www.ckwop.me.uk/Why-Exceptions-Suck.html

That's just another theoretical talk. You may find tons of such stuff explaining how J2EE

is a superior next-generation technology or whatever other nonsense.

You may even find articles insisting that explicit error checks in Go "is the right way".

Instead of theoretical debates it is more interesting to compare the actual code. You can compare

the async and sync versions of code above, so you can judge by yourself which is simpler and easier.

> Yes, like all benchmarks this one is horribly flawed. 

That's very convenient point of view. If benchmark shows something you don't like just say that

it's flawed and proves nothing.

About the multicore - yes that maybe the case, but it can be run on single core machines - like

AWC EC2 or something similar to see how it perform on single core.

> Alexey, that's an interesting comparison. Your benchmark does almost no work in the node/ruby code itself.

Exactly. "No work" is the use case node.js is designed for - to be work-less network router.

So, it is the case node.js should shine. And, instead it turned out that """"webserver that is based 

on lightweight threads""" provides almost the same performance.

P.S. I will try to run benchmark on single core machine.

[Jimb Esser]

Just to add a slightly shorter/arguably cleaner example for using the async module (if using waterfall, embrace that it's a waterfall!):

CDN.use(config.CDN.baseUrl, function(req, res, next) {
  var req_path = config.base+"/cdn"+req.url;
  async.waterfall([
    fs.lstat.bind(fs, req_path),
    function (stats, cb) {
      if (!stats.isSymbolicLink()) {
        return next();
      }
      cb(req_path);
    },
    fs.readlink,
    fs.readFile,
    function (data) {

      res.writeHead(200,{"Content-type":"text/html"});
      res.end(data);
    }

  ], next);
});

I've spent most of my time in Node writing high performance network layers and related things and less time writing the "business logic", so I prefer to write as close to the metal as possible, mostly so that I can look at code and know at a glance what the async steps are, and I find this rather readable.  Additionally, when really trying to get the most performance, things like closures or .bind calls (which allocate memory that won't get collected in the early/quick generational garbage collection if your async tasks take any significant amount of time) are to be avoided (irrelevant in this case, since file I/O is going to have so much more overhead than anything you could possibly change syntactically here).

However, there's a pretty big barrier to entry here - getting a new (perhaps not as awesome as all of you) person to a team asynchronous Javascript code, they can be incredibly baffled (though, usually, only when it gets to things like loops, not for simple chained stuff like the examples in this thread).  I suspect that if I had them use streamline instead, I would have spent a lots less time unravelling the first async code they wrote (admittedly, once they've done an async loop themselves a couple times to understand it, and then learn how to write it much more succinctly with the async module, they're generally fine after that).

Also, assuming we want what the original posted requested (middleware that just handles symlinks, leaving regular files alone to the rest of his stack), the streamline version, I think, looks like this:

CDN.use(config.CDN.baseUrl, function(req, res, next, _) {
  var path = config.base + "/cdn" + req.url;

  if (!fs.lstat(path, _).isSymbolicLink()) {
    return next();

  }
  path = fs.readlink(path, _);
  res.writeHead(200, { "Content-type" : "text/html" });
  res.end( fs.readFile(path, _));
});

Though, as someone very familiar with the pitfalls of Javascript, and very unfamiliar with streamline, I'd have to look at the generated code to assure myself that's actually doing what I think I want it to, as I could imagine many ways in which that code could be converted to raw asynchronous code incorrectly (I'll assume for now it's smart enough and does it right ; ).

  Jimb Esser

[Bruno Jouhier]

Though, as someone very familiar with the pitfalls of Javascript, and very unfamiliar with streamline, I'd have to look at the generated code to assure myself that's actually doing what I think I want it to, as I could imagine many ways in which that code could be converted to raw asynchronous code incorrectly (I'll assume for now it's smart enough and does it right ; ).

Streamline is like CoffeeScript, it is a preprocessor.

People who use streamline, like people who use CoffeeScript, don't look at the generated code (at least our engineers don't), they trust the preprocessor. And you have to balance the odds of hitting a compiler bug vs the odds of making a mistake by crafting the callbacks by hand: omitting a callback, calling a callback twice, etc. From my experience the compiler is much more reliable.

Bruno