Re: MVC / flow control structure - Please review and comment

[Tim Oxley]

Only looked though this briefly but it looks pretty good to me. Separation of concerns via middleware, short methods, offhanding messy async logic to async lib, looks good. Off to a great start if this is your first app. I'd print this out on some nice 120gsm paper and put it in a frame for the mantlepiece. Make them grandkids proud.

-Tim

On Sunday, 14 October 2012 19:58:31 UTC+10, Mil Werns wrote:

Hi,

I want to implement a REST API with node.js and restify. Because it is my first node.js project, I spent some time to learn how to structure the application to avoid the callback hell. Please have a look to the following solution.


Assumptions
- This approach is for writing an end-user application, not a module or a driver in the sense of a public library.
- REST services are closely related to CRUD, so we have a lot of simple sequential/interdependent operations (get_data + validate_data [+ change_data] + save_data).


Goals
===> Focus is writing maintainable, clean and easy to understand code! <===
Conversely, this means...
- Performance is secondary (scaling horizontally if necessary).
- Avoid "low level" methods like process.nextTick(), instead use "high level" methods/libs like middleware and control flow modules.
- Use modules to implement a well known MVC structure.


Code
Please ignore the poor logging, the not very useful view tasks etc. in the following example. It should only demonstrate the MVC / callback structure.


/* server.js */

var restify = require("restify");
var router = require("./router");
...
router.route(server);
server.listen(...);


/* router.js */

var userController = require("./controllers/user");
... // more controllers

exports.route = function route(server) {
  server.get("/users",
    [
      userController.checkRead,
      userController.read,
    ]
  );
  server.put("/users",
    [
      userController.checkUpdate,
      userController.update,
    ]
  );
...
};


/* controllers/user.js */

var restify = require("restify");
var async = require("async");
var check = require('validator').check;
var sanitize = require('validator').sanitize;
var m = require("../models/user");
var v = require("../views/user");
...

var checkUpdate = function userCheckUpdate(req, res, next) {
  // validate user input
  try {
    check(req.params.id, "id").isInt();
    check(req.params.forename, "forename").len(1,50);
    check(req.params.surname, "surname").len(1,50);
    check(req.params.email, "email").isEmail();
  } catch(err) {
    if(err) {
      return next(new restify.InvalidArgumentError("Invalid arguments"));
    }
  }
  return next();
};


var update = function userUpdate(req, res, next) {
  // create model
  var model = {
    forename: sanitize(req.params.forename).xss(),
    surname: sanitize(req.params.surname).xss(),
    email: sanitize(req.params.email).xss(),
  };

  //*** "sub-methods" part ***

  // _example_ for data check
  function checkOwner(curData, callback) {
    if(curData.ownerId !== req.userId) {
      return callback(new restify.InvalidArgumentError("NotAuthorizedError"));
    }
    return callback(null, curData);
  };

  // _example_ for simple model manipulation
  function addSimple(model, callback) {
    model.updateDate = new Date().getTime();
    return callback(null, model);
  };

  // _example_ for complex model manipulation
  function addComplex(model, callback) {
    // use async again to simulate synchronous/parallel/... flow
    async.waterfall(
      [
        // do something
      ]
      function(err, model) {
        if(err) {
          return callback(new restify.InternalError());
        }
        return callback(null, model);
      }
    );
  };


  // *** main part ***
  // use async to simulate synchronous flow
  async.waterfall(
    [
      function(aNext) { m.get(req.params.id, aNext); },
      function(curData, aNext) { checkOwner(curData, aNext); },
      function(curData, aNext) { addSimple(model, aNext); },
      function(model, aNext) { addComplex(model, aNext); },
      function(model, aNext) { m.update(model, aNext); },
      function(model, aNext) { v.update(model, aNext); },
    ],
    function(err, model) {
      if(err) { return next(err); }
      res.send(200, model);
    }
  );
  return next();
};
...


/* models/user.js */
...
var get = function userGet(id, callback) {
  var curData = ... // DB query for ID
  if(err) {
    console.log("DB error: " + err);
    return callback(new restify.InternalError());
  };
  return callback(null, curData);
};

var update = ...
...


/* views/user.js */
...
var update = function userUpdate(model, callback) {
  // adjust model for output
  delete model.updateDate;
  ...
  callback(null, model);
};


As you can see, this approach uses the middleware concept + async module to separate the application in small (MVC) pieces that are hopefully easy to understand.
But as I said before: I'm a node.js newbie :)

What do you think about this structure?
Are there any drawbacks?

Thanks a lot for your help
Mil

[Eldar]

I recommend to have a look at the-box just because it suggests build systems style computational model which is definitely worth to consider. Also there is
an interesting "Out of the tar pit" article. It's not directly related to node but it mentions an idea of separation between business logic and control flow. The interesting point is that it seems that Node kinda encourages you to do this separation. While an emulation of classic sync control flows may be a bad idea.

[Eldar]

What about code, the goal " Focus is writing maintainable, clean and easy to understand code!" is not satisfied IMHO.

[Mil Werns]

@Tim:
Thanks, positive feedback is always welcome :)

@Eldar:
the-box looks very interesting. I am curious to see what in this project happens next. But currently, it is important for me to use only standard (within the meaning of established/mainstream) componentes and libraries (like e.g. async) that every node programmer knows.

Can you please point out some more details why you don't find this code clean/easy maintainable? And give me some example code for improvements?

[john.tiger]

On 10/15/2012 01:18 AM, Mil Werns wrote:

@Tim:
Thanks, positive feedback is always welcome :)

@Eldar:
the-box looks very interesting. I am curious to see what in this project happens next. But currently, it is important for me to use only standard (within the meaning of established/mainstream) componentes and libraries (like e.g. async) that every node programmer knows.

Can you please point out some more details why you don't find this code clean/easy maintainable? And give me some example code for improvements?

It seems you are targeting db activities - since db's are so specific, I'm not sure it makes sense to try for a generic framework but might be more useful to have implementations for specific dbs.  Couchdb is already REST - with node it makes sense to have a change listener and maybe an auth system.  Mongodb is popular with Noders but lacks a js REST interface (the one available is not very useful) so anything targeted here would be quite useful.  Might want to examine what python based sleepy mongoose does along these lines.  might also be nice to do a REST interface for Postgres on the sql side. 

--
Job Board: http://jobs.nodejs.org/
Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nod...@googlegroups.com
To unsubscribe from this group, send email to
nodejs+un...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en

[Eldar]

But currently, it is important for me to use only standard (within the meaning of established/mainstream) componentes and libraries (like e.g. async) that every node programmer knows.

No one encourages you to use the-box as is. It's essentials just about 50-100 lines of code. Each app might want to have it's own optimized version. But the pattern "hey I want this and that" is very common. Delegating tasks of getting dependencies to well known infrastructure is what makes code more simple and declarative. Here we see an idea of separating control flow from business logic in action.    

Can you please point out some more details why you don't find this code clean/easy maintainable? And give me some example code for improvements?

Without concrete use case it's very hard to do. What you showed just has some smells for me, but that's all pretty subjective. For example using of async utils for sequential execution. They are adding their own noise both in code and runtime. I personally can tolerate up to 5 levels of nesting with 2 space tabs but can't remember when I had more than 3 levels. At the same time async.waterfall adds 2 levels just by default. Another thing is 

function doSomething () {

var data = {}

function a () {}

function b () {}

a()

b()

}

or

function doSomething () {

var data = {}

a(data)

b(data)

}

function a (data) {

// body...

}

function b (data) {

// body...

}

probably it's better to create object-method for such use case

function DoSomething (data) {

this.data = data

}

DoSomething.prototype.a = function  () {

// body...

}

DoSomething.prototype.b = function  () {

// body...

[greelgorke]

totally agree about nesting and async is a smell.

[Mil Werns]

@John: I'm using couchDb. Currently, my db-query function is just a wrapper around the "request" module. Perhaps I will use the nano module in the future.

@Manuel: For small projects, I prefer a central routing module, but for bigger projects you are right of course. Thanks for this hint and the mvc link.

@Eldar, @greelgorke: I think you are right that the async module shouldn't be used for simple functions. I will adjust my project code. The only thing that bothers me is the costly error handling

m.get(..., function(err, body) {
  if(err) { return next(err); }
    m.upate(..., function(err, model) {
      if(err) { return next(err); }
...

A lot of same "if(err)" lines. So, using the async module for _complicated_ (deep nested) cases keeps the code cleaner.

@All: Thanks for your responses, I will optimize my code :)

[greelgorke]

you can use domains to replace the err handling with centralized approach