Where do you put the meat and potatoes code?

[netpoetica]

I've been working with Node for a few months now, mainly out of books and tutorials, just sort of trying to get a handle on using JavaScript on the backend. It all seems well and good, but I'm still unclear about how to properly organize code in your Node program. For example, if I'm porting a completely client-side app over to Node, and I have functions that I normally use on the client side to, let say, build out an HTML widget, but I want to be able to access them from one of my route files as well so that I can build the widget behind the scenes and serve it up from the backend - where do you put this kind of code? 

Essentially, where do you put/how do you use model/controller functions/value objects/files across multiple views in an organized, modular manner?

We all know we don't want to pollute the global namespace, but, there are some functions that will need to be used site-wide on both the client and server side. It seems silly to me to use a require for the same function to be stored in memory again if two separate routes will need that function in order to build their instance.

Most of the tutorials I've seen and even some Github projects have these bloated app.js files where it seems they are storing the values globally - but even then there are instances where it's unclear whether or not specific routes (files starting with module.export = ) will have access to these values (let's say, Model-level functions and value objects).

I hope this isn't completely silly. I've done a pretty good deal of research into this, and haven't quite found anything that really speaks to this. I'm completely open to being redirected to articles you may have found and/or Github repos where simply studying the structure would make this clearer. If it's any help, I'm mostly working with Express and using a routing structure similar to typical Express apps, and just curious how to share functions and objects across multiple views, and where to put them for proper loading.

Thanks much, and happy thanksgiving!

[Nuno Job]

npm sir :)

nuno

--
Job Board: http://jobs.nodejs.org/
Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nod...@googlegroups.com
To unsubscribe from this group, send email to
nodejs+un...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en

[Marco Rogers]

One thing you may be missing is that requiring a module multiple times does not load multiple version of it. Modules are cached after the first time they are required, so you get the same reference every time. So one answer to your question is that people re-use modules by requiring them in multiple contexts. You might require('helper') in multiple views or controllers and it's not a problem. You might also check out geddyjs http://geddyjs.org/, which takes a rails-like approach in providing conventions of where to place your code.

As for sharing modules between client and server, I agree there isn't a shared convention for this yet. I personally haven't found compelling reasons to share a bunch of code between environments yet.

:Marco

[themitchy]

Not a silly question at all.

In the most general terms, you want to:

• put your code into modules
  
• expose that code through module.export
  
• require that module wherever you may need it
  

Specific to express you could put your entire request handler in seperate module:

var loginHandler = require('./loginHandler');

app.get('/login', loginHandler);

And then make sure that loginHandler.js does something like:

module.exports = function(req,res) { /* perform auth */ };

Or you could put smaller blocks of functionality into logical modules. For example, your db storage code could live in one place:

var itemStore = require('./itemStore');

app.post('/item/:id', function(req,res) {

var item = req.body;

var id = req.params.id;

itemStore.saveItem(id, item, function(err, result) {

if (err) return res.send(err,500);

res.send(result);

}); 

});

The itemStore module could export several functions to make your life easier:

module.exports.saveItem = function(id, data) { /* parse some data, save an item */ };

module.exports.retrieveItem = function(id) { /* retrieve item, format data, return data */ };

Another common and useful pattern is to have your module return classes.  So item store could be a class that you instantiate with config data and reuse.  I won't get into how to build up JavaScript classes but you could build something that works along these lines:

var Store = require('genericStore');

var itemStore = new Store('items');

var userStore = new Store('users');

itemStore.init(/* some config or bootstrap data */);

userStore.init(/* some config or bootstrap data */);

itemStore.find(...);

userStore.updateName(...);

These are just a few patterns that I've found useful in various situations.  I don't think there's any right answer so don't be afraid of making mistakes.  Find out what works for you and then share it back with the community.

-mitch

On Thursday, November 22, 2012 8:05:37 AM UTC-8, netpoetica wrote:

[Naouak]

There is also putting you express app variable in a module and doing routing in the different modules you load.

var app = require("server.is");
app.get("/foo",function(req,res,next){});

--

[netpoetica]

Thanks for the responses everyone. Def. interesting about require() working from cache - I'll have to experiment with that a bit, but it does make sense especially since each route file will be a unique instance, it's not like you're really requiring twice. Technically, the codebase itself may require the same file, but, it's only really being called in if that particular route is served up, so yeah I can see how require could be used very efficiently.

@themitchy your response way very thorough. I like the idea of using require to export a specific handler like that, that is a very clean solution. I guess my concern would be what if you have10 request handlers, 4 or 5of which use the same code held in another module? How would you link up that system of sharing?

[themitchy]

I really like Naouak's idea of requiring the server into the modules that declare the routes.  That's also a good way to demonstrate how require works to your advantage here.  Here's a short example of what that might look like:

app.js:

var express = require('express');

var app = express();

// ... configure express, etc...

module.exports = app;

index.js:

var app = require('./app');

require('./userRoutes');

require('./itemRoutes');

// ... start your app, etc...

userRoutes.js:

var app = require('./app'); // same instance as in index.js

app.get('/user/:id', function(req, res) { ... });

app.get('/user/:id/friends', function(req, res) { ...});

itemRoutes.js:

var app = require('./app'); // same instance as in index.js and userRoutes.js

app.get('/item/:id', function(req, res) { ... });

app.post('/item/:id', function(req, res) { ...});

What's key here is that the first time you var app = require('./app') in index.js, all the code in app.js will be executed. The value of app will be what we put into module.exports(in app.js).  You could think of modules as functions that get executed and return their module.exports when 'required'.  When we called require('./app') again in the other two modules we got back the exact same instance.  The code in app.js is never run again but everything that was exported still has a reference to any variables or objects you created in that module.  I only mention this last part as a reminder to brush up on js closures so you don't start writing memory leaks.

-mitch

[Ryan Schmidt]

On Nov 22, 2012, at 22:42, themitchy <mi...@nodefly.com> wrote:

> I really like Naouak's idea of requiring the server into the modules that declare the routes. That's also a good way to demonstrate how require works to your advantage here. Here's a short example of what that might look like:
>
> app.js:
>
> var express = require('express');
> var app = express();
> // ... configure express, etc...
> module.exports = app;
>
> index.js:
>
> var app = require('./app');
> require('./userRoutes');
> require('./itemRoutes');
> // ... start your app, etc...
>
> userRoutes.js:
>
> var app = require('./app'); // same instance as in index.js
> app.get('/user/:id', function(req, res) { ... });
> app.get('/user/:id/friends', function(req, res) { ...});
>
> itemRoutes.js:
>
> var app = require('./app'); // same instance as in index.js and userRoutes.js
> app.get('/item/:id', function(req, res) { ... });
> app.post('/item/:id', function(req, res) { ...});
>
> What's key here is that the first time you var app = require('./app') in index.js, all the code in app.js will be executed. The value of app will be what we put into module.exports(in app.js). You could think of modules as functions that get executed and return their module.exports when 'required'. When we called require('./app') again in the other two modules we got back the exact same instance. The code in app.js is never run again but everything that was exported still has a reference to any variables or objects you created in that module. I only mention this last part as a reminder to brush up on js closures so you don't start writing memory leaks.

Although I've been dabbling in node for over a year, this point that the code in a file is only executed once no matter how many times you require the file, and that you'll get back the same instance each time, is something I'd only recently come to realize. I could have worked it out sooner if I'd thought about it, but it never even occurred to me to formulate the question. If anyone reading this ever writes a node book or tutorial or presentation, please make this nifty feature very clear as early as is practical.

My problem was due in part to a node tutorial I read early on in my learning, which as I recall went to what seemed like great lengths to require everything only once, and then pass the reference around between other required files, for example:

var express = require('express');
var app = express();

var routes = require('routes')(app);

That tutorial also used a database, so it got a mongoose connection, and handed that object on to the routes and other places as well. I kept trying to emulate this structure and finding it very confusing and tedious, especially as I kept rearranging what type of code went into each file, something which happened rather a lot in the beginning when I didn't have a good understanding of how and when to modularize my code. And all the examples that do everything in a single app.js file didn't help me understand proper code modularization either. It even took quite awhile before I realized that I didn't have to store and retain and pass along all the mongoose models that I received as the return value of the mongoose.model() call; I could just call mongoose.model() again later with the model name to get them back.

Prior coding experience in other languages also colored my thinking. I jumped ship to node from PHP, which I imagine other weary PHP developers have done and will do. Unlike in node, in PHP, require doesn't create a new scope, and doesn't usually return anything; it just includes the code, as if you had pasted it in. And in PHP, as in client-side JavaScript, it's common, if possibly bad practice, to use a few global variables here and there. For example in the web sites that we wrote at my last web programming job, there would typically be a MySQL connection, which got assigned to the global variable $db. My early learning on node told me node allows no global variables, which reinforced my incorrect thought about needing to pass variables around between files myself.

In my current app design, the app variable is created and only used in the entrypoint, app.js, like most examples (or the boilerplate express creates for you). I like your idea above to make the app an export of a module and include it from the route files instead. It's given me much to think about; thanks.

[Alexey Petrushin]

I believe exposing code via `module.exports` is a right way to go, You should use it when in doubt. I prefer it when I build libraries and reusable components that should be robust and live well in the wild.

But, although it's good the `module.exports` way is too verbose for my taste, so, in case of my own application, when I control the environment I usually prefer more compact way - `globals.app = {}`, and later just use this `app` as a globally accesible variable `app.settings = {}, app.MyClass = fun(){}, ....`

[Jeff Schwartz]

You have to make sure when exposing app (or any object or value) via exports that it is initialized prior to being required by any other module. So in this case the routing module would break if it were required by server.js prior to app having been initialized. I use this technique a lot. It allows routing modules to act as controllers. Combined with common.js ability to modularize whole folders through index.js then allows encapsulating all your controllers in a structured manner. I wrote an article on this very subject which can be found at http://jefftschwartz.wordpress.com/2012/09/10/taming-those-unruly-routes/. This article was a follow up to another article which explores using folders for encapsulation in common.js and that article can be found at http://jefftschwartz.wordpress.com/2012/08/23/use-express-route-specific-middleware-to-access-mongodb/.

[Mark Hahn]

> As for sharing modules between client and server, I agree there isn't a shared convention for this yet

I highly recommend using a library in the client to support the identical modules used in the backend.  They  are quite easy to use.  Then get in the habit of using modules everywhere including client-only code.

On Fri, Nov 23, 2012 at 6:12 AM, Jeff Schwartz <jeffts...@gmail.com> wrote:

You have to make sure when exposing app (or any object or value) via exports that it is initialized prior to being required by any other module. So in this case the routing module would break if it were required by server.js prior to app having been initialized. I use this technique a lot. It allows routing modules to act as controllers. Combined with common.js ability to modularize whole folders through index.js then allows encapsulating all your controllers in a structured manner. I wrote an article on this very subject which can be found at http://jefftschwartz.wordpress.com/2012/09/10/taming-those-unruly-routes/. This article was a follow up to another article which explores using folders for encapsulation in common.js and that article can be found at http://jefftschwartz.wordpress.com/2012/08/23/use-express-route-specific-middleware-to-access-mongodb/.

[netpoetica]

Mark - could you elaborate a little more on that? It seems like in Node the words "exports" "modules" and "libraries" are all used somewhat interchangeably although I don't think people always mean them the way they are using them.

When you say library, I almost start to think of npm and the modules in there, and I most certainly don't think you mean that, or that anyone would want my super-specific application code in NPM.

When I think of a library, I think of an API like jQuery or a plugin like TouchSwipe for touch events. I'm just a bit unsure what you mean by library here

Thanks a bunch
-Keith

[Mark Hahn]

By library I meant a single piece of code that runs in the client to load and link modules such as Yabble.

I use commonJS modules, both public ones available on npm and elsewhere, and my own private modules.  All code is in modules and many run on either the server or the client.