Best practices for sharing code and data between the server and client

[Amjad]

Hello everybody!

I'm new to the great world of Node.js and have been playing around with different modules and frameworks. However, there seems a lot of methods for sharing server code with the client, and there are no default way for doing that.

I have come across many ideas:

express-expose: this is a nice replacement for parsing JSON objects rendered by the the server.
https://github.com/visionmedia/express-expose

This article exposes a hack to share backbone models on the Node server, and they are trying to neat their hack with their Capsule and Thoonk frameworks:
http://andyet.net/blog/2011/feb/15/re-using-backbonejs-models-on-the-server-with-node/

Syncrhonizing Backbone model using socket.io
https://github.com/scttnlsn/backbone.io

DNode and RPC as a method to use server methods on the client and vice versa
https://github.com/substack/dnode

I know I've mixed a lot of topics, I thought sharing my confusion would help to get more clarification, especially that there are a lot of production projects based on Node, andI think that the community certainly have reached a good collection of concepts and tools for sharing code between the server and client. I would be very grateful for sharing your thoughts about that.

Best regards,

Amjad

[substack]

On Wednesday, May 30, 2012 4:58:46 AM UTC-7, al-Amjad Tawfiq Isstaif wrote:

Hello everybody!

I'm new to the great world of Node.js and have been playing around with different modules and frameworks. However, there seems a lot of methods for sharing server code with the client, and there are no default way for doing that.

browserify lets you do node-style require()s in the browser, so a lot of node code will just work browser-side, including many packages from npm 

https://github.com/substack/node-browserify

[Mark Hahn]

It should be called nodeify, not browserify.

--
Job Board: http://jobs.nodejs.org/
Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nod...@googlegroups.com
To unsubscribe from this group, send email to
nodejs+un...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en

[tjholowaychuk]

express-expose is just for small things like maybe exposing some
configuration
to the client, or a little bit of info about a user etc. If you
perform IO for
things like this it can really hurt your UX, but other than that you
should use
something like browserify if you want to share entire modules

On May 30, 4:58 am, Amjad <isst...@gmail.com> wrote:
> Hello everybody!
>
> I'm new to the great world of Node.js and have been playing around with
> different modules and frameworks. However, there seems a lot of methods for
> sharing server code with the client, and there are no default way for doing
> that.
>
> I have come across many ideas:
>
> express-expose: this is a nice replacement for parsing JSON objects

> rendered by the the server.https://github.com/visionmedia/express-expose

>
> This article exposes a hack to share backbone models on the Node server,
> and they are trying to neat their hack with their Capsule and Thoonk

> frameworks:http://andyet.net/blog/2011/feb/15/re-using-backbonejs-models-on-the-...
>
> Syncrhonizing Backbone model using socket.iohttps://github.com/scttnlsn/backbone.io
>
> DNode and RPC as a method to use server methods on the client and vice versahttps://github.com/substack/dnode

>
> I know I've mixed a lot of topics, I thought sharing my confusion would
> help to get more clarification, especially that there are a lot of
> production projects based on Node, andI think that the community certainly
> have reached a good collection of concepts and tools for sharing code
> between the server and client. I would be very grateful for sharing your
> thoughts about that.
>
> Best regards,
>
> Amjad
>

> *
> *

[Martin Cooper]

On Wed, May 30, 2012 at 4:58 AM, Amjad <iss...@gmail.com> wrote:
> Hello everybody!
>
> I'm new to the great world of Node.js and have been playing around with
> different modules and frameworks. However, there seems a lot of methods for
> sharing server code with the client, and there are no default way for doing
> that.

That's in part because it rather depends on what you're trying to do.
For example, you might be:

* trying to take existing code written for Node and run it in the browser
* trying to take existing code written for the browser and run it in Node
* trying to write a new codebase that will run in both places

As has been mentioned, browserify is probably the best solution for
the first of these. I think it's safe to say that it garners the most
attention, in part because it's good at what it does.

For the last case, you might want to take a look at YUI, which
provides the abstractions to let you write your code once and run it
in both places. Yahoo! has built Mojito on top of YUI and is using
that to create multi-device apps that transparently share code between
client and server. (BTW, you'll likely find that YUI is much more
lightweight and modular than you think.)

--
Martin Cooper

> I have come across many ideas:
>
> express-expose: this is a nice replacement for parsing JSON objects rendered
> by the the server.
> https://github.com/visionmedia/express-expose
>
> This article exposes a hack to share backbone models on the Node server, and
> they are trying to neat their hack with their Capsule and Thoonk frameworks:
> http://andyet.net/blog/2011/feb/15/re-using-backbonejs-models-on-the-server-with-node/
>
> Syncrhonizing Backbone model using socket.io
> https://github.com/scttnlsn/backbone.io
>
> DNode and RPC as a method to use server methods on the client and vice versa
> https://github.com/substack/dnode
>
> I know I've mixed a lot of topics, I thought sharing my confusion would help
> to get more clarification, especially that there are a lot of production
> projects based on Node, andI think that the community certainly have reached
> a good collection of concepts and tools for sharing code between the server
> and client. I would be very grateful for sharing your thoughts about that.
>
> Best regards,
>
> Amjad
>
>

[Amjad]

Thank you very much everybody for your notes!

@Martin
I tried to have a fast look at YUI. What gives YUI the ability to run client code on the server? I mean what makes it different from other Javascript "MVC" frameworks such as Backbone.js?

[Martin Cooper]

On Sat, Jun 2, 2012 at 6:20 AM, Amjad <iss...@gmail.com> wrote:
> Thank you very much everybody for your notes!
>
> @Martin
> I tried to have a fast look at YUI. What gives YUI the ability to run client
> code on the server? I mean what makes it different from other Javascript
> "MVC" frameworks such as Backbone.js?

A couple of quick examples:

* You specify your modules' dependencies declaratively, which means
that YUI can use the appropriate mechanism to load them, depending on
whether your code is running in the browser (and either static- or
lazy-loading) or on the server. It's transparent to you.

* With YUI, you would write your XHR / XDR code using the YUI.io API.
When that code is running on Node.js, YUI provides a different
transport behind the same YUI.io API, such that your code works
transparently.

* When you're working with the DOM, you use YUI's (very thin) Node
abstraction. This is the browser DOM in a browser, and jsdom on the
server, so that your DOM manipulation code continues to work in either
place.

Dav Glass has done a few presentations around how all this works. If
you're at all interested, I'd encourage you to watch at least one.
Dav's an entertaining presenter, and his demos are always fun.

http://www.yuiblog.com/blog/2010/04/05/running-yui-3-server-side-with-node-js/
http://www.yuiblog.com/blog/2010/04/09/node-js-yui-3-dom-manipulation-oh-my/
http://www.yuiblog.com/blog/2011/11/07/rocking-yui-on-node-js-and-mobile/

--
Martin Cooper

[Amjad]

Really interesting thank you Martin.

[Jeff Barczewski]

Requirejs is currently one of the best ways to work with AMD modules both on browser and in node.js. Requirejs "wrap" plugin is nice for wrapping browser modules that don't support AMD. You can use in the browser without a build step (so that is nice for making source change and just hitting refresh) and then when you are ready for deploy, it can bundle things up and minimize.

Also for small projects that you want them to work both in AMD and non-AMD, you can use some boilerplate like these examples in UMD (universal module definition) https://github.com/umdjs/umd, specifically https://github.com/umdjs/umd/blob/master/returnExports.js which is one that works in all places. If you wrap your code with this, then it will work in all places without adding any dependencies.

I use mocha with chai to be able to support browser and node.js testing.

I believe there are still things that can be done to improve sharing, but these are a good start.

On Saturday, 2 June 2012 11:26:55 UTC-5, Nils Lattek wrote:

I found it a little difficult to setup a module sharing solution, because of the two different module formats (AMD and CommonJS). Projects such as browserify are awesome, but I wanted to try it with AMD modules.

So after a lot of googling and experimenting I created a small demo project which shows how to share code between backbonejs and nodejs.

I do not use it to share the complete model, because most db-modules (like sequelizejs or mongoose) in node have their own way of defining models. So you would have to merge the definition code of a backbone model with the one of a mongoose model.

What I tried to do is share the validation logic between the client and the server. This kind of code can be executed regardless of which model format (backbone, mongoose, sequelize) you are using.

You could also use it for other kind of modules or to share Constants/enums across the client and server.

The example also shows how to setup mochajs unittesting for these modules.

There are still some things which could be solved better and I am interested in seeing more examples from other people.

You can find it here:

https://github.com/NilsLattek/backbone-requirejs-node-boilerplate

[Nils Lattek]

Shoshy,

thanks for your kind feedback. Yes you can just move any javascript file to the shared/js folder and adjust the file to require amdefine at the top (like in the example entry_validation file).

Then make sure to use the correct (relative) path on the client and server. Take a look at client/js/entry.js and server/models/entry.js

If you are using a Backbone.Model do not forget to require backbone on your server.

Regarding your SQL question: I do not have a solution for you. After experimenting a little bit with MySQL using http://www.sequelizejs.com/ (works with MySQL and Postgresql) and https://npmjs.org/package/mysql my code quickly turned into deep callback nesting. Especially when dealing with relationships. This made me think about that maybe NoSQL is a better fit for the Nodejs style, but I am still experimenting and looking for the opinion of other people.

On a side note if your are using a serverside ORM framework like Sequelizejs which has its own Model-Class/Syntax you cannot use the same model on the server and client because Backbone models are created by extending Backbone.Model and Sequelize models by using sequelize.define. Thats why I was going to just share the validation logic.

  

Nils

On Friday, August 10, 2012 10:38:29 PM UTC+2, shoshy wrote:

Nils ,

Thanks so much for your boilerplate, i'm new to node.js but i've been working with backbone, we have the same boilerplate structure and bits of code for that matter. 

My question is , i need to share the SAME model of the client. Rather it's for extending it or use it as-is. Does that mean that i can just put the models directory from the client in the shared and use it from backbone as is? 

Also what is your suggestion for generalised SQL driver for node? (lets say i want to connect to mysql OR to postreSQL but i don't want 2 . Like PDO in php)

Thanks again! would love to keep in touch. I'm adopting your boilerplate. 

P.S. it needs updating (express changed their commands a bit)

Shoshy

On Saturday, June 2, 2012 7:26:55 PM UTC+3, Nils Lattek wrote:

I found it a little difficult to setup a module sharing solution, because of the two different module formats (AMD and CommonJS). Projects such as browserify are awesome, but I wanted to try it with AMD modules.

So after a lot of googling and experimenting I created a small demo project which shows how to share code between backbonejs and nodejs.

I do not use it to share the complete model, because most db-modules (like sequelizejs or mongoose) in node have their own way of defining models. So you would have to merge the definition code of a backbone model with the one of a mongoose model.

What I tried to do is share the validation logic between the client and the server. This kind of code can be executed regardless of which model format (backbone, mongoose, sequelize) you are using.

You could also use it for other kind of modules or to share Constants/enums across the client and server.

The example also shows how to setup mochajs unittesting for these modules.

There are still some things which could be solved better and I am interested in seeing more examples from other people.

You can find it here:

https://github.com/NilsLattek/backbone-requirejs-node-boilerplate

On Wednesday, May 30, 2012 1:58:46 PM UTC+2, al-Amjad Tawfiq Isstaif wrote:

[shoshy]

Nils,

Thanks a lot for your quick reply. 

I'll test it, i did see the entry_validation, but didn't know if backbone likes to be used on both sides :) same goes with require.js... i'll keep you updated.

regarding SQL driver - is think this is a major pitfall for node.js. node.js should have a native (core) , out-of-the-box generalised drivers to SQL DB , just like php's PDO.

Regarding the deep callbacks, have you used async (for flow control)  https://github.com/caolan/async/  , and  http://fennb.com/nodejs-a-giant-step-backwards  ? 

thanks about the ORM tip

shoshy 

[greelgorke]

to the topic: take a look at derbyjs.org :)

[Mariusz Nowak]

See also https://github.com/medikoo/modules-webmake the idea is to use same module "format" on server and client side, so you can easily share same code on both sides.

I use it with success. Currently I work with application for which client-side code is built of over 200 Node.js style modules and 60% of those modules are also used on server-side.

-- 

Mariusz Nowak

http://github.com/medikoo

http://twitter.com/medikoo

[Jeff Barczewski]

Mariusz,

After reviewing your readme, webmake looks pretty nice!

One idea, if you made your API signature use 

  function (require, exports, module)

then it would be compatible with AMD's commonjs signature, and there could be some nice ways to integrate with AMD modules and tools. Just something to consider.

I like the elegance of your approach, seems to really make things easy. I will have to give it a try soon.

All the best,

Jeff

[Phoscur]

Server not found

[Mariusz Nowak]

@Jeff Thanks for that note, I might change that signature in near future just to resemble some standard, however I doubt any external tools can benefit from that, at least I can't imagine any valid use case.

[Alexey Petrushin]

Recently switched from Brunch to Browserify, Brunch is nice project but I needed more control and 

custom folder structure for the project.

One problem - was no easy way (or I didn't found it) to browserify also templates for my backbone.js app.

So I wrote tiny plugin resourcify (https://github.com/alexeypetrushin/resourcify) that handles 

templates (eco, ejs, handlebars, hogan, jade) same way as ordinary js files.

Maybe it can be useful for someone.

[Ted Young]

>
> One problem - was no easy way (or I didn't found it) to browserify also templates for my backbone.js app.
> So I wrote tiny plugin resourcify (https://github.com/alexeypetrushin/resourcify) that handles
> templates (eco, ejs, handlebars, hogan, jade) same way as ordinary js files.
> Maybe it can be useful for someone.

Nice! I have a little module called stitchit https://github.com/tedsuo/stitchit that I use for templates in non-browserify situations, but i will use yours next time I switch to browserify.

[Gustavo Machado]

Here's a pretty good article about doing it with AMD: http://joseoncode.com/2012/08/30/share-your-server-side-templates-to-the-browser/

Gus