Reassurance for a new user (node.js + express + tedious)

[Chris Platts]

Hi all,

I've started reimplementing a clumsy old C# web API as a way to get to know node.js.  This is also the first time I'll be using modern JS for anything practical.  So, I'm well aware that I might be biting off more than I can chew here :)

To begin with, I made the rookie error of slamming everything into a single JS file.  Since it worked (but was ugly), I'm now trying to refactor into something reasonably sensible.

To explain, I'm using:

• express (web server, routing, etc)
• tedious (connection to a Microsoft SQL Server database)
• tedious-connection-pool

I've now got a main.js file which defines the routes, launches the server and receives requests (with routes/methods organised RESTfully).  Originally, I had too much database-related stuff in my route handlers - *facepalm*.  So, I've now also got a dblayer.js file containing a class to encapsulate db queries via tedious.

The dblayer class provides an execute function with these parameters:

• sql - the sql statement to execute
• params - an array of parameter objects.  Optional, but required if sql is parameterised
• response - an express response object.  Optional - If present, the function will stream the SQL result as JSON to the response as each row is received -- keeps men usage low for large queries.  If absent, then the result array is stored in memory and returned via the callback function
• callback - a function to call when the query is complete.  If response is null, then the callback function will receive any result set as an in-memory array.

So, for db access, the main route handlers instantiate a new dblayer instance on each request which, depending on what the operation is (e.g. GET), may either directly stream the result set as json to the express response object, or may execute a provided callback.  Also, a tedious-connection-pool is shared amongst the instances of the class so that each call to .execute() may draw on the pool.

Now... this all seems sort of logical to me.  There's clearly use-cases I've not implemented yet, but I should be able to add them without too much cruft.  However, I'd like to know if this is a sensible way to go?  

Many thanks,

Chris

 

[greelgorke]

for a beginning it could do. here my opinions (without really seen anything):

1. more files.smaller files. this will force you to separate concerns better. one module, one file. a module is a singe function (or class) that does one thing. route handlers may be an exception, if they are organized in resource way.

2. your db layer is to lowlevel imho. you should not pass sql from your routes to db layer. else it's not a real separation. your db layer should better provide functions or objects with access methods. look at the DataAccessObject pattern here: not dbLayer.execute('select * from foo'), but dbLayer.findFoosByColor('green', responseHandler)

3. you can squash response and callback to responseHandler. then you can check wether it is a function (then use it as callback) or a WritableStream (presence of write and end methods i.e.). Even better, you make your DAOs as streams so the user can just dbLayer.find().pipe(otherstream)

so far

[Chris Platts]

On Friday, 2 August 2013 08:28:30 UTC+1, greelgorke wrote:

for a beginning it could do. here my opinions (without really seen anything):

 

<snip>

 

Excellent -- thank you very much for your reply!

 

So essentially, I should be replicating exactly what I'd do in the languages I'm used to (C#, etc) -- A service layer handling requests, a business layer and a 'DataAccessObjects' layer.

 

Thanks in particular for your third point -- I really like the idea of the data objects being streams which can be fed through the business layer and straight through to the response.

 

Now, the last real ''huh?' point I've got is how to handle object (table) relationships so I can return nested JSON containing related objects to the API consumer.  It seems that if I were using a NoSQL database, I'd have no problems at all.  But since I'm tied to MS SQL, my options seem rather limited ORM-wise.  I suppose I could construct the model object graph manually when querying the data source, but that seems painful!  Anyway, that's probably not a question for here -- I'll go hang around the sequelize forums :)

 

Thanks again for your help,

 

Regards,

Chris

[greelgorke]

arm, yes, sequelize implements an Active Record pattern, which is different from DAO. Active Record don't load relatiosn by default, but offers you methods to load them if you want. So Group.find().members.find() creates 2 sql fetches. DAO- friendly ORMs like Hibernate works with lazy-loading proxy here. In Document-DB's like Mongo you have 2 options: denormalize (the the all is loaded and is may be redundant) or normalize and join on application level.

[Iain Plimmer]

Hi Chris

I'm in the same position as you were two years ago at the moment, i've been working on the Microsoft WebApi for a few years now, and it's great, although i'm getting more and more into server side Javascript so thought to look at TDS, Express, NodeJS and Angular.

I'm interested to know, how you did you get on with this? Did you ever turn back or was it an inspiration to you?

Iain