ORM and Resource Ramblings

[the_angry_angel]

Hi guys,

I'll start off by saying I'm considering adopting sails for at least prototyping an internal project at work - it would be good to know what direction you're taking sails - I don't want to end up in a situation that I end up having to fork sails to shoehorn it into what I need :)

I'm looking at waterline, and just wondering if you had considered using JugglingDB? It does seem to tick most, if not all, of the boxes that waterline is going to tick. There's the additional benefits of; There are a pile of adapters that already exist, integrating it shouldn't be too difficult, the API wouldn't have to change massively for defining models, or passing models to controllers, it seems to actively maintained, and would allow you to concentrate on other parts of sails. My main concern is that db abstractions/ORMs tend to be dull to write and I've been stung by one disappearing before.

The second question I've got revolves around model scaffolds. The way in which scaffolds work strike me as having significant potential to be surprising/dangerous.

Right now if you have a model and create a controller you still get default behaviour of scaffolds unless you specifically overwrite all possible scaffold actions.

Although the possibility for code duplication would be higher for some applications, I believe for the vast majority of applications you'd want to customise the default scaffold behaviour anyway (my use cases for sails might be unusual, but I'd imagine not). It makes more sense to me if a controller was generated at the same time as a model, with the default actions. That way users of sails explicitly know how this works, can use the generated code as a starting point, and know that if they delete an action it's gone.

- taa

[Mike McNeil]

Hey Karl,

(copying sailsjs-dev on this)

That's awesome-- Sails is a perfect choice for prototyping an application.  The comforting thing is that the core pieces are at least as stable as Express and Socket.io are-- and the ORM is a thin layer that uses existing, awesome datastore drivers whenever possible.

Re: jugglingDB:  

Originally, Sails used Sequelize.  I thought long and hard about using Juggling before I wrote waterline.  It's not a bad engine at all, but I wanted us to have the freedom to some things that are fundamentally new.  In Sails, I see a great opportunity to take an MVC formula that developers are used to and revolutionize it

Take sails-twitter, for instance.  Or imagine a model you want to use to serve and manipulate images, and imagine being able to quickly move that between different binary adapters (sails-fs, sails-s3, sails-swift, etc.)

// api/models/Image.js

module.exports = {

  adapter: 'sails-fs',

  path: 'uploads',

  accept: ['image/gif', 'image/jpeg', 'image/png', 'image/svg+xml']

};

That said, I'm totally open towards efforts to extend Sails to support other ORMs like juggling for folks who don't want to wait for features like built-in associations, etc.

Re: scaffolds

( I think this will answer the question about our direction as well).

Good point.  At the very least, I would agree that we need a flag you can set on the command line to actually generate the scaffold controller code.  My only question there is whether it should be the default behavior.  The reason I didn't do this originally is I remembered my own apprehension the first time I saw Grails' generated scaffolds.  What does everyone else think about that?

On the other hand, in production, for several of our customer apps, 80% of our backend is served using the scaffolds, and for a lot of basic functionality, they serve the purpose (search, sorting, pagination, 

What I'd like to do though, over time, is focus on putting as many conventions in the scaffolds as possible.  For instance, right now, if you set ?redirect=http://google.com when hitting /model/create, you'll get redirected to the destination you specify. This was added to make it easier to use Sails.js to handle forms.

-Mike

--

Mike McNeil

Founder

    

       C      O
      NFI    DEN
      TIA   L i
      nfo  rma
      tion in
       tended
    only for t      he addressee(s).
  If you are not the intended recipient, empl
 oyee or agent responsible for delivery to the
  intended recipient(s), please be aware that
   any review, dissemination, use,distribut
     ion or copying of this message and its
       contents is strictly prohibited. If
      you receive this email in error, ple
    ase notify the sender and destroy any
   paper or           electronic copies    immediately.

[the_angry_angel]

Thanks for the response Mike, much appreciated :) It makes me feel more comfortable about using a new project when the contributors are vocal :) I'll prefix this by saying that I'm not a professional developer, but rather a sysadmin who happens to also code tools and products for the company I work at, so forgive me if my thoughts are archaic or err on the side of stability and overly cautious security and predictability :)

On the other hand, in production, for several of our customer apps, 80% of our backend is served using the scaffolds, and for a lot of basic functionality, they serve the purpose (search, sorting, pagination, 

The projects I put together are multi-tennant, in some way shape or form - that could literally be multiple customers or multiple departments that have very strict requirements on what other can and can't see. This makes scaffolds in their current form for me nearly useless as I need to overwrite their behaviour almost across the board. Being able to generate static code would be handy as a basis.

Additionally with my sysadmin hat on I see one 1 major benefit from generating static code that affects everyone - it doesn't change. If your deployment scripts aren't exact, or god forbid there's a manual process, and a new version of sails has different scaffold behaviour, there's the potential for information leak if new method(s) are added, or changed. A sad fact is that mistakes happen, and this is entirely possible.

For instance, right now, if you set ?redirect=http://google.com when hitting /model/create, you'll get redirected to the destination you specify.

I hadn't noticed that feature, but it makes me a little nervous in of itself if you can redirect to arbitrary URLs? Most users know not to put important information into a site that doesn't have the right URL (i.e. from an email). But if a malicious user can get the legitimate site to redirect to their site, and the user doesnt notice the redirect?

- taa

[Mike McNeil]

re: access control/multitenancy

In general, you shouldn't be mixing authentication/access control code into your controllers.  Business logic and auth don't mix, and have very ugly offspring.  In my experience, this is where most leaks occur.

You should control access using policies.  Your policies can be as detailed or as simple as you like:  For instance, we're using blueprints to support a multitenant dropbox-style application by applying an ACL and file upload quotas.  But you can also use policies to apply a "guest" login which does nothing more than attach a tracking cookie for analytics.

Check out this part of the screencast: http://youtu.be/GK-tFvpIR7c?t=8m34s

As long as your policy middleware is written properly, and you use '*' or cover any actions you add to the controller, the logic will be protected. The controller doesn't even have to exist for the policy to apply- that's the benefit of the convention.  

When writing business logic, you shouldn't have to even think about whether the request has been authenticated.   Developers need complete certainty that a user is allowed access to that particular chunk of logic before they ever reach the controller level.  All of that verification lives in policies, and you can talk to as many services, asynchronous processes as you like to get that security requirement satisfied.

re: ?redirect:

The whole idea here is to make it easier for people making forms-- all you have to do is add a hidden "redirect" field with the destination to redirect to for after the form is complete.  Keep in mind, the 302 redirect occurs after the action takes place-- none of the request params are handed to it.

This is an experimental feature, and granted, we want to avoid this.  There's no direct system security risk here, but it does make it easier for people who don't look at the URLs they visit to be misled, so I can see the value of replacing it with something else.

Thanks for your thoughts!  Hope that helps clarify the reasoning behind things, and how we're able to use Sails in production.

Mike

PS: There are some other thoughts around ways to still make it very easy to modify the functionality of scaffolds declaratively without having to write functional code, discussed here, that I'm happy to talk about.  I think we should probably move those sort of roadmap discussions over to the sailsjs-dev group.

[the_angry_angel]

re: access control/multitenancy

In general, you shouldn't be mixing authentication/access control code into your controllers.  Business logic and auth don't mix, and have very ugly offspring.  In my experience, this is where most leaks occur.

Auth I'd agree should be taken care of before controllers. What I'm talking about is authorisation.

Policies don't seem to be cover a simple use case of returning a subset of data. i.e. Take a simple example
Users model
Groups model, which is a nested set
Users can belong to many groups
Users can see all other users, in the same group or child groups

This doesn't seem to be something that I can acheive with a policy - I'd expect to have to write my own controller to emulate the generic scaffold, but with the relevant business logic?