Data validation

94 views
Skip to first unread message

Alan Hoffmeister

unread,
Jan 18, 2013, 6:07:13 AM1/18/13
to nodejs
Hello fellows!

How are you doing data validation like forms and other user inputs? I know that there is some modules to do that but my intention is to do a brainstorm to gather ideas for a new module that I'm developing.

Thanks.

--
Att,
Alan Hoffmeister

Jake Verbaten

unread,
Jan 18, 2013, 10:33:30 PM1/18/13
to nod...@googlegroups.com
I use a module called valid-schema ( https://github.com/Colingo/valid-schema )

It basically turns a JSON schema declaration into a validation function that either returns null or a list of validation errors.

The unique thing about it is that every validator is just an arbitrary function so you effectively build a domain specific validation language out of high order functions.


--
Job Board: http://jobs.nodejs.org/
Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nod...@googlegroups.com
To unsubscribe from this group, send email to
nodejs+un...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en

Glenn Block

unread,
Jan 21, 2013, 12:45:28 PM1/21/13
to nod...@googlegroups.com

Katsumoto

unread,
Jan 21, 2013, 12:52:51 PM1/21/13
to nod...@googlegroups.com
take a look at https://github.com/chriso/node-validator

пятница, 18 января 2013 г., 13:07:13 UTC+2 пользователь Alan Hoffmeister написал:

Alan Hoffmeister

unread,
Jan 21, 2013, 1:11:08 PM1/21/13
to nodejs
Katsumoto, thats a nice well known package, but I think that lacks async support.

--
Att,
Alan Hoffmeister


2013/1/21 Katsumoto <shog...@gmail.com>
--

Jake Verbaten

unread,
Jan 22, 2013, 4:25:43 AM1/22/13
to nod...@googlegroups.com
Your validating logic is only async if it does IO. 

If your validation does IO you fucked up. You don't need async support.

Alan Hoffmeister

unread,
Jan 22, 2013, 6:06:48 AM1/22/13
to nodejs
So how do you validate duplicated username, e-mail, etc..? As far as I could see I need to split the validation process for this, and I think that this is fucking up with the validation.

--
Att,
Alan Hoffmeister


2013/1/22 Jake Verbaten <ray...@gmail.com>

Pedro Teixeira

unread,
Jan 22, 2013, 6:17:44 AM1/22/13
to nod...@googlegroups.com
Duplicate validation should be handled at the persistence layer, not in Node, since you have a concurrency issue there (think of 2 simultaneous requests).

-- 
Pedro

Alan Hoffmeister

unread,
Jan 22, 2013, 6:56:26 AM1/22/13
to nodejs
Well, generaly I don't wait for the db to throw a duplication error to warn the user that he needs to pick up another username. But of course, if the form is 100% valid I still need to take care about his username at the db layer.

Another example: I need to check if a file exists, send a request to a rest server to check the file metadata and convert a markdown field to HTML using the GitHub's API, that would be a hell of a process that I can't encapsulate inside all those validation modules mentioned here without spliting the process a couple of times.

That's the main idea about the validation/sanitization module that I'm building: async from ground up.


--
Att,
Alan Hoffmeister


2013/1/22 Pedro Teixeira <pedro.t...@gmail.com>

Dan Milon

unread,
Jan 22, 2013, 7:08:36 AM1/22/13
to nod...@googlegroups.com, Alan Hoffmeister
comments inline.

On 01/22/2013 01:56 PM, Alan Hoffmeister wrote:
> Well, generaly I don't wait for the db to throw a duplication error to
> warn the user that he needs to pick up another username. But of course,
> if the form is 100% valid I still need to take care about his username
> at the db layer.
>

Then you're doing register wrong.

> Another example: I need to check if a file exists, send a request to a
> rest server to check the file metadata and convert a markdown field to
> HTML using the GitHub's API, that would be a hell of a process that I
> can't encapsulate inside all those validation modules mentioned here
> without spliting the process a couple of times.
>

Be more specific, what do you want to validate against?

If validation depends on external data then pre-fetch them yourself.

eg: When you're handling a sign in request, you query the db for a user
with that username, hash the provided password, and compare with the
hashed password in the user record from the db. And you want to validate
that they are equal. If you were to validate this using a validation
library, you'd fetch the user record from the db, and then use the
validation library. You wouldn't ask the validation library to fetch the
user record from the db.

danmilon.

> That's the main idea about the validation/sanitization module that I'm
> building: async from ground up.
>
>
> --
> Att,
> Alan Hoffmeister
>
>
> 2013/1/22 Pedro Teixeira <pedro.t...@gmail.com
> <mailto:pedro.t...@gmail.com>>
>
> Duplicate validation should be handled at the persistence layer, not
> in Node, since you have a concurrency issue there (think of 2
> simultaneous requests).
>
> --
> Pedro
>
> On Tuesday, January 22, 2013 at 11:06 AM, Alan Hoffmeister wrote:
>
>> So how do you validate duplicated username, e-mail, etc..? As far
>> as I could see I need to split the validation process for this,
>> and I think that this is fucking up with the validation.
>>
>> --
>> Att,
>> Alan Hoffmeister
>>
>>
>> 2013/1/22 Jake Verbaten <ray...@gmail.com <mailto:ray...@gmail.com>>
>>> Your validating logic is only async if it does IO.
>>>
>>> If your validation does IO you fucked up. You don't need async
>>> support.
>>>
>>>
>>> On Mon, Jan 21, 2013 at 10:11 AM, Alan Hoffmeister
>>> <alanhof...@gmail.com <mailto:alanhof...@gmail.com>> wrote:
>>>> Katsumoto, thats a nice well known package, but I think that
>>>> lacks async support.
>>>>
>>>> --
>>>> Att,
>>>> Alan Hoffmeister
>>>>
>>>>
>>>> 2013/1/21 Katsumoto <shog...@gmail.com
>>>> <mailto:shog...@gmail.com>>
>>>>> take a look at https://github.com/chriso/node-validator
>>>>>
>>>>> пятница, 18 января 2013 г., 13:07:13 UTC+2 пользователь Alan
>>>>> Hoffmeister написал:
>>>>>> Hello fellows!
>>>>>>
>>>>>> How are you doing data validation like forms and other user
>>>>>> inputs? I know that there is some modules to do that but my
>>>>>> intention is to do a brainstorm to gather ideas for a new
>>>>>> module that I'm developing.
>>>>>>
>>>>>> Thanks.
>>>>>>
>>>>>> --
>>>>>> Att,
>>>>>> Alan Hoffmeister
>>>>>
>>>>> --
>>>>> Job Board: http://jobs.nodejs.org/
>>>>> Posting guidelines:
>>>>> https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
>>>>> You received this message because you are subscribed to the Google
>>>>> Groups "nodejs" group.
>>>>> To post to this group, send email to nod...@googlegroups.com
>>>>> <mailto:nod...@googlegroups.com>
>>>>> To unsubscribe from this group, send email to
>>>>> nodejs+un...@googlegroups.com
>>>>> <mailto:nodejs%2Bunsu...@googlegroups.com>
>>>>> For more options, visit this group at
>>>>> http://groups.google.com/group/nodejs?hl=en?hl=en
>>>>
>>>> --
>>>> Job Board: http://jobs.nodejs.org/
>>>> Posting guidelines:
>>>> https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
>>>> You received this message because you are subscribed to the Google
>>>> Groups "nodejs" group.
>>>> To post to this group, send email to nod...@googlegroups.com
>>>> <mailto:nod...@googlegroups.com>
>>>> To unsubscribe from this group, send email to
>>>> nodejs+un...@googlegroups.com
>>>> <mailto:nodejs%2Bunsu...@googlegroups.com>
>>>> For more options, visit this group at
>>>> http://groups.google.com/group/nodejs?hl=en?hl=en
>>>
>>> --
>>> Job Board: http://jobs.nodejs.org/
>>> Posting guidelines:
>>> https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
>>> You received this message because you are subscribed to the Google
>>> Groups "nodejs" group.
>>> To post to this group, send email to nod...@googlegroups.com
>>> <mailto:nod...@googlegroups.com>
>>> To unsubscribe from this group, send email to
>>> nodejs+un...@googlegroups.com
>>> <mailto:nodejs%2Bunsu...@googlegroups.com>
>>> For more options, visit this group at
>>> http://groups.google.com/group/nodejs?hl=en?hl=en
>>
>> --
>> Job Board: http://jobs.nodejs.org/
>> Posting guidelines:
>> https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
>> You received this message because you are subscribed to the Google
>> Groups "nodejs" group.
>> To post to this group, send email to nod...@googlegroups.com
>> <mailto:nod...@googlegroups.com>
>> To unsubscribe from this group, send email to
>> nodejs+un...@googlegroups.com
>> <mailto:nodejs+un...@googlegroups.com>
>> For more options, visit this group at
>> http://groups.google.com/group/nodejs?hl=en?hl=en
>
> --
> Job Board: http://jobs.nodejs.org/
> Posting guidelines:
> https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
> You received this message because you are subscribed to the Google
> Groups "nodejs" group.
> To post to this group, send email to nod...@googlegroups.com
> <mailto:nod...@googlegroups.com>
> To unsubscribe from this group, send email to
> nodejs+un...@googlegroups.com
> <mailto:nodejs%2Bunsu...@googlegroups.com>

Dan Milon

unread,
Jan 22, 2013, 7:42:26 AM1/22/13
to nod...@googlegroups.com
It is not about performance, it is about forcefully trying to merge two
completely different things. I might be wrong, but I can't think of a
pretty and useful API for a validation library that would transparently
issue async operations, and then let you get those results if validation
passed. Application logic gets in the way and doesn't allow for much
flexibility IMO.

I'd be interested to see what you come up with.

danmilon.

On 01/22/2013 02:29 PM, Alan Hoffmeister wrote:
> Dan I see the point, but would it be any performance issue accessing
> external data from inside the validation process instead of prefetching it?
>
> --
> Att,
> Alan Hoffmeister
>
>
> 2013/1/22 Dan Milon <danm...@gmail.com <mailto:danm...@gmail.com>>
> > <mailto:pedro.t...@gmail.com <mailto:pedro.t...@gmail.com>>>
> >
> > Duplicate validation should be handled at the persistence
> layer, not
> > in Node, since you have a concurrency issue there (think of 2
> > simultaneous requests).
> >
> > --
> > Pedro
> >
> > On Tuesday, January 22, 2013 at 11:06 AM, Alan Hoffmeister wrote:
> >
> >> So how do you validate duplicated username, e-mail, etc..? As far
> >> as I could see I need to split the validation process for this,
> >> and I think that this is fucking up with the validation.
> >>
> >> --
> >> Att,
> >> Alan Hoffmeister
> >>
> >>
> >> 2013/1/22 Jake Verbaten <ray...@gmail.com
> <mailto:ray...@gmail.com> <mailto:ray...@gmail.com
> <mailto:ray...@gmail.com>>>
> >>> Your validating logic is only async if it does IO.
> >>>
> >>> If your validation does IO you fucked up. You don't need async
> >>> support.
> >>>
> >>>
> >>> On Mon, Jan 21, 2013 at 10:11 AM, Alan Hoffmeister
> >>> <alanhof...@gmail.com
> <mailto:alanhof...@gmail.com> <mailto:alanhof...@gmail.com
> <mailto:alanhof...@gmail.com>>> wrote:
> >>>> Katsumoto, thats a nice well known package, but I think that
> >>>> lacks async support.
> >>>>
> >>>> --
> >>>> Att,
> >>>> Alan Hoffmeister
> >>>>
> >>>>
> >>>> 2013/1/21 Katsumoto <shog...@gmail.com
> <mailto:shog...@gmail.com>
> >>>> <mailto:shog...@gmail.com <mailto:shog...@gmail.com>>>
> >>>>> take a look at https://github.com/chriso/node-validator
> >>>>>
> >>>>> пятница, 18 января 2013 г., 13:07:13 UTC+2 пользователь Alan
> >>>>> Hoffmeister написал:
> >>>>>> Hello fellows!
> >>>>>>
> >>>>>> How are you doing data validation like forms and other user
> >>>>>> inputs? I know that there is some modules to do that but my
> >>>>>> intention is to do a brainstorm to gather ideas for a new
> >>>>>> module that I'm developing.
> >>>>>>
> >>>>>> Thanks.
> >>>>>>
> >>>>>> --
> >>>>>> Att,
> >>>>>> Alan Hoffmeister
> >>>>>
> >>>>> --
> >>>>> Job Board: http://jobs.nodejs.org/
> >>>>> Posting guidelines:
> >>>>>
> https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
> >>>>> You received this message because you are subscribed to
> the Google
> >>>>> Groups "nodejs" group.
> >>>>> To post to this group, send email to
> nod...@googlegroups.com <mailto:nod...@googlegroups.com>
> >>>>> <mailto:nod...@googlegroups.com
> <mailto:nod...@googlegroups.com>>
> >>>>> To unsubscribe from this group, send email to
> >>>>> nodejs+un...@googlegroups.com
> <mailto:nodejs%2Bunsu...@googlegroups.com>
> >>>>> <mailto:nodejs%2Bunsu...@googlegroups.com
> <mailto:nodejs%252Buns...@googlegroups.com>>
> >>>>> For more options, visit this group at
> >>>>> http://groups.google.com/group/nodejs?hl=en?hl=en
> >>>>
> >>>> --
> >>>> Job Board: http://jobs.nodejs.org/
> >>>> Posting guidelines:
> >>>>
> https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
> >>>> You received this message because you are subscribed to the
> Google
> >>>> Groups "nodejs" group.
> >>>> To post to this group, send email to
> nod...@googlegroups.com <mailto:nod...@googlegroups.com>
> >>>> <mailto:nod...@googlegroups.com
> <mailto:nod...@googlegroups.com>>
> >>>> To unsubscribe from this group, send email to
> >>>> nodejs+un...@googlegroups.com
> <mailto:nodejs%2Bunsu...@googlegroups.com>
> >>>> <mailto:nodejs%2Bunsu...@googlegroups.com
> <mailto:nodejs%252Buns...@googlegroups.com>>
> >>>> For more options, visit this group at
> >>>> http://groups.google.com/group/nodejs?hl=en?hl=en
> >>>
> >>> --
> >>> Job Board: http://jobs.nodejs.org/
> >>> Posting guidelines:
> >>>
> https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
> >>> You received this message because you are subscribed to the
> Google
> >>> Groups "nodejs" group.
> >>> To post to this group, send email to nod...@googlegroups.com
> <mailto:nod...@googlegroups.com>
> >>> <mailto:nod...@googlegroups.com
> <mailto:nod...@googlegroups.com>>
> >>> To unsubscribe from this group, send email to
> >>> nodejs+un...@googlegroups.com
> <mailto:nodejs%2Bunsu...@googlegroups.com>
> >>> <mailto:nodejs%2Bunsu...@googlegroups.com
> <mailto:nodejs%252Buns...@googlegroups.com>>
> >>> For more options, visit this group at
> >>> http://groups.google.com/group/nodejs?hl=en?hl=en
> >>
> >> --
> >> Job Board: http://jobs.nodejs.org/
> >> Posting guidelines:
> >>
> https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
> >> You received this message because you are subscribed to the
> Google
> >> Groups "nodejs" group.
> >> To post to this group, send email to nod...@googlegroups.com
> <mailto:nod...@googlegroups.com>
> >> <mailto:nod...@googlegroups.com <mailto:nod...@googlegroups.com>>
> >> To unsubscribe from this group, send email to
> >> nodejs+un...@googlegroups.com
> <mailto:nodejs%2Bunsu...@googlegroups.com>
> >> <mailto:nodejs+un...@googlegroups.com
> <mailto:nodejs%2Bunsu...@googlegroups.com>>
> >> For more options, visit this group at
> >> http://groups.google.com/group/nodejs?hl=en?hl=en
> >
> > --
> > Job Board: http://jobs.nodejs.org/
> > Posting guidelines:
> >
> https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
> > You received this message because you are subscribed to the Google
> > Groups "nodejs" group.
> > To post to this group, send email to nod...@googlegroups.com
> <mailto:nod...@googlegroups.com>
> > <mailto:nod...@googlegroups.com <mailto:nod...@googlegroups.com>>
> > To unsubscribe from this group, send email to
> > nodejs+un...@googlegroups.com
> <mailto:nodejs%2Bunsu...@googlegroups.com>
> > <mailto:nodejs%2Bunsu...@googlegroups.com
> <mailto:nodejs%252Buns...@googlegroups.com>>

Jake Verbaten

unread,
Jan 23, 2013, 3:38:22 AM1/23/13
to nod...@googlegroups.com
validation and ensuring global constraints are met at the database layer are two completely seperate things.

One validates that a piece of data is valid and sanitized by some local constraint. The other checks global constraints across your entire system.

Why would you want to split these up? Because validation without global constraint checks is a very simple pure function, take input, return boolean or errors.

greelgorke

unread,
Jan 23, 2013, 3:59:30 AM1/23/13
to nod...@googlegroups.com
just 2 cents:

guys, please don't mix up validity and conformity to business rules. sometimes it's hard to not to mix them up, but it is important doe distinguish. That a form field should be an email address is the first one, that this address should be unique in the system, is the other thing. Validation proves the format and type of the data, Businessrules proves integrity of the data model. first is made by simple sync functions, the second one needs interaction with the business model layer including db-access. 
Reply all
Reply to author
Forward
0 new messages