password storage with per-user salt

[Dave]

I have just started using web2py but already, I'm quite impressed. In
the past couple days I've already rolled out an entire site rewrite
and I'm working on my second project.

The project I'm working on right now is currently in PHP. I was in
the process of converting it to a Java / Spring MVC project when I
discovered web2py and decided that'd be a much easier, simpler and
quicker way to roll the app.

So, let me get to my point.. The current application utilizes the php
sha1() function with a per-user salt stored in the database. The salt
is randomly generated each time the password is changed. This is
similar to the default configuration on most linux boxes.

I need to make some changes to the Auth class to support the per-
record password salt instead of application-wide salt. Does it make
sense for me to provide my edits as part of the project, in case
someone else thinks the functionality is useful? I plan on basically
checking to see if there is a 'salt' field in the user auth table, and
if so, append that to the plain text password before passing it to the
appropriate hashlib function.

Thoughts?

[Massimo Di Pierro]

This will be useful put presents a technical difficulty because of the
way CRYPT works. CRYPT is the validator that check is a password is
valid, and it does not know what is stored in db therefore it does not
know the salt. Anyway, let me know if you have a suggestion.

Massimo

[pbreit]

It might not be a bad idea to improve password handling at this time. I think the biggest problem is that password hases are not currently salted. The hmac_hash function appears to take a salt but I didn't see any evidence that is ever actually used.

The Django model seems sufficient:

https://docs.djangoproject.com/en/dev/topics/auth/#passwords

I think this is the code:

https://github.com/django/django/blob/master/django/contrib/auth/utils.py

Passwords would be stored like this:

sha1$a1976$a36cc8cbf81742a8fb52e221aaeab48ed7f58ab4

(ie, algo$salt$hash)

The hash algo is stored with the password. This makes it easier to switch algos in the future. The salt is also stored with the password which many people mistakenly think is unsecure. Also note that the salt is just a simple random string and does not have to be particularly long to be effective.

If we did implement this approach, the next question is, could we also implement a scheme whereby if the algo is changed, when someone goes to change their password, the system can confirm that the old password is provided correctly and then store the new password under the new scheme?

[João Gulineli]

I had the same problem as you, I think. We wrote a cutomize login method , attached, to check if the entered password was the same that was stored in the database, we use the http://packages.python.org/passlib/lib/passlib.hash.sha512_crypt.html.
If you wish you could write a new validator to encrypt using the library cited.

João Henrique Gulineli Fachini;
Eng° Mecânico;
Ilha Solteira - SP;
Cel: (18) 8809 1391
tel LABSIN: (18) 3743 1000 ramal 1343

2011/9/21 pbreit <pbreit...@gmail.com>

[Ross Peoples]

If we are talking about improving security a bit, I would recommend that we also hash passwords in the browser before sending them to the server. I just read about this here:

http://dustwell.com/how-to-handle-passwords.html

Basically, when a user logs in, registers, or otherwise enters a password, a JavaScript method hashes the password with a salt, and submits that as the password instead of sending plain text to the server.

[Anthony]

+1

[Michele Comitini]

Plain WRONG.  Even the guy knows it: " Note that a hacker could still sniff the hashed password going over the network, and use that hash later to send to the server and impersonate you. But at least the hacker can't use your real password for other purposes."

You can try with two salts.  One of them must be one time random for each auth request. Then you apply the hashing twice with the 2 salts on the client and once on the server with the one time only.

mic

[Anthony]

Yes, I think the salting and hashing we're discussing is meant to protect the passwords once on the server (i.e., if the server/database is compromised). We'd need additional protection to protect them in transit. Another option would be login over SSL, no?

Anthony

[David J]

Why double salt. That's the point of SSL. we should only be concerned with application level details not transport

[pbreit]

Sending passwords over SSL should be sufficient in most cases. Hashing the password on the client side is slightly better and would provide better protection for those who use SSL (wouldn't want to encourage that, though).

I think the only immediate need, though, is support for per-password salting.

[pbreit]

That should be "for those who do not use ssl".

[Michele Comitini]

Doing as in the article above, if x compromises the server x can login since the hashed password *are the secret* . To do things right the hash of a secret  arriving from the client must be computed every time on the server before comparison.

SSL hides the secret so it's a good choice.

mic

2011/9/21 Anthony <abas...@gmail.com>:

[pbreit]

I think if you were going to do client side hashing you would send out a unique "secret" on each request. I don't think this is necessary at this time. At least SSL gives those who want to be secure the option.

[Dave]

Well clearly I've sparked plenty of discussion. I am working on this
to fit my app need. Once I have a working model that doesn't break
other applications that use the default hashing and CRYPT functions
I'll post my work. As others have commented, the typical way for
storing the password would be {algorithm}$salt$hash. I have no
problem with that. The previous developer of the app I am working on
just chose to store the salt in a separate field in the table. It's
fairly trivial for me to convert the 1500 or so user password strings.

Stay tuned and I'll post something later this week or next.

On Sep 20, 11:36 pm, Massimo Di Pierro <massimo.dipie...@gmail.com>
wrote:

[Dave]

I've made this work. I still need a little bit of time to make it
backward compatible with non-salted passwords though.

First things first, I had to disable the CRYPT validator. Although it
makes handling passwords easier, encrypting them at the validator
level really limits a lot of account enforcement options.

Next I created an extra utility library with two methods:
crypt(password='', algorithm='sha256', salt='') and
check_password(plain_password='', encrypted_password=''). The crypt
function is pretty simple. It returns a hexdigest given a specified
algorithm, string and salt. If no salt is specified, a 16 character
salt is randomly generated.

Finally I had to do some tweaking in the Auth class. First I changed
the login method to check the password with my check_password instead
of a simple string compare. Next I had to add a "onvalidate" method
to the 'register' and 'change_password' methods to encrypt the
password. The change_password was actually a little more involved
because I had to customize the old_password validator to use my
check_password method.

I'm not quite ready to share the diff files because I want to re-work
it so that it's reverse compatible with auth_user data created with
the default settings.

On Sep 21, 1:05 pm, Dave <dave.st...@gmail.com> wrote:
> Well  clearly I've sparked plenty of discussion.  I am working on this
> to fit my app need.  Once I have a working model that doesn't break
> other applications that use the default hashing and CRYPT functions
> I'll post my work.  As others have commented, the typical way for
> storing the password would be {algorithm}$salt$hash.  I have no
> problem with that.  The previous developer of the app I am working on

> just chose to store thesaltin a separate field in the table.  It's

> fairly trivial for me to convert the 1500 or so user password strings.
>
> Stay tuned and I'll post something later this week or next.
>
> On Sep 20, 11:36 pm, Massimo Di Pierro <massimo.dipie...@gmail.com>
> wrote:
>
>
>
>
>
>
>
> > This will be useful put presents a technical difficulty because of the
> > way CRYPT works. CRYPT is the validator that check is a password is
> > valid, and it does not know what is stored in db therefore it does not

> > know thesalt. Anyway, let me know if you have a suggestion.

>
> > Massimo
>
> > On Sep 20, 9:25 pm, Dave <dave.st...@gmail.com> wrote:
>
> > > I have just started using web2py but already, I'm quite impressed.  In
> > > the past couple days I've already rolled out an entire site rewrite
> > > and I'm working on my second project.
>
> > > The project I'm working on right now is currently in PHP.  I was in
> > > the process of converting it to a Java / Spring MVC project when I
> > > discovered web2py and decided that'd be a much easier, simpler and
> > > quicker way to roll the app.
>
> > > So, let me get to my point..  The current application utilizes the php
> > > sha1() function with aper-usersaltstored in the database.  Thesalt
> > > is randomly generated each time the password is changed.  This is
> > > similar to the default configuration on most linux boxes.
>
> > > I need to make some changes to the Auth class to support the per-

> > > record passwordsaltinstead of application-widesalt.  Does it make

[Massimo Di Pierro]

I was thinking about this... how about editing the validate function
in dal.py and allow it to pass the current record to the validators?
We cannot change the APIs but a validator is an object so we can just
pass the current record by attaching it as an attribute to the
validator object. CRYPT could check if the current record exists and
use it to extract the salt. In this case you woud change only the
validate function in dal.py and CRYPT.

[Dave]

That may be an even better way to accomplish this task. I actually
considered using some sort of self-reference first, but didn't find it
there.





On Sep 28, 9:00 pm, Massimo Di Pierro <massimo.dipie...@gmail.com>

[Massimo Di Pierro]

Anyway, I have not looked into this with sufficient detail to know
what is cleaner (or dirtier).
I we pass an optional record argument to validate we will need to
change sqlhtml as well.

Massimo

[Dave]

I struggled with that a little bit as I was following the application
flow particularly in the login method. All the other methods (well,
except the reset request method) are after login and there is already
a user object. That's why it was so easy for me to just put
onvalidate modifications in the change pass and register methods.

The flow tripped me up a bit on the login side. Since we haven't
tried to select the user record from the database at the time of
validation, we don't know if the user exists at all. The user lookup
happens in the model code a bit farther down... well, actually pretty
much immediately after the form is accepted post-validation.

However.... If we could pass the entire form object to the validator,
the validator could then fetch the user row from the database and
perform the necessary initialization of salt & algorithm for this
user's stored password. Like you, I'm not sure if that's cleaner or
not though...

Something I was aiming for when I started looking at this was to be
able to easily change the hash algorithm used without requiring all
users to reset their passwords. I made my edits with this in mind.
When doing the check password routine I extract the algorithm and salt
from the stored password string. That means whatever the "preference"
is for hashing, the stored password hash algorithm is honored. When a
password is encrypted, as in register or change_password, the
preferred hashing preferences are honored.

That mindset is important to me for two reasons. First, my day-job is
information security. Several times in the recent past I've had to
tell people "don't use ____ anymore because it's weak". The
application I'm working on now has about 1300 users and the passwords
were hashed with sha1. My preference is nothing less than sha256.
This approach will allow users to authenticate with what's in place,
but when they change their password it gets stored with the new hash
algo.



On Sep 28, 9:46 pm, Massimo Di Pierro <massimo.dipie...@gmail.com>

[Massimo Di Pierro]

Perhaps the safer bet is to continue along your original idea of
tweaking auth.
In general it may not be a good idea to have validators access the
record before validation, it could cause DoS problems.

Massimo

[Dave]

Hm. yeah, good point.

Let me do some cleaning up on what I have... and make sure it will be
backward compatible with people doing requires=CRYPT on the password
field. Then I'll send you a diff and see what you think.



On Sep 29, 1:38 am, Massimo Di Pierro <massimo.dipie...@gmail.com>

[Dave]

I have the patch. I wasn't sure where to put the file, so I emailed
it directly to you Massimo. The only thing that seems to be missing
is updating the login_bare method to use the new hashing.

The code is backward compatible with pre-exiting non-salted passwords.

[Eli Collins]

I'm coming slightly late to this thread, but wanted to add a note...

 

If we did implement this approach, the next question is, could we also implement a scheme whereby if the algo is changed, when someone goes to change their password, the system can confirm that the old password is provided correctly and then store the new password under the new scheme?

If you do decide to use the passlib library (per João Gulineli's post), it also offers a CryptContext class which can configurably let you support multiple hashes at once; but more importantly (with regards the quoted question), it also lets you mark certain hash schemes as "deprecated", and offers a method verify_and_update() method which takes care of that exact operation you described: verify password, and re-encode if previously stored using a deprecated hash (the function docs link to a usage example).

- Eli Collins

[Massimo Di Pierro]

I have a pending patch for per-user last. it is in my todo list....

On Nov 1, 10:32 pm, Eli Collins <e...@astllc.org> wrote:
> I'm coming slightly late to this thread, but wanted to add a note...
>
> > If we did implement this approach, the next question is, could we also
> > implement a scheme whereby if the algo is changed, when someone goes to
> > change their password, the system can confirm that the old password is
> > provided correctly and then store the new password under the new scheme?
>

> If you do decide to use the passlib <http://packages.python.org/passlib>library (per João
> Gulineli's post <http://../d/msg/web2py/WEiNGgDya58/zoKCedgl-qwJ>), it also
> offers a CryptContext<http://packages.python.org/passlib/lib/passlib.context-usage.html#bas...>class which can configurably let you support multiple hashes at once; but

> more importantly (with regards the quoted question), it also lets you mark
> certain hash schemes as "deprecated", and offers a method

> verify_and_update()<http://packages.python.org/passlib/lib/passlib.context-interface.html...>method which takes care of that exact operation you described: verify