Generating Salt for encryption using golang

[Sankar P]

Hi,

This might sound more of a security question than a golang question.

I have a web application where I need to store passwords of users.
Instead of saving the passwords, I am saving the SHA 512 hash of the
password. However, instead of generating the SHA hash for the password
alone, I am thinking of generating it for salt+password.

Are there best-known method(s) or strategies to generate the salt for
getting password digests with golang ?

Thanks.

--
Sankar P
http://psankar.blogspot.com
P.S: Please CC me in replies

[Archos]

Use bCrypt

http://go.pkgdoc.org/code.google.com/p/go.crypto/bcrypt

> Sankar Phttp://psankar.blogspot.com

[Sankar]

bcrypt is not listed in the go1 godoc pages that I have. I believe it may not be supported by 1.0 

I need to rely on the stable version.

[Maxim Khitrov]

An explanation of why use this over SHA-256 or SHA-512 would have been nice.

To answer the original question, a salt can be any random value. You
can use crypt/rand to generate 8 or more random bytes and use that as
the salt.

To improve the security of your system, you should run the hash
algorithm in a loop so that the final output takes a non-trivial
amount of time to calculate (e.g. 1 second). This isn't always
practical in real-world systems, but even a 100ms loop would be better
than a single hash. This way, an attacker that gets a hold of your
database will have a much harder time trying to brute-force the
passwords.

This is where bcrypt is supposed to be advantageous, but I don't have
any first-hand experience with it:

https://en.wikipedia.org/wiki/Bcrypt

- Max

[roger peppe]

On 29 May 2012 13:04, Sankar P <sankar.c...@gmail.com> wrote:
> Hi,
>
> This might sound more of a security question than a golang question.
>
> I have a web application where I need to store passwords of users.
> Instead of saving the passwords, I am saving the SHA 512 hash of the
> password. However, instead of generating the SHA hash for the password
> alone, I am thinking of generating it for salt+password.
>
> Are there best-known method(s) or strategies to generate the salt for
> getting password digests with golang ?

i'd probably crypto/rand.

something like this would probably do the trick (although you might consider
encoding the salt length in the result too):


package main

import (
"bytes"
"crypto/rand"
"crypto/sha1"
"fmt"
"io"
)

const SaltSize = 16

func saltedHash(secret []byte) []byte {
buf := make([]byte, SaltSize, SaltSize+sha1.Size)
_, err := io.ReadFull(rand.Reader, buf)
if err != nil {
panic(fmt.Errorf("random read failed: %v", err))
}
h := sha1.New()
h.Write(buf)
h.Write(secret)
return h.Sum(buf)
}

func match(data, secret []byte) bool {
if len(data) != SaltSize+sha1.Size {
panic("wrong length of data")
}
h := sha1.New()
h.Write(data[:SaltSize])
h.Write(secret)
return bytes.Equal(h.Sum(nil), data[SaltSize:])
}

func main() {
h := saltedHash([]byte("hello"))
fmt.Printf("%v\n", match(h, []byte("wrong")))
fmt.Printf("%v\n", match(h, []byte("hello")))
}

[Sankar P]

On Tue, May 29, 2012 at 6:24 PM, Peter Thrun <peter...@ymail.com> wrote:
>> I have a web application where I need to store passwords of users.
>> Instead of saving the passwords, I am saving the SHA 512 hash of the
>> password.
>
>

> A SHA hash of salt+password is not a safe way to store a password. Use
> bcrypt as suggested earlier.
>
> The bcrypt package
> at http://go.pkgdoc.org/code.google.com/p/go.crypto/bcrypt works with Go 1,
> is stable and is supported by the Go team.
>

I did a godoc -http=:1818 in go1 installed in a mac machine, but
couldn't find anything related to bcrypt. Should I file a bug ? Are
you sure it is supported by the Go 1 officially.

[roger peppe]

On 29 May 2012 14:16, Peter Thrun <peter...@ymail.com> wrote:
> Oops, I gave the documentation link for the package.  You can install the
> package using the command:
>
> go get code.google.com/p/go.crypto/bcryp
>
> After installing the package, your local godoc will show doc for the
> package.

still not quite right :-)

to be honest, i think i prefer giving the doc link, as the first
thing i always do is to look at the docs (and the doc page shows
the import path).

[minux]

On Tue, May 29, 2012 at 9:18 PM, Sankar P <sankar.c...@gmail.com> wrote:

> A SHA hash of salt+password is not a safe way to store a password. Use
> bcrypt as suggested earlier.
>
> The bcrypt package
> at http://go.pkgdoc.org/code.google.com/p/go.crypto/bcrypt works with Go 1,
> is stable and is supported by the Go team.
>

I did a godoc -http=:1818 in go1 installed in a mac machine, but
couldn't find anything related to bcrypt. Should I file a bug ? Are
you sure it is supported by the Go 1 officially.

Yes. It's supported by the Go team and works with Go 1 officially.

It is in a subrepository of the official Go project; the reason it is not

in the main repository is that it is not mature enough, and can't yet

provide a guaranteed compatible API like what is provided for Go 1.

But it will surely work with Go 1, in fact, it is tested with every commit

of Go by the builders.

[roger peppe]

On 29 May 2012 13:54, Peter Thrun <peter...@ymail.com> wrote:
>> I have a web application where I need to store passwords of users.
>> Instead of saving the passwords, I am saving the SHA 512 hash of the
>> password.
>
>

> A SHA hash of salt+password is not a safe way to store a password. Use
> bcrypt as suggested earlier.
>
> The bcrypt package
> at http://go.pkgdoc.org/code.google.com/p/go.crypto/bcrypt works with Go 1,
> is stable and is supported by the Go team.

+1

[Patrick Mylund Nielsen]

Unfortunately it's a very widespread misconception that MD5, SHA, et al, by themselves are suitable for password "storage" because they are "secure hashes". Their use in e.g. cryptographic HMACs is very different (speed is a good thing, for example.) In reality they provide very poor password digests: they are easily recognizable and very quick to run.

bcrypt is generally accepted as being the strongest password storage scheme that is easy to deploy/has any libraries. If you want to be really future-proof, you could look into something like 'scrypt' by Colin Percival (although this has been used significantly less, and thus has a higher chance of having some as-of-yet undiscovered weakness.) bcrypt is the default crypt scheme in OpenBSD, a BSD variant that's notorious for being obsessed with security.

You don't have to use bcrypt if you want to follow standards, though, and even bcrypt hasn't been scrutinized enough that it's safe to call it a silver bullet. The adaptive key derivation function (what these are called) used in government computers, and the one endorsed by NIST, is PBKDF2 (short for password-based key derivation function--and it's in go.crypto as well). It's a container for any hash function, e.g. SHA-256, SHA-512, that does what others have mentioned early--supplying a salt, iterating the hash function so it takes a significant amount of time to generate a single digest, etc. It's older than bcrypt, and it's easier to bruteforce (in a parallelized manner, especially), but it's still a million times better than a regular digest with no salt or stretching.

Whatever you choose, be careful about using just a hash function over an adaptive key derivation scheme. It is notoriously hard to get right (iterating a hash function tends to lose you a fairly significant amount of entropy, for instance) -- and, well, it's just ridiculously easy to use bcrypt in go.crypt. And PBKDF2 just takes a hash.Hash!

Here's the reason people always say "Use bcrypt.": http://codahale.com/how-to-safely-store-a-password/ :)

On Tue, May 29, 2012 at 2:54 PM, Peter Thrun <peter...@ymail.com> wrote:

I have a web application where I need to store passwords of users.
Instead of saving the passwords, I am saving the SHA 512 hash of the
password. 

[Patrick Mylund Nielsen]

Here are the papers:

bcrypt:  http://static.usenix.org/events/usenix99/provos/provos.pdf 

scrypt: http://www.tarsnap.com/scrypt/scrypt.pdf

PBKDF2:  http://csrc.nist.gov/publications/nistpubs/800-132/nist-sp800-132.pdf 

[Sankar P]

Thank you everyone for the excellent comments and the links.

On Tue, May 29, 2012 at 7:44 PM, Patrick Mylund Nielsen

[Patrick Mylund Nielsen]

I should clarify: NIST's recommendation is in regard to deriving a key to use for subsequent en-/decryption of protected information, not specifically to storing and using the digest for authentication purposes.

Conversely, bcrypt was designed specifically for use in password authentication.

[Patrick Mylund Nielsen]

Glad it was useful!

Btw. this table from the scrypt paper really speaks loads:  http://i.imgur.com/iUpUk.png 

[Guillermo Estrada]

As everyone suggested BCrypt is the best way of storing passwords. As it is very CPU intensive (as it is an adaptative hash) and "slow" compared to SHA, it is harder to break by brute force attacks.

(BCrypt already implements a salting in its algorithm to prevent this)
As the Hash + Salt combo, the Salt can be any random string, and you store it next to your hash, in another field or such. It does not gives more security to the hash, it just prevents some kind of cryptographic attacks (like the Rainbow table attack) to be as effective.

http://en.wikipedia.org/wiki/Rainbow_table

The reasoning behind the salt is that if I have 1 million pre computed hashes of commonly used passwords and I get access to the hashes, I can just compare them. With a random salt, the only way to check is to recalculate each hash and its pretty costly.

[Kyle Lemons]

I'll point out that you can also use hmac (supported in the standard library).  The benefit you get with bcrypt (at the expense of the computation time) or salted hash in general is that an attacker who obtains your database has cost-sharing on his side and can attack all passwords at once.

[Dmitry Chestnykh]

On Tuesday, May 29, 2012 2:28:19 PM UTC+2, Sankar wrote:

bcrypt is not listed in the go1 godoc pages that I have. I believe it may not be supported by 1.0 

I need to rely on the stable version.

$ go get code.google.com/p/go.crypto/bcrypt

Alternatively,

$ go get code.google.com/p/go.crypto/pbkdf2

Alternatively,

$ go get github.com/dchest/scrypt

but please don't ever store passwords hashed with SHA/MD5/whatever (salted or not). If you do, that is, decide to invent your own KDF, at least iterate your fast hash a few *thousand* times.

[David Anderson]

Also, don't invent your own KDF. Designing them is not as trivial as "md5() 1000 times".

[Patrick Mylund Nielsen]

This is actually a great idea to do as a complement to something like bcrypt if you want to be really secure. You generate a large nonce, or pepper, to use in HMAC(userpassword, nonce), and you store the nonce (NOT the HMAC digest) on the disk and bcrypt(hmacdigest, gensalt()) in the database. Even if your database is compromised, or bcrypt is broken somehow, people still won't be able to do anything with your digests. This is what e.g. Mozilla does:   https://wiki.mozilla.org/WebAppSec/Secure_Coding_Guidelines#Password_Storage /  https://github.com/fwenzel/django-sha2

I would not recommend just a HMAC. It's better than a single digest, but it's still way too fast (with MD5, or SHA-*) for something like passwords at rest. Salting really doesn't do all that much by itself.

I really don't think a login form taking a little longer to process is going to be the bottleneck in one's application, and even if it is, just reduce the work factor. bcrypt with a work factor of just one or two is still a thousand times better than MD5, or SHA, or HMAC-SHA.

[Patrick Mylund Nielsen]

Here is a helper library/sample implementation: https://github.com/pmylund/go-hmaccrypt

[Archos]

On May 30, 8:34 am, Patrick Mylund Nielsen <patr...@patrickmylund.com>
wrote:

> This is actually a great idea to do as a complement to something like
> bcrypt if you want to be really secure. You generate a large nonce, or
> pepper, to use in HMAC(userpassword, nonce), and you store the nonce (NOT
> the HMAC digest) on the disk and bcrypt(hmacdigest, gensalt()) in the
> database. Even if your database is compromised, or bcrypt is broken
> somehow, people still won't be able to do anything with your digests. This

> is what e.g. Mozilla does:https://wiki.mozilla.org/WebAppSec/Secure_Coding_Guidelines#Password_.../https://github.com/fwenzel/django-sha2
To say that "even if your database is compromised, or bcrypt is broken

somehow, people still won't be able to do anything with your digests."

gives a false sensation of security to people without basic knowledge
in stuff related to crypto. Why do you think that it could avoid
cracking by brute force?

In addition it's being added an unnecessary point of failure in your
system since if you loose the nonce then your users won't login in
your system.

[Patrick Mylund Nielsen]

If your DB digests are compromised by an SQL injection, and the attacker does not have the HMAC nonce, there is no way that an attacker can determine what a user's password is, given any amount of time. At best, they can figure out what the HMAC digest is. Use a pepper with 256 bits of entropy or more, and you're essentially guaranteed that an attacker can never guess a user's password from information retrieved using an SQL injection. If they retrieve both the pepper and the bcrypt digests, they still have to brute force both the bcrypt digest and e.g. sha512(password).

You can just use the same pepper. It doesn't even need to be in a file--you could just include it as a variable in your application. After a compromise, you change the pepper. I don't think this is a point of failure at all.

[Patrick Mylund Nielsen]

Keep in mind that it's much, much harder to brute force bcrypt(512-bit hmac digest) than bcrypt("batman").

Also, if an attacker changes the digest to a manually generated bcrypt digest using an SQL injection, he won't be able to log in, because he doesn't know what the pepper is.

[Archos]

So you are using that complex method (using HMAC) with a point of
failure to get a value of 256 bits, which is a pre-defined security
level for data protection lifetimes by the NIST.
Ok. There are another ways to get a value of 256 bits (or 128, or
192), but with that PoF.

When you are going to cracking passwords hashed using bcrypt, you can
use rainbow tables for values of n bits. I know that it's expensive
computational but you have to see this point because it doesn't matter
how you are generated the value passed to bcrypt; you are going to do
cracking by brute force.

In addition, today you can get easily a great power computational
using simply several GPUs in your system. And you should have in mind
that the list with the requirements and recommendations regarding
symmetric encryption algorithms according to [NIST][] was done in
2005, when there were not GPUs so powerful.

[nist]: http://www.secg.org/collateral/proposal-for-sec1v2.pdf

On May 30, 12:06 pm, Patrick Mylund Nielsen

[Archos]

On May 30, 12:26 pm, Archos <raul....@sent.com> wrote:
> So you are using that complex method (using HMAC) with a point of
> failure to get a value of 256 bits, which is a pre-defined security
> level for data protection lifetimes by the NIST.
> Ok. There are another ways to get a value of 256 bits (or 128, or
> 192), but with that PoF.

I mean *without* that PoF.

[Patrick Mylund Nielsen]

You are still missing the point. The pepper in this example, and just
the pepper, has 256 bits of entropy. It makes brute forcing
bcrypt("a") without the pepper completely infeasible. (You cannot
precompute a "rainbow table" for bcrypt that encompasses values with
such high entropy--and you'll still have to include every possible
salt in your computation.) The number of bits of security of the whole
system still depends on the entropy of the user's password.

It is only a "point of failure" if you lose your source code, and it
is not a complex method. It is very, very easy:

pepper := []byte("NcJZ28VfuZl@i0cWb2j7O1qWc1a#I1B74MrP#qptPvgdT#BW!QnfJU7kQ$vCs7Mo")
crypt := hmaccrypt.New(sha512.New, pepper)

password := []byte("f00b4r!")
digest, err := crypt.Bcrypt(password, 10)
if err != nil {
...
}
// save the bcrypt digest in the database

if err := c.BcryptCompare(digest, password); err == nil {
// the password is a match
...

[Patrick Mylund Nielsen]

> they still have to brute force both the bcrypt digest and e.g. sha512(password).

Clarification: They'd have to brute force bcrypt(SHA512(guess))

[Dmitry Chestnykh]

Patrick is correct that this scheme will prevent one specific type of attack under one specific type of deployment: dictionary attack with the assumption that crackers steal your passwords database, but not your HMAC key ("pepper"). The scheme is pretty much useless if you store the database and HMAC key in the same storage, e.g. both in SQL database, or both on a filesystem (this includes the database on the same server).

I think this scheme was devised under the assumption that SQL injection is the most common type of stealing a passwords database, so the added complexity is worth a try.

Archos is correct that the added complexity of this scheme may not worth it. 

[Patrick Mylund Nielsen]

Thanks. Just nitpicking, but all users usually don't have read rights
on e.g. MySQL or PostgreSQL data files.

I'm not saying it doesn't add any complexity, I just don't think it's
a significant amount of complexity. But indeed, this is subjective.

SQL injections are by far the most common method used to obtain
password digests, but if you use exp/sql / prepared
statements/parameter binding in Go, you're arguably better off than
e.g. 90% of PHP applications to begin with. So, for Go, this method is
less important, but not in any way useless.

[Patrick Mylund Nielsen]

And -- just nitpicking again -- since bcrypt has been tested
significantly less than SHA, this method protects you against any kind
of vulnerability in bcrypt that might leak information about the
password, even if the attacker gets his hands on both the digests in
the database and the HMAC key. An attacker can do significantly less
with a little information about the HMAC digest.

Although it's very unlikely, if bcrypt is completely broken, the
attacker still has to bruteforce a SHA-512 digest of the password.
Unfortunately this would still be more secure than many web
applications :-(

[Archos]

On May 30, 12:47 pm, Patrick Mylund Nielsen

<patr...@patrickmylund.com> wrote:
> You are still missing the point. The pepper in this example, and just
> the pepper, has 256 bits of entropy. It makes brute forcing
> bcrypt("a") without the pepper completely infeasible. (You cannot
> precompute a "rainbow table" for bcrypt that encompasses values with
> such high entropy--and you'll still have to include every possible
> salt in your computation.) The number of bits of security of the whole
> system still depends on the entropy of the user's password.
>
> It is only a "point of failure" if you lose your source code, and it
> is not a complex method. It is very, very easy:

It's complex when it's used a method that adds an unnecessary point of
failure, mainly because there are another ways to get the same result,
which would be get a key of strength-256 bits.

Here you have a simple way to get the same result, but without PoF:

+ take the password of the user, "pass"
+ use PKBF2 to get the password derive of "pass" with the strength-
bits that you desire, "derive"
+ and finally, use bcrypt with the value of "derive".

[Patrick Mylund Nielsen]

There is no separate salt and pepper. This, not key strength, was the point.

[Archos]

The function of a salt is avoid the use of rainbow tables; it should
be random and different for each password but it is not necessary to
be hidden.
bcrypt already adds automatically a different random salt to each
hash.

Respect to the pepper, you are not going increase the strength of your
encryption by adding extra secrets, independently of if you're adding
a point of failure in your system with that system.

On May 30, 4:36 pm, Patrick Mylund Nielsen <patr...@patrickmylund.com>
wrote:

[Patrick Mylund Nielsen]

Ok. I'm going to assume you are trolling.

[Archos]

Ok. I'm going to assume you are without arguments.

http://stackoverflow.com/questions/420843/how-does-password-salt-help-against-a-rainbow-table-attack
http://en.wikipedia.org/wiki/Bcrypt

On May 30, 4:58 pm, Patrick Mylund Nielsen <patr...@patrickmylund.com>
wrote:

[Patrick Mylund Nielsen]

I don't understand why you keep bringing up rainbow tables.

https://wiki.mozilla.org/WebAppSec/Secure_Coding_Guidelines#Password_Storage

If you are not trolling, please take a look at that page and this
thread (from my reply to Kyle.) If you are, oh well. This will be my
last reply :)

[Archos]

On May 30, 5:10 pm, Patrick Mylund Nielsen <patr...@patrickmylund.com>
wrote:

> I don't understand why you keep bringing up rainbow tables.

I bring up about rainbow tables because you said your system has a
separate salt which is unnecessary since bcrypt adds it automatically.

> https://wiki.mozilla.org/WebAppSec/Secure_Coding_Guidelines#Password_...

>
> If you are not trolling, please take a look at that page and this
> thread (from my reply to Kyle.)

Why do you think that what it's said in that page is better than said
by an expert in crypto. like Bruce Schneider.

I recommend this book of that author: Practical Cryptography

http://www.amazon.com/dp/0471223573

[Daniel Theophanes]

Security is great; we should all use secure methods.

SQL injection should NEVER happen, it is so incredibly easy to prevent and easy to check for.

http://xkcd.com/538/

Go has nice hash functions and crypto functions. Some additional crypto functions are in the go. crypto subrepo.

...

Security is important. But data breaches, the ultimate goal security prevents, don't tend to come through these routes. Suggest discussions by example like:

http://its.ucsc.edu/security/breaches.html

for starters. For example, SSN numbers and other sensitive info that can't be just put through a one way hash. That kind of information leaking can be much more serious and harder to contain.

Pointers to bcrypt and cracking time is good. But, I might suggest auditing (how do you know if you are being attacked or if you were attacked) and sensitive information you can't just put into one-way algorithms as much more worthwhile discussions.