SHA 1 going away

[L Snider]

Hi All,

I saw an article recently on how Microsoft is going to be getting rid of SHA 1 (phasing it out until 2017). I was just curious if anyone knows more about this? Do we care? I was wondering why they would get rid of it?

I use SHA 256, and I assume this will be okay?

Cheers

Lisa

-- 

Lisa Snider
Electronic Records Archivist
Harry Ransom Center
The University of Texas at Austin
P.O. Box 7219
Austin, Texas 78713-7219
P: 512-232-4616
www.hrc.utexas.edu

[Ben O'Steen]

https://www.schneier.com/blog/archives/2005/02/sha1_broken.html

SHA-1 has been shown to be 'broken' for nearly 9 years (according to the above link). What that means is that SHA1 shouldn't be used when generating digital signatures if avoiding collisions is important. Given the plethora of hashing routines, there is little to no reason to use it any more.

The SHA-2 routines (of which SHA-256 is a member) are seemingly okay for use and definitely should be used instead of SHA-1. (Wikipedia tells me that NIST have picked the routines for SHA-3 last year - anyone planning on using them?)

Ben

--
You received this message because you are subscribed to the Google Groups "Digital Curation" group.
To unsubscribe from this group and stop receiving emails from it, send an email to digital-curati...@googlegroups.com.
To post to this group, send email to digital-...@googlegroups.com.
Visit this group at http://groups.google.com/group/digital-curation.
For more options, visit https://groups.google.com/groups/opt_out.

[Eby]

There are more posts in the schneier archive that gives more examples of those using it and why it should be retired.

https://www.schneier.com/cgi-bin/mt/mt-search.cgi?tag=SHA-1

[Stephen Marks]

Are you using it for encryption, or just checksumming? These are two different applications, and the Microsoft advisory is against using it is in a cryptographic context (e.g., for stored passwords, signatures, etc.). 

The real question for curators[1] is whether or not you think you're likely to be subject to attempts to replace files you're curating with bogus files with a colliding checksum. In this context, I think you're better off generating multiple checksums, rather than using a single one, no matter how strong it is.

[1]: Outside of system administration, that is. Yes, if you are developing or administering systems that use broken hashing algorithms (MD5, SHA-1, whatever) cryptographically, then by all means STOP NOW.

 

[Robert Schmidt]

On Mon, Nov 25, 2013 at 9:55 AM, Stephen Marks <sbm...@gmail.com> wrote:
>
> The real question for curators[1] is whether or not you think you're likely
> to be subject to attempts to replace files you're curating with bogus files
> with a colliding checksum. In this context, I think you're better off
> generating multiple checksums, rather than using a single one, no matter how
> strong it is.
>

Yeah, and there are lots of interesting reasons you might want to keep
more than one hash around. CRC32 is very fast computationally, so if
you have a lot of data you'll save some time if you are just doing
error detection.

We use crc32 a lot for error detection for that reason.

--
Robert Schmidt robert....@canadiana.ca

[Trevor Owens]

There is a nice paper on comparisons between different kinds of hashes NIST ran on files in the software reference library back in 2006 http://www.nsrl.nist.gov/Documents/analysis/draft-060530.pdf 

[Simon Spero]

SHA-2 was not replaced by SHA-3 ; however an SHA-3  was chosen. You can use both to give extra protection against possible unknowns, but if you do so, you should use the hashes in parallel. Hashing a hash is as weak as the weakest of the two hashes.

CRC32 is not suitable for large amounts of data as the possibility of random collisions is too high.

[Chris Adams]

crc32 is widely used for performance but also a good cautionary reminder of the trade-offs: on a 2.1GHz Core 2 Duo, there's no significant difference between crc32, md5, and sha1 – below the order of 100MB files, overhead is going to dominate. Once you start talking 1+GB, there's more room for differentiation:

crc32.py:  4.164s

openssl md5:  6.879s

openssl sha1: 8.281s

openssl sha -sha256: 12.549s

openssl sha -sha512: 6.375s

(The numbers above are on a 1GB file from /dev/random – see https://gist.github.com/acdha/7647086; if you want a simpler comparison try openssl speed md5 sha1 sha256 sha512)

Even here, though, the slowest algorithm is sha256 and that's still managing around 680Mb on a single CPU core – unless your systems have faster storage and less CPU capacity, all of this will be moot. Processors are increasingly shipping with hardware which can further accelerate this[1] so any performance wins are likely to be obsolete beyond the next few years.

For most users, the real thing to look at are the odds of a collision in normal usage (see http://preshing.com/20110504/hash-collision-probabilities/#small-collision-probabilities) and under an active attack. The first case is fairly easy to answer: anything other than simple CRCs is likely to be sufficient and using a second algorithm, preferably from a different family, provides a significant safety check. The second case is more complicated because it depends on whether the attacker is trying to replace a file outside of their control (this is known as a pre-image attack and is much harder than simply generating two files which hash to the same value) or simply to construct two files which hash to the same value. For most of us this is a low risk and can be addressed by using multiple good modern hashes in addition to MD-5/SHA-1.

Chris

1. http://software.intel.com/en-us/articles/improving-the-performance-of-the-secure-hash-algorithm-1/ details an optimization which shipped in ~2004 and is supported by OpenSSL 1.0.1+

[L Snider]

Hi All,

Thanks so much for all your links and information-it is much appreciated. This would be just for checksumming, but encryption may happen at a later date.

I like using two checksums when possible, so more to read now. I didn't know about the performance factors, a good thing to learn more about!

Thanks again!

Cheers

Lisa

--