BLAKE2 as hash function

[Dmitry Yakimenko]

I recently stumbled upon a new hash function that claims to be more secure than SHA2 and faster than MD5. It's called BLAKE2.

Here's the blog post about it's history and motivation behind it: https://leastauthority.com/blog/BLAKE2-harder-better-faster-stronger-than-MD5.html

And here are some benchmarks: https://blake2.net/

Since clearskies would have to hash huge amounts of data on mobile devices it might make sense to pick a more performant hash function. It's not too late into the project to consider such a change.

What do you think?

Dima.

[Pedro Larroy]

I think it makes sense what you say, specially if its cheaper than sha256.

What would be the drawback? Libraries?  Less battlefield tested?

Pedro.

--
You received this message because you are subscribed to the Google Groups "ClearSkies Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clearskies-de...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Dmitry Yakimenko]

What would be the drawback? Libraries?  Less battlefield tested?

Yes, these two. As a consequence it might look less credible to people evaluating this software for their use. I think it's minor though.

There's a native (including optimized) and .NET official implementation here: https://github.com/BLAKE2/BLAKE2

It would have to be packaged as Ruby gem I guess for the Jewel's project.

I would benchmark it on some commonly used hardware before diving it to verify the authors' claims. 

Anyhow, changing hash can be done at any time before the software goes live. It doesn't affect the architecture.

[Steven Jewel]

On 03/23/2014 07:58 AM, Dmitry Yakimenko wrote:
> I recently stumbled upon a new hash function that claims to be more
> secure than SHA2 and faster than MD5. It's called BLAKE2.
>
> Here's the blog post about it's history and motivation behind
> it: https://leastauthority.com/blog/BLAKE2-harder-better-faster-stronger-than-MD5.html
> And here are some benchmarks: https://blake2.net/
>

I've tried to avoid innovating in the area of cryptography with
clearskies, focusing on trying to be the best at peer-to-peer sync
instead. (I'm also not a cryptographer, so I've tried to be as
conservative as possible in this area.)

> Since clearskies would have to hash huge amounts of data on mobile
> devices it might make sense to pick a more performant hash function.

The benchmark you linked, while impressive, doesn't show what
performance is like on 32-bit ARM processors, which is the only place
where the SHA256 computation is going to the bottleneck [1]. However,
most mobile devices have limited storage, so I don't think it will be a
practical issue.

The biggest exception is something like a raspberry pi, since they can
have terabytes of data attached. I have a similar device (a
sheevaplug), which can do sha256 at 28MB/s (according to `openssl speed
sha256`). That works out to ten hours per terabyte, but since
checksumming is a one-time thing, and the network syncing can start
before the checksumming has finished, I think that's OK.

> It's not too late into the project to consider such a change.

It's certainly not too late. I think we should keep the protocol in
draft status through the alpha period of the software, only locking it
down as we get to a "beta". That way we can see how things work in the
real world and make adjustments, including changing out the checksum
algorithm.

Also as the project gains momentum, I hope we can attract some
cryptographers who can help us make some informed improvements.

Steven

[1]: On a normal desktop, the disk is probably going to be the
bottleneck. My i7 can do 300MB/s on a single core. If there are
multiple files to checksum, more cores can be involved.

[Alex Chamberlain]

To future proof the protocol, it's probably worth building in a
mechanism for the hash to be changed/configurable per peering? Clients
may *have* to support SHA-256 now, with BLAKE-2 being optional or
something along those lines?

Alex

[Steven Jewel]

On 03/24/2014 10:49 AM, Alex Chamberlain wrote:
> To future proof the protocol, it's probably worth building in a
> mechanism for the hash to be changed/configurable per peering? Clients

> may*have* to support SHA-256 now, with BLAKE-2 being optional or
> something along those lines?

Good idea.

There are a few forwards compatibility mechanisms built-in the protocol
already that would facilitate this:

* An extension could be added for blake2. Extensions are automatically
negotiated at connection time.

* Messages of unrecognized type should be ignored by old implementations.

* Extra properties can be added to messages. They should be ignored by
old implementations.

* Extra properties can be added to the database entry's "value" hash for
a file. They should be ignored by old implementations, but still
propagated by them.

The last option would probably be the right way to add another hashing
mechanism.

In the "directory" extension I am using the sha256 as the unique
identifier for files instead of the path. I will change this to use the
UUID instead.

The SHA256 is also used as input the the bloom filter. I will see if it
makes sense to change this.

Steven