pluggable hashes in crypto/tls

[Shane Hansen]

Is there any reason that the crypto/tls package

directly instantiates sha1 hashes through crypto/sha1.New

rather than looking it up in the hash registry? The latter has the advantage

that users can swap out hash implementations with their own.

My specific use case is go on arm, where dedicated crypto chips are common,

and being able to specify my own hash function and get free SSL acceleration would

be crazy awesome.

I have a patchset ready with this change, assuming it doesn't conflict

with some existing go design decision.

Thanks!

[Shane Hansen]

For review:

http://codereview.appspot.com/6568049/

Allowing pluggable sha1 implementations dovetails really well with go on arm devices which often

have dedicated crypto hardware accessible from userspace.

It's also nice just for playing around. It makes the libraries alot more "hackable" if you can plug

in your own hash implementations.

--
 
 

[minux]

On Tuesday, September 25, 2012, Shane Hansen wrote:

My specific use case is go on arm, where dedicated crypto chips are common,

and being able to specify my own hash function and get free SSL acceleration would

be crazy awesome.

Slightly off topic, but I wonder if we can add support of OS specific

crypto API to std. lib (for example, Linux's AF_ALG or OenBSD's

/dev/crypto)?

Then we automatically support many crypto accelerators out there,

and this is especially important for ARM devices, as crypto accelerators

are pretty common nowadays, but user space can only access it via the

kernel.

[Shane Hansen]

You beat me to the punch minux. I actually have, in my possession,

a pure go (no cgo) /dev/crypto implementation which exposes

hash.Hash and cipher.Blocks. It's been tested on arm, i386, and amd64.

However, I wasn't quite ready to throw it on github yet, so apologies for the

rapidly thrown together README.

https://github.com/shanemhansen/gocryptodev

My real master plan is to:

1) Make hashes pluggable so I can get hardware accelerated hashing on crypto/tls

2) Make a cipher registry like the hash registry, and get crypto/tls to use it.

3) ...

4) speed!

[agl]

On Tuesday, September 25, 2012 2:57:17 AM UTC-4, Shane Hansen wrote:

Is there any reason that the crypto/tls package

directly instantiates sha1 hashes through crypto/sha1.New

rather than looking it up in the hash registry? The latter has the advantage

that users can swap out hash implementations with their own.

The hash registry wasn't designed for different implementations so much as it was designed for optional hashes: some packages want to support lots of hashes but don't want to depend on (and therefore link in) every possible hash.

The fact that RegisterHash allows a hash to be registered multiple times is a bug.

In other cases where we have optional hardware support (i.e. CRC support in some Intel chips, or AESNI), the package itself has done the check at runtime. Does that not work for you?

Cheers

AGL

[Shane Hansen]

Thanks for explaining how that was supposed to work. Working within that design, I guess

I'm just interested in the possibility of adding support for using OCF (linux cryptodev) and AF_ALG, but that seems like a much larger change for a smaller audience. The reason I'm looking for changes in go rather than just libraries is that I would *really* like the crypto/tls library to get improved performance from the alternative hash/cipher implementations, and the registry is a

perfect mechanism for that, and I'd love to see something similar for the cipher api.

I was assuming that the audience of folks wanting to use the kernel-space crypto API's was a small

subset of go users, since these are likely people working on ARM platforms with coprocessors or something like a dedicated load balancer appliance (which I think go could be awesome at).

Using the existing registry seemed like a great way to accomplish that, but if violates some design principle of the standard libraries then it's a no-go.

I'm open to suggestions.

--Shane

AGL

--
 
 

[Adam Langley]

On Tue, Sep 25, 2012 at 8:04 PM, Shane Hansen <shanem...@gmail.com> wrote:
> Using the existing registry seemed like a great way to accomplish that, but
> if violates some design principle of the standard libraries then it's a
> no-go.

Although it's not the intended use, that doesn't mean that it
shouldn't change. However, overriding the registry of hash functions
does often leave you with both the generic and platform-specific
versions linked in, right? It's just that the generic version would be
idle.

(It's also important to note that math/big is not constant time at the
moment so it may be possible to extract key material via timing side
channels. Also, the Go crypto code has not received anything like the
sort of review that more common crypto libraries have. For these
reasons alone you may wish to use something like OpenSSL.)

Are you sure that using the hardware support is actually faster than a
good software implementation? TLS is often going to be dealing with
~1400 byte chunks. (crypto/tls probably doesn't do a good job of
fragmentation at the moment, but that has other performance
implications.) I assume that the overhead of a kernel call every 1400
bytes is going to be significant, and possibly might overwhelm the
benefits.

But, if the benefits are clear then I think the design that would most
closely gel with the current code would be a package that implements
the kernel calls and then code in crypto/aes, crypto/sha1 etc that
figures out whether to call it at run time. (That code would have to
include suitable build tags, of course.)


Cheers

AGL

[Shane Hansen]

Thanks for such a thoughtful reply.

I actually agree with you 100%. According to my benchmarks for the device I'm working with

software the blocks must be above a certain size before you really see a performance gain,

and it's questionable whether tls chunks are large enough.

Maybe I'll refocus on my openssl cgo bindings and abandon that patchset.

Thanks!

[Shane Hansen]

As a shameless plug, for anyone following this conversation who's interested in playing with

a drop-in replacement for crypto/tls.Listener or Hash or Cipher which does have access to any hardware engines

you might have, you can check out my gossl project.

https://github.com/shanemhansen/gossl/

It's still pre-alpha, but it's successfully served thousands of requests without crashing.

On Wed, Sep 26, 2012 at 7:46 AM, Adam Langley <a...@golang.org> wrote:

[Shane Hansen]

Sorry to reply to my own post, but one of the most useful features it that you

can use it to load exotic types of encrypted keys and then pass those to regular

crypto/tls. I believe zeebo just added support for passphrase protected keys to the

go stdlib, but gossl/evp and gossl/compat might help for those of you not running tip.