ANN: Keyczar Go Implementation

[Damian Gryski]

Hi,

I'm sending this both to the Keyczar and the Go mailing lists.

In the past week I hacked up a quick proof-of-concept Go
implementation of Keyczar using the Go cryptography libraries as the
backend. It's based mainly on the Python version since it matched
more closely with my view of how I would want to use the library in
Go.

There's still a lot to do in terms of code cleanup, but all the main
functionality should be there. I am able to load all the keys in the
testdata directory and use them for encryption, decryption, signing,
and verification (as appropriate). Key generation is not (yet)
supported. The only outstanding change to my API is making everything
return appropriate error codes. After that, everything else should be
internal: refactorings, checking return values, comments, more test
cases etc.

I'm planning on finishing this work over the holidays and will put
together a more full release "when it's done".

The code is at https://github.com/dgryski/dkeyczar . It's licensed
under Apache 2.0 like the rest of Keyczar.

For the Go folks: Keyczar is a cryptography library from Google housed
at http://www.keyczar.org/

Thanks for a great library and a great language,

Damian

[Shawn Willden]

On the part of the Keyczar team, that's awesome!  It's great that you find the tool useful enough to be worth the effort of implementing it yourself, and I look forward to seeing the completed, cleaned-up version.

I do have one concern, though... are you signing up to be the maintainer of the Keyczar Go implementation?  If so, that's perfect.  If not... I think we'll probably have to merge your implementation into the Keyczar project tree as "unsupported".  We've found that it's a significant effort to maintain three versions of the library with feature equivalence and cross-language interoperability.  Indeed, we're not even 100% sure that we have it; one of my holiday TODOs is to write some tools that test cross-language interoperability for the whole supported feature set.

As an example of the problem, the Python implementation upon which you based your Go implementation is fairly deficient right now.  Specifically, it lacks:

• Session encryption (close to being merged; see the python_sessions branch)
• Certificate import
• PKCS#8 private key import/export
• Password-based key encryption

The C++ implementation has all these features.  Java has all but password-based key encryption and PKCS#8 import/export -- but that's basically done and just needs to be merged.

I should mention that we're also going to be adding a C# implementation.  That will also increase the maintenance burden, but C# is being requested by many.  Go is awesome... but we haven't had any requests for it.

--
You received this message because you are subscribed to the Google Groups "Keyczar Discuss" group.
To post to this group, send email to keyczar...@googlegroups.com.
To unsubscribe from this group, send email to keyczar-discu...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/keyczar-discuss?hl=en.

--

Shawn Willden |

Software Engineer |

swil...@google.com |

Commerce Team

[Damian Gryski]

Oops, responded with the wrong email address and it bounced.

---------- Forwarded message ----------
From: Damian Gryski <d...@pobox.com>
Date: Fri, Dec 16, 2011 at 4:38 PM
Subject: Re: [go-nuts] ANN: Keyczar Go Implementation
To: roger peppe <rogp...@gmail.com>
Cc: keyczar...@googlegroups.com, golan...@googlegroups.com

On Fri, Dec 16, 2011 at 4:13 PM, roger peppe <rogp...@gmail.com> wrote:

> On 16 December 2011 12:47, Damian Gryski <dgr...@gmail.com> wrote:
>> The code is at https://github.com/dgryski/dkeyczar .  It's licensed
>> under Apache 2.0 like the rest of Keyczar.
>

> cool.
>
> a few initial reactions (based only on reading
> http://gopkgdoc.appspot.com/pkg/github.com/dgryski/dkeyczar)
> and a 1-second glance at the source.
>
> - documentation! there are no comments on the exported types,
> functions or the package.

 No, my late-night hacking was targeted towards functional
completeness first.  As I said, this really was a "preview release".
Getting the docs done is certainly on my list and, now that I've
announced the project, probably next up.

> - why bother exporting interfaces? why not just export the keyCzar
> type  (you could follow the example of a few other crypto packages
> and just call it "Cipher")

  I think because the the key-type you load limits you to certain
functionality, so I wanted to make sure that if you though you were
loading a particular type that you thought you could do something
with, it either errored on load or whatever you were asking it to do
would work.

  The other option is to check the key purpose at the start of each
Encrypt/Decrypt/Sign/Verify and immediately return with "Invalid
Purpose" at that point, rather than at object creation time.  I
suppose if I just export the Keyczar type, I can probably get both of
these, either by assigning to an object of the type I'm expecting to
use it for.  Most of this code was written while sick and generally
after 10pm.  It's possible I didn't make the best decisions ... :)

> - following on from the above, it would definitely be worth following
> existing conventions as much as possible (for instance, using
> Encrypt(dst, src []byte)).
> that way your type can potentially be share an interface with existing
> algorithms.

  Good idea, I hadn't thought about reusing Keyczar objects with
other Go modules.  Although, I'm not sure it makes sense to implement
something like cipher.Block for this.  We're not just wrapping a raw
encrypt/decrypt routine, we're adding headers and change the encoding
(web64).  We still want the ability to return error objects, which
we've now just lost because Encrypt and Decrypt are assumed not to
fail.  However, our methods have more failure modes (invalid encoding,
invalid version, unknown key, ...)

  I'll look through some of the Go standard interfaces and see if any
make sense to use here.

 Thanks for your input!

 Damian

[Damian Gryski]

Le vendredi 16 décembre 2011 17:13:31 UTC+1, Shawn Willden a écrit :

I do have one concern, though... are you signing up to be the maintainer of the Keyczar Go implementation?

   Yes, that was my plan.

 

As an example of the problem, the Python implementation upon which you based your Go implementation is fairly deficient right now.  Specifically, it lacks:

• Session encryption (close to being merged; see the python_sessions branch)
• Certificate import
• PKCS#8 private key import/export 
• Password-based key encryption
  

   I could tell the Python version was missing things, but that also made it a good starting point since it showed which things were needed for a minimal implementations.  I've now added these items to my TODO list.

  Go is awesome... but we haven't had any requests for it.

   My reason for building it was that I had seen some complaints in the #golang-nuts irc channel about how complicated it was to use the standard crypto libraries.  Keyczar I believe is a solution to this problem, they just didn't know it existed :)

   Damian

[Damian Gryski]

Le vendredi 16 décembre 2011 17:13:31 UTC+1, Shawn Willden a écrit :

• Session encryption (close to being merged; see the python_sessions branch)
• Certificate import
• PKCS#8 private key import/export
• Password-based key encryption

 

These have now been implemented:

• Session encryption
• Password-based key encryption.
• Importing unencrypted PEM RSA private and public keys

    It looks like Go only has partial support for PKCS8 import/export (RSA import only), so I think I'll either need to patch the standard library or wait until somebody else does.

    Can you be more specific on what you mean by "Certificate import" ?

    Thanks,

    Damian

[Shawn Willden]

On Tue, Dec 20, 2011 at 4:34 PM, Damian Gryski <dgr...@gmail.com> wrote:

These have now been implemented:

• Session encryption
• Password-based key encryption.
• Importing unencrypted PEM RSA private and public keys

Very nice!

 

    It looks like Go only has partial support for PKCS8 import/export (RSA import only), so I think I'll either need to patch the standard library or wait until somebody else does.

That's unfortunate.  Does it support import of password-protected private keys?  Just trying to get a feel for the scope of what's not covered.

 

    Can you be more specific on what you mean by "Certificate import" ?

Sure.  Import the public key from an X.509 public key certificate into an existing, appropriately-typed Keyczar key set.  Arguably we should look at the extended attributes of the cert to figure out what Keyczar purpose to attach, but that's not always unambiguous so for the moment we just require the user to specify during the import.

 

[Damian Gryski]

Le mercredi 21 décembre 2011 00:54:16 UTC+1, Shawn Willden a écrit :

 

    It looks like Go only has partial support for PKCS8 import/export (RSA import only), so I think I'll either need to patch the standard library or wait until somebody else does.

That's unfortunate.  Does it support import of password-protected private keys?  Just trying to get a feel for the scope of what's not covered.

   I'd have to say "no".  There doesn't seem to be anything related to passwords or encryption in the x509 code ( http://tip.golang.org/pkg/crypto/x509/ )

 

    Can you be more specific on what you mean by "Certificate import" ?

Sure.  Import the public key from an X.509 public key certificate into an existing, appropriately-typed Keyczar key set.  Arguably we should look at the extended attributes of the cert to figure out what Keyczar purpose to attach, but that's not always unambiguous so for the moment we just require the user to specify during the import.

   Ok, that sounds pretty straight-forward.  The x509 library should handle this easily.

   Damian 

[Shawn Willden]

On Tue, Dec 20, 2011 at 5:17 PM, Damian Gryski <dgr...@gmail.com> wrote:

Le mercredi 21 décembre 2011 00:54:16 UTC+1, Shawn Willden a écrit :

That's unfortunate.  Does it support import of password-protected private keys?  Just trying to get a feel for the scope of what's not covered.

   I'd have to say "no".  There doesn't seem to be anything related to passwords or encryption in the x509 code ( http://tip.golang.org/pkg/crypto/x509/ )

I'm talking about PKCS#8, not X.509.

X.509 is a standard that defines the structure and content of public-key certificate files.  There's no need for encryption or passwords because they're public keys.

PKCS#8 is a standard that defines the structure and content of private key files.  They aren't certified, but contain private keys, so it also specifies the an optional mechanism for password-based encryption of the key data, using the PKCS#5 password-based encryption standard.

http://www.rsa.com/rsalabs/node.asp?id=2130

   Ok, that sounds pretty straight-forward.  The x509 library should handle this easily.

Cool.  X.509 import has been quite straightforward on all of the platforms, so that doesn't surprise me. 

[Damian Gryski]

Le mercredi 21 décembre 2011 17:46:29 UTC+1, Shawn Willden a écrit :

On Tue, Dec 20, 2011 at 5:17 PM, Damian Gryski <dgr...@gmail.com> wrote:

Le mercredi 21 décembre 2011 00:54:16 UTC+1, Shawn Willden a écrit :

That's unfortunate.  Does it support import of password-protected private keys?  Just trying to get a feel for the scope of what's not covered.

   I'd have to say "no".  There doesn't seem to be anything related to passwords or encryption in the x509 code ( http://tip.golang.org/pkg/crypto/x509/ )

I'm talking about PKCS#8, not X.509.

X.509 is a standard that defines the structure and content of public-key certificate files.  There's no need for encryption or passwords because they're public keys.

PKCS#8 is a standard that defines the structure and content of private key files.  They aren't certified, but contain private keys, so it also specifies the an optional mechanism for password-based encryption of the key data, using the PKCS#5 password-based encryption standard.

http://www.rsa.com/rsalabs/node.asp?id=2130

   The only code that deals with PKCS#8 stuff in Go is under the x509 library.  It currently defines a single routine that has the header comment

             // ParsePKCS8PrivateKey parses an unencrypted, PKCS#8 private key. See

             // http://www.rsa.com/rsalabs/node.asp?id=2130"

  ( http://code.google.com/p/go/source/browse/src/pkg/crypto/x509/pkcs8.go#23 )

   This is one of the library areas that I was saying needed work.

   Damian

[Shawn Willden]

On Wed, Dec 21, 2011 at 1:37 PM, Damian Gryski <dgr...@gmail.com> wrote:

   The only code that deals with PKCS#8 stuff in Go is under the x509 library.  It currently defines a single routine that has the header comment

             // ParsePKCS8PrivateKey parses an unencrypted, PKCS#8 private key. See

             // http://www.rsa.com/rsalabs/node.asp?id=2130"

Ah, okay.  Now I see why you referred to the X.509 lib.

Implementing PKCS#5 PBE support wouldn't be terribly difficult, but it's not completely trivial either.  Given that, it would then be easy to support encrypted PKCS#8 keys.

For the moment I'm okay with just documenting that the Go implementation doesn't support PKCS#8 import of encrypted private keys or DSA keys.

[Damian Gryski]

Le mercredi 21 décembre 2011 22:38:19 UTC+1, Shawn Willden a écrit :

Implementing PKCS#5 PBE support wouldn't be terribly difficult, but it's not completely trivial either.  Given that, it would then be easy to support encrypted PKCS#8 keys.

   I had to implement PBKDF2 for the 'pbe_json' key-set, so I've already done that much.  Implementing the rest of PKCS#5 can't be _too_ much harder ;)

 

For the moment I'm okay with just documenting that the Go implementation doesn't support PKCS#8 import of encrypted private keys or DSA keys.

   I'm more likely to implement it and get it submitted upstream so I can get my name in the Go contributors file :)

   Damian 

[Damian Gryski]

The Go Keyczar library still has some rough edges, but should otherwise now support all major features of the Python and Java versions. 

There's also an (ugly/undocumented) 'keyczart' implementation for creating and managing keys so in theory you shouldn't need any of the other versions installed to start using keyczar in your Go code.

I've tested it for compatibility with keyczar-py, so hopefully there aren't any interoperability issues.

The pace of development will slow down a bit in January to give priority to some other areas, but I will continue to improve the codebase -- there are still a number of items listed in the TODO file.

Again, the code is hosted on my github account at: https://github.com/dgryski/dkeyczar

Any feedback is welcome,

Damian

[Shawn Willden]

Very cool, Damian.

FYI, I'm going to be slammed for the first part of January, so it'll be a while before I can look at pulling this into the keyczar.org repo.  I'll take a few minutes today and link your repo from the keyczar.org site, however.

Damian

--

You received this message because you are subscribed to the Google Groups "Keyczar Discuss" group.

To view this discussion on the web visit https://groups.google.com/d/msg/keyczar-discuss/-/_Gvv5uX3dngJ.

To post to this group, send email to keyczar...@googlegroups.com.
To unsubscribe from this group, send email to keyczar-discu...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/keyczar-discuss?hl=en.