As mentioned in the thread about cookie-based notifications, at the
DjangoCon Sprints I raised the subject of adding signing (and signed
cookies) to Django core.

I've found myself using signing more and more over time, and I think
it's a concept which is common enough to deserve inclusion in Django -
if anything, its use should be actively encouraged by the framework.

It's also something that's hard to do correctly. At the sprints Armin
pointed out that I should be using hmac, not straight sha1, for
generating signatures (something Django itself gets wrong in the few
places that implement signing already). Having a cryptographer-
approved implementation will save a lot of people from making the same
mistakes.

Signed cookies
==============

On top of signing (which I imagine would live in django.utils) I'd
like to add a signed cookie implementation. Signed cookies are useful
for all sorts of things - most importantly, they can be used in place
of sessions in many places, which improves performance (and overall
scalability) by removing the need to access a persistent session
backend on every hit. Set the user's username in a signed cookie and
you can display "Logged in as X" messages on every page without any
persistence layer calls at all.

I think signed cookies should either be a separate API from
response.set_cookie or should be provided as an additional argument to
that method. I'm not a fan of signing using middleware (as seen in
http://code.google.com/p/django-signedcookies/ ) since that approach
signs everything - some cookies, such as those used by Google
Analytics, need to remain unsigned.

So the API could either be:

    response.set_signed_cookie(key, value)

Or...

    response.set_cookie(key, value, signed=True)

(I prefer the latter option)

Proposed signing implementation
===============================

I'd be happy to donate my signing code from django-openid to the
cause, which was written to be usable entirely separately from the
rest of the django-openid codebase:

http://github.com/simonw/django-openid/blob/master/django_openid/sign...
http://github.com/simonw/django-openid/blob/master/django_openid/test...

This offers two APIs: sign/unsign and dumps/loads. sign and unsign
generate and append signatures to bytestrings and confirm that they
have not been tampered with. dumps and loads can be used to create
signed pickles of arbitrary Python objects.

Here's what the API would look like with this library:

'hello.9asVJn9dfv6qLJ_BYObzF7mmH8c'

The signature is a URL-safe base64 encoded digest of the hmac/sha1. I
used base64 rather than .hexdigest() for space reasons - base64
digests are 27 characters, hexadecimal digests are 40. When you're
including signatures in cookies and URLs (especially account recovery
URLs sent out in plain text, 80 character wide e-mails) every byte
counts.

Traceback (most recent call last):
...
BadSignature: Signature failed: badsignature

BadSignature is a subclass of ValueError, meaning lazy developers
(like myself) can do the following rather than importing the exception
itself:

try:
    value = signed.unsign(signed_value)
except ValueError:
    return tamper_error_view(request)

'KGRwMApTJ2EnCnAxClMnZm9vJwpwMgpzLg.mYepoYkzWwXRmsCTVJm3Mb0HHz4'
{'a': 'foo'}

Again, the pickle is URL-safe base64 encoded to take up less valuable
cookie space and generally make it easier to pass around on the Web. A
nice thing about URL-safe base64 is that it uses 64 out of the 65 URL-
safe characters (by URL-safe I mean characters that are left unchanged
by Python's urllib.urlencode function) - the remaining character is
the period, which I use to separate the pickle from the signature.

signed.dumps takes a couple of extra optional arguments. The first is
compress=True (default is False) which zlib compresses the pickle if
doing so will save any space:

637

By default, all signatures use Django's SECRET_KEY. If you want to
sign with a different key, you can pass it as an argument to the
various functions:

'hello.o6MKehoOfZ2b2FU84wzibW6IWxI'
'hello'

The dumps and loads methods also take a key argument, as well as an
additional optional extra_key argument for if you want to generate
different signatures for different parts of your application (useful
for the extra paranoid):

'UydoZWxsbycKcDAKLg.1XYDpILo5xqSwImfa3WuJJT4RPo'
'hello'

We'd want to get a proper cryptographer to give this the once-over
before adding it to core, but I'm generally happy with the API. It
could be argued that it's over kill and just sticking signed.sign and
signed.unsign in would be enough, but I'm pretty keen on the
convenience of dumps and loads.

Thinking about it further, an additional API that just gives you the
signature without including the original value would mean it could be
used for hashing passwords as well.

Potential uses
==============

Lots of stuff:

- Signed cookies (obviously)
- Generating CSRF tokens
- Secure /logout/ and /change-language/ links
- Securing /login/?next=/some/path/
- Securing hidden fields in form wizards
- Recover-your-account links in e-mails

We already use signing in a few places in Django core (mainly sessions
and form wizards), currently using md5 without hmac - sha1/hmac would
be an instant improvement.

SECRET_KEY considerations
=========================

One thing that worries me slightly about increasing the amount of
signing going on in Django is that it elevates the importance of the
SECRET_KEY. I'm currently ignorant of best practices regarding
protecting this kind of shared secret, but the steps we take (***ing
it out from the debug pages and otherwise ignoring it) could almost
certainly be improved.

One thing that's particularly interesting to me is what happens when
you change your secret. If you're changing your secret because it's
leaked then obviously you want stuff signed with the old secret to
become invalid immediately, but I can imagine some users wanting to
rotate their secret keys on a continual basis for added security
against brute force attacks.

If you're rotating your secret, invalidating all of your users signed
cookies etc is a bit of an annoyance. It might be worth supporting two
secrets - the current SECRET_KEY and an optional OLD_SECRET_KEY - with
unsigning operations falling back on the old key if the current key
fails. This would allow users to deploy a new secret while keeping the
old one valid for a week or so, upgrading any tokens that use the old
key in the process. This suggestion is inspired by Amazon's recent
announcement of a similar feature for handling web service access
credentials:

http://aws.typepad.com/aws/2009/09/aws-access-credential-rotation.html

This is probably all too much complication, but it's something that's
been nagging at me since I started increasing my dependence on the
SECRET_KEY setting.

So... what do people think? Is this a feature suitable for Django
(obviously I think so)? Is this as simple as getting a cryptographer's
input and dropping signed.py in to django.utils or are there other
design factors we should consider?

Cheers,

Simon