Pluggable secret key backend

[Andreas Pelme]

Hi,

settings.SECRET_KEY can be used for sessions, password resets, form wizards and
other cryptographic signatures via the signing APIs. Changing SECRET_KEY means
that all of those will be invalidated and the users will be affected in weird
ways without really knowing what happened. (Why am I logged out? Where did my
form submission go? Why does not this password reset link work?). This is
desirable in case the key is compromised and swift action must be taken.

There are other situations when it would be nice to change the SECRET_KEY when
this sudden invalidation is not desirable:

- When someone leaves a project/company that had access to the production
system. After SSH keys/login credentials is revoked the developer could
potentially have a copy of the secret key. It is essentially a backdoor with
full remote access. It would be wise to rotate the key in those cases.

- Periodic and automatic rotations of keys to make it less useful in the
future.

The current situation of a single SECRET_KEY makes key rotation impractical. If
you run a busy site with active users 24/7, there is never a nice time to
change the SECRET_KEY.

A solution for this problem would be sign new secrets with a new key while
still allow signatures made with the old key to be considered valid at the same
time. Changing keys and having a couple of hours of overlap where signatures
from both keys are accepted would mitigate most of the user facing problems
with invalidating sessions, password reset links and form wizard progress.

You could do this today by implementing your own session backend, message
storage backend and password reset token generator but that is cumbersome and
does not work across reusable apps that directly use low level Django signing
APIs unless they too provide hooks to provide your own secret.

I propose a pluggable project wide secret key backend
(settings.SECRET_KEY_BACKEND maybe?) with an API something like:

class SecretKeyBackend:
def get_signing_key(self): …
def get_verification_keys(self): ...

The default (and backward compatible) backend would then be implemented as
something like:

class SecretKeySettingsBackend:
def get_signing_key(self):
return settings.SECRET_KEY
def get_verification_keys(self):
return [settings.SECRET_KEY]

django.core.signing.Signer.{sign,unsign} would need to be updated to use this
backend instead of directly using settings.SECRET_KEY.

That would solve the problem project wide and work across any third party
application that uses django.core.signing directly.

This would open the door for third party secrets backend packages that
retrieves keys from systems such as Hashicorp Vault, AWS Secrets Manager,
Google Cloud KMS, Docker Secrets etc.

Having a method that retrieves the key would allow changes to secret key during
run time instead of relying on a hard coded setting would allow the key to
change without restarting the server process.

Would something like this be worth pursuing? Could it be designed in som other
way? I could not find any previous discussion/tickets on this and thought it
would be a good idea to discuss it here before opening a ticket or making an
attempt at a PR. :)

Cheers,

Andreas

[ludovic coues]

I don't see how this would work.

For example the session. You take the user cookie. You try to validate with your secret key. That doesn't work because the current key is the new one.

With a custom cookie backend, you could check if the old secret could validate the cookie. But you need to change your cookie backend to handle the case of multiple secret key. And all third party session backend need to update.

--
You received this message because you are subscribed to the Google Groups "Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/20D8A2BD-BC9C-4F02-9038-044687165DE9%40pelme.se.
For more options, visit https://groups.google.com/d/optout.

[Adam Johnson]

Hi Andreas

I like your proposal, moving to a backend is an elegant way of solving both the immediate problem and opening up the other possibilities you mentioned.

I think it would also be nice to have an "out of the box" way of rotating the key, without needing to implement a custom backend. Perhaps a second setting OLD_SECRET_KEYS that may contain a list of old keys that are returned for verification too? Or we could allow SECRET_KEY to be a list/tuple, and if so, sign with the first and verify with all of them.

Adam

--
You received this message because you are subscribed to the Google Groups "Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/20D8A2BD-BC9C-4F02-9038-044687165DE9%40pelme.se.
For more options, visit https://groups.google.com/d/optout.

--

Adam

[Andreas Pelme]

> On 10 Nov 2018, at 13:00, ludovic coues <cou...@gmail.com> wrote:
>
> I don't see how this would work.
>
> For example the session. You take the user cookie. You try to validate with your secret key. That doesn't work because the current key is the new one.
>
> With a custom cookie backend, you could check if the old secret could validate the cookie. But you need to change your cookie backend to handle the case of multiple secret key. And all third party session backend need to update.

I propose that we make the low level django.core.signing aware of multiple keys. Everything that is already using django.core.signing such as signed cookies, sessions and password reset tokens would need *not* need to change.

Cheers,
Andreas

[Andreas Pelme]

On 10 Nov 2018, at 13:29, Adam Johnson <m...@adamj.eu> wrote:
>
> Hi Andreas
>
> I like your proposal, moving to a backend is an elegant way of solving both the immediate problem and opening up the other possibilities you mentioned.

Thanks Adam, I am glad you like the proposal. :)

> I think it would also be nice to have an "out of the box" way of rotating the key, without needing to implement a custom backend. Perhaps a second setting OLD_SECRET_KEYS that may contain a list of old keys that are returned for verification too? Or we could allow SECRET_KEY to be a list/tuple, and if so, sign with the first and verify with all of them.

Agreed, I will add something like that then! :)

Cheers,
Andreas

[Dan Davis]

Maybe a LoFi way to accomplish this is just to make sure that the SECRET_KEY is cast to bytes() before use.   That way, a non-bytes object placed there during settings will be asked to convert it to bytes before use.   I use the same trick with an internal module that retrieves database passwords from a web service.   For cx_Oracle, I only had to implement __str__, but for PostgreSQL, MySQL, and pyodbc to SQL Server I eventually I collected many other string methods to be duck typed as a string.

The same trick might work today with SECRET_KEY, depending on how it is used.   If anyone ever does a check that isinstance(settings.SECRET_KEY, bytes), then we'd have problems, but if Django has the discipline to iterate it, get its length, and cast it to bytes before use, then it would be OK.

--
You received this message because you are subscribed to the Google Groups "Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/A16A11DF-1439-46EF-BF0D-85C483F53608%40pelme.se.

[Aymeric Augustin]

Hello,

I think this is a great idea.

As suggested by others, an even better default implementation would be:

class SecretKeysBackend:

    def get_signing_key(self):

        if isinstance(settings.SECRET_KEY, (list, tuple)):

            return settings.SECRET_KEY[0]

        else:

            return settings.SECRET_KEY

    def get_verification_keys(self):

        if isinstance(settings.SECRET_KEY, (list, tuple)):

            return settings.SECRET_KEY

        else:

            return [settings.SECRET_KEY]

Once Django is updated to take advantage of this feature, hat would make key rotation practical for every Django user!

(And it seems easier to adjust the semantics of SECRET_KEY than to introduce a SECRET_KEYS settings.)

Best regards,

-- 

Aymeric.

--
You received this message because you are subscribed to the Google Groups "Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/20D8A2BD-BC9C-4F02-9038-044687165DE9%40pelme.se.

[Tom Forbes]

Is it going to be easy to adjust the semantics of SECRET_KEY to support sequences like that? I’d imagine a lot of third party packages that expect SECRET_KEY to be a string would break in weird ways (thanks to both strings and tuples of strings being iterables that yield strings).

Here’s a quick search on GitHub: https://github.com/search?q=%22settings.SECRET_KEY%22&type=Code

To ease the backward compatibility concerns we could use SECRET_KEYS, then make SECRET_KEY (if it is not explicitly defined) map to SECRET_KEYS[0]? Third party packages using would not necessarily work with the backwards verification but they would at least not break and continue to work as expected.

Tom

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/F62259DE-AA39-40ED-8E13-472CFAB9AFBA%40polytechnique.org.

[Aymeric Augustin]

Good point, I can think of at least two apps of mine that would break. Transitioning to a new setting makes more sense.

--
Aymeric.

> To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAFNZOJMaM_512quK0yzsZiRO9Kada7jhUJg-G-vpu0Nfn-y8FQ%40mail.gmail.com.

[Adam Johnson]

Very good point. I'd prefer a second setting though, named like OLD_SECRET_KEYS or VERIFICATION_SECRET_KEYS. If we're going to add a new setting, we might as well not force users who aren't rotating their keys to the new one, especially if they are semantically different.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/2A3BF993-E132-4F93-8C59-1DD49F3EB9F0%40polytechnique.org.

For more options, visit https://groups.google.com/d/optout.

--

Adam

[Tobias McNulty]

Personally, I like the simplicity and elegance of a single SECRET_KEYS setting. It's also a good way to raise awareness that rotation is A Good Thing to be doing anyways.

In any case, I second all of those who've already endorsed this idea. If I can help, let me know.

Tobias

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAMyDDM1YGrV3EiN16acGjzDviBJRSsXZwdhSXv1AB0nxyU%3D_oA%40mail.gmail.com.