A new approach for secret retrieval in NetBox

[Jeremy Stretch]

I've been pondering a way to re-implement the manner in which secret data is accessed in NetBox, and wanted to run it by the list before opening a feature request. Please let me know what you think.

Background

The "secrets" app in NetBox allows for the secure storage of sensitive data. This is accomplished by symmetrically encrypting the payload of each Secret object with a symmetric key known only to NetBox. Each user can provide a personal RSA key pair to store his or her own encrypted copy of the master key, thus allowing users to encrypt and decrypt secret data without exposing the master key.

Currently, NetBox requires a user to POST his or her private RSA key for each request that requires encryption or decryption of a secret. To avoid requiring manual re-entry of the key each time, the key is cached locally using HTML5's local storage. This has two disadvantages:

• The local storage scope is limited to the active browser tab. If the user opens a second tab to access NetBox secrets, he or she must re-enter the private key so that it can be stored locally for that tab. (This is a limitation of the HTML5 implementation and not something that can be controlled by NetBox.)

• The private key must be re-transmitted as part of each request. This introduces significant overhead as the encoded form of a private key is typically several kilobytes in size. Additionally, it necessitates the use of POST requests, because data appended to the body of GET requests may or may not be honored by a web server. (Notably, jQuery also does not support sending AJAX GET requests with data in the body.)

This second point has become particularly problematic while developing API 2.0. In adherence with REST methodology, the retrieval of objects should be done using only GET requests. This hampers our ability to provide a decryption key along with the request, as there's no good place to put it:

• We can't put it in the request body, since this is a GET request and it might be ignored by the server.
• We can't put it in the querystring, because it may be logged or otherwise recorded.
• We can't put it in a cookie, because it may be stored locally on the user's machine (and it's relatively large).
• We probably don't want to pass it as a custom header due to its size (and the need to re-encode the key to avoid line breaks).

The Proposed Solution

We can implement an API endpoint which is used solely to securely cache a user's private key on the NetBox server. I am proposing the following workflow:

• The user provides his or her private key to the client (e.g. the web browser).
• The client POSTs the key to the designated API endpoint.
• NetBox generates a random symmetric encryption key, encrypts the user's private key, and stores the encrypted form in the database.
• The symmetric encryption key is returned to the user as a signed cookie set by the API response.

This approach essentially replaces the user's large and very sensitive private RSA key with a small throwaway key which is much more practical to transmit with each request. If the key is lost, a new one can be generated by POSTing the user's private key via the API again.

All requests to retrieve secret data from NetBox are checked for the presence of the signed cookie. If the cookie is absent, secret objects are returned without being decrypted. If the cookie is present, NetBox attempts to decrypt the user's cached private key, which is in turn used to decrypt secret data.

The cookie will be set to expire with the user's session. A administratively-configurable maximum age may also be enforced on the server side.

The implementation of this proposal would involve adding a field to the UserKey model to store the cached, encrypted private key. Alternatively, we could store the encrypted key using a backend such as memcached, but that would require users to install and configure a separate caching engine.

It's possible that I've overlooked something, but I think it's a decent plan to improve the current scheme. Interested to hear what others think.

Jeremy

[Brian Candler]

> We can implement an API endpoint which is used solely to securely cache a user's private key on the NetBox server.

This "random symmetric encryption key" sounds like it performs a similar role to the "passphrase" you'd normally use with GPG - except that because it's chosen at random rather than selected by the user, it's not vulnerable to offline brute force attacks.

I guess it doesn't harm to review the threat model:

1. In the current system, people post their private key in the clear with every request, and the responses contain decrypted secrets.  If an attacker has access to either the incoming requests or outgoing responses, you are toast.

2. You're adding a cache of private keys, and you're trying to protect against an attacker who has access to the cache, but not to the requests/responses

3. An attacker might also obtain access to private keys on the client endpoint. If so, they'd also need the user's netbox login to use it.  And if they have sufficient access to the client machine, they could intercept decrypted secrets there anyway.

> The "secrets" app in NetBox allows for the secure storage of sensitive data. This is accomplished by symmetrically encrypting the payload of each Secret object with a symmetric key known only to NetBox. Each user can provide a personal RSA key pair to store his or her own encrypted copy of the master key, thus allowing users to encrypt and decrypt secret data without exposing the master key.

It does make me nervous that Netbox is repeatedly receiving users' private keys, in plain text. (I'd never trust a server with my real private key and passphrase, so I'd always generate a separate single-use key for Netbox)

It also makes me nervous that there is a single system-wide master key, even if that is never persisted on the server side.

I note that GPG has an analogous shared access mechanism built-in.  You can encrypt a file to multiple recipients, i.e. multiple public keys.  A single random session key is created, and the document encrypted with a symmetric cipher.  That session key is then encrypted with each of the public keys, and the whole blob forms the encrypted file. It can be decrypted by a holder of any one of the corresponding private keys.

If this mechanism were used:

* the API for "fetch secret" would just be a GET of the encrypted secret

* the decryption could be done entirely browser-side (e.g. with openPGP.js)

* there would be no single master key to worry about

* different secrets could be encrypted to different groups of people

* the encrypted secrets could be cached client side and made available even when off-line

You'd still have to trust your browser with your private key and passphrase, but it would never leave your machine.

When adding new secrets, I'd still probably POST them to the server to deal with, so the clients don't need to learn all the other GPG public keys.  But the server would store only public keys, and would never handle a private key.

To me, this feels much stronger - although we can argue forever about server security versus browser security :-)

Specific limitations:

1. The encrypted secrets would be larger, depending on the number of keys they are encrypted with, and these larger objects would be sent back to the browser (typically a few KB per secret). But I don't see that as a problem; since they are plain GET objects, they would cache easily just like large javascript files.

2. If you need to add or remove a member from your team, you'd have to decrypt and re-encrypt all the secrets to the new set of keys. This requires implementing a new bulk operation. If it were done entirely server-side it would mean the server *would* need a private key for the duration of that operation - or it would need assistance from a client.

General weaknesses:

3. If you encourage client-side caching of encrypted material, arguably it opens it to client-side brute-force attacking of passphrases etc.

Of course, at the moment nothing stops people copy-pasting the unencrypted secrets into a local plain text file.  You just *hope* that by making them easily available via the web, they will not feel the need to do so.  Making them available encrypted off-line may actually reduce the need for people to store local unencrypted copies.

Regards,

Brian.

[Jeremy Stretch]

I opened a feature request for this last night (#869) after some experimentation. I realized that it made more sense to store a symmetrically-encrypted copy of the master key itself, rather than the user's private key. The feature request explains the new approach in more detail.

> It does make me nervous that Netbox is repeatedly receiving users' private keys, in plain text.

This is analogous to the risk posed by logging into NetBox with a username and password. I'd hope people are running NetBox only with TLS. (Django has a limited ability to enforce this, but it's dependent on configuration of the front end httpd.)

> I note that GPG has an analogous shared access mechanism built-in.  You can encrypt a file to multiple recipients, i.e. multiple public keys.  A single random session key is created, and the document encrypted with a symmetric cipher.  That session key is then encrypted with each of the public keys, and the whole blob forms the encrypted file. It can be decrypted by a holder of any one of the corresponding private keys.

The problem here is that asymmetric encryption is very slow. If you encrypt each secret using a different, random symmetric key, and encrypt each symmetric key with each user's public key, then a separate asymmetic decryption must be run to recover the random key for each secret being retrieved. This is problematic if, for example, you need to retrieve the RADIUS secret for every device on your network.

Alternatively, you could create a per-user symmetric key, and encrypt a copy of each secret with each user key. I experimented with this approach a bit in the early days of NetBox, but the complexity and validation needed made it unpalatable. And you essentially still have the same situation as the current design, but instead of one master key, you have many (one per user).

The other aspect of this is whether the client should be expected to perform decryption of ciphertext. Given that the session between the client and NetBox is expected to be secure anyway (otherwise, your authentication has already been exposed), I don't have any issue with delivering plaintext directly via the API. It greatly simplifies client implementation.

Jeremy

--
You received this message because you are subscribed to the Google Groups "NetBox" group.
To unsubscribe from this group and stop receiving emails from it, send an email to netbox-discuss+unsubscribe@googlegroups.com.
To post to this group, send email to netbox-discuss@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/netbox-discuss/11d4788d-a506-44d8-af4f-8c29f1cc140f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Brian Candler]

> > It does make me nervous that Netbox is repeatedly receiving users' private keys, in plain text.
> 

> This is analogous to the risk posed by logging into NetBox with a username and password. I'd hope people are running NetBox only with TLS.

I don't really worry about the transport of the key from the client to the server. I worry very much about flaws in the server which could cause the key to become compromised - both in the application itself, and the huge framework it depends on (django).  It's not like there's never been a security hole in Django or Ruby on Rails, for example :-)

I also worry about applications using home-brewed crypto systems instead of known-robust applications like GPG/PGP.  The fact that people like you and I can't spot a flaw doesn't mean there isn't one.

The problem here is that asymmetric encryption is very slow. If you encrypt each secret using a different, random symmetric key, and encrypt each symmetric key with each user's public key, then a separate asymmetic decryption must be run to recover the random key for each secret being retrieved. This is problematic if, for example, you need to retrieve the RADIUS secret for every device on your network.

That's partly a case of how you use it.  What I do is put my devices into classes, and then all devices in a particular class have the same key, tagged with a label like "RADIUS-1". Many devices will use "RADIUS-1".  When it's time to rotate the keys, I'll generate a new one called "RADIUS-2" and then migrate devices over to it.  So at any point in time there might be a subset on RADIUS-1 and another subset on RADIUS-2, but I only need to decrypt two things in total.

But that's just my way of working. If somebody wants to generate a separate random key for every device, they're entitled to do so.

At the other extreme, I note that access controls for keys are at the level of "roles".  So if performance is an issue, you could put all the keys relating to one role into a single file. Even if there were ten thousand different passwords, that role file would probably be well under 100KB (given that the data is compressed before being encrypted). I know what you're going to say - adding a single new password becomes an expensive operation :-)

Finally, you could just punt the whole thing out to an external password management system like Hashicorp Vault (https://www.vaultproject.io/). Then all you need to store in Netbox is key labels like "secret/radius-1" or "secret/radius/radsrv123.example.com"

Actually, this last option would suit me very nicely because it maps directly to how I do things anyway (with password-store.org) 

 

The other aspect of this is whether the client should be expected to perform decryption of ciphertext. Given that the session between the client and NetBox is expected to be secure anyway (otherwise, your authentication has already been exposed), I don't have any issue with delivering plaintext directly via the API. It greatly simplifies client implementation.

You're right, I need to remember there are non-browser API clients as well.  And this is also the approach taken by vaultproject.

Regards,

Brian.