Using Vault to provide public/private keys for JWE

[Andrew Schneider]

Hello,

I've looked through the documentation and I don't believe I saw anything like this already to support what I'm trying to do.  We would like to use JOSE (Javascript Object Signing and Encryption) within our system to be able to sign and encrypt objects for certain use cases (mostly encryption at rest, but also encryption in flight).  I have already executed a quick PoC with vault using a symmetric AES key that I use to encrypt, store the key in the generic secret backend, and on the other end retrieve the key and use it to decrypt the data.  However, we would like to use an asymmetric public/private key like RSA to handle this.  

What we would like to do is have a backend that the publisher of the data can retrieve the public key from vault for encryption and then the consumer of the data retrieve the private key for decryption.  We would like separate security ACLs around the public versus the private so that we can control who can decrypt that data.  

I know that the transit secret backend exists, but we would prefer to use JOSE for the reason that we do not want the latency of having to hit the server every time we want to encrypt and decrypt the message.  That undesired network overhead that will affect performance, plus additional load on the server.  With JOSE, we can keep the key cached in memory and encrypt and decrypt on the fly, with periodic renewals of the lease being the only server communication.

I know we can do the RSA key generation and store the keys in Vault ourselves, but we start getting into a key maintenance and revocation concern.  If something already exists within Vault, we would prefer to use it.  We would have multiple public/private pairs for use depending on the data being encrypted and signed.

Thanks for any insight,

Andrew

[vishal nayak]

Hi Andrew,

Vault currently doesn't support creation of RSA keys. A backend to create and manage keys will be a nice feature.

But it's currently not on the roadmap. If anyone is interested in writing one, we'd be happy to review a PR :-)

Regards,

Vishal

--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
 
GitHub Issues: https://github.com/hashicorp/vault/issues
IRC: #vault-tool on Freenode
---
You received this message because you are subscribed to the Google Groups "Vault" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vault-tool+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/vault-tool/89f774c2-6f7e-461a-ac97-e661174028e5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

vn

[Andrew Schneider]

Hi Vishal,

I figured that it didn't exist, but thought I would ask.

We may consider writing a back end, but we are still in the evaluation phase of Vault as well, so it wouldn't be until we were definitely in the adopt phase.

If we were to start writing one, can we ask DEV support questions on this group, or is there another avenue for that? I am figuring we would have a few questions.

Thanks,
Andrew

[Jeff Mitchell]

Hi Andrew,

On Mon, May 9, 2016 at 12:02 AM, Andrew Schneider <asch...@gmail.com> wrote:

> What we would like to do is have a backend that the publisher of the data
> can retrieve the public key from vault for encryption and then the consumer
> of the data retrieve the private key for decryption. We would like separate
> security ACLs around the public versus the private so that we can control
> who can decrypt that data.
>
> I know that the transit secret backend exists, but we would prefer to use
> JOSE for the reason that we do not want the latency of having to hit the
> server every time we want to encrypt and decrypt the message. That
> undesired network overhead that will affect performance, plus additional
> load on the server. With JOSE, we can keep the key cached in memory and
> encrypt and decrypt on the fly, with periodic renewals of the lease being
> the only server communication.
>
> I know we can do the RSA key generation and store the keys in Vault
> ourselves, but we start getting into a key maintenance and revocation
> concern. If something already exists within Vault, we would prefer to use
> it. We would have multiple public/private pairs for use depending on the
> data being encrypted and signed.

I'm honestly not quite understanding what you're looking for here. It
sounds like the things you want are already handled by the generic
backend -- reading and writing of values, with ACLs to control access.

I'm guessing what you're trying to solve is the key maintenance and
revocation capabilities. Can you explain these in more detail?

Thanks,
Jeff

[Andrew Schneider]

Yes, you described it perfectly.  We are trying to solve the key maintenance and revocation capabilities.  We'd like to have Vault generate and store an RSA key and provide an API that we could have publishers request against to get a public key.  Then we could have another API call that consumers of the data could call to get the private key to decrypt the data.  Both APIs could have different ACLs so that data publishers would only have access to public keys and data consumers would only have access to the private keys.

This is primarily for use within a message broker situation where we will have significant traffic flowing through the system.  We want that data to be encrypted in flight and at rest because my company deals with PHI and PII.  We would also potentially use it with something like Redis where it does not provide SSL or encryption at rest natively.  

We don't want to use the transit mechanism to provide the encryption and decryption primarily due to the amount of traffic through the brokers.  JOSE allows in-memory encryption and decryption via the APIs.  If we had a security backend that did what I described above, we could have the key generation, maintenance, and revocation capabilities in Vault, but be able to retrieve the key on startup of the application and utilize the lease time to make the application periodically renew.

--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.

GitHub Issues: https://github.com/hashicorp/vault/issues
IRC: #vault-tool on Freenode
---

You received this message because you are subscribed to a topic in the Google Groups "Vault" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/vault-tool/tf7E7jEmvtM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to vault-tool+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/vault-tool/CAORe8GGOrenQgnNc7o4CCUvL-y-K3mZxp5wQ64PB9f07agyfYw%40mail.gmail.com.

[Jeff Mitchell]

On Mon, May 9, 2016 at 6:50 PM, Andrew Schneider <asch...@gmail.com> wrote:
> Yes, you described it perfectly. We are trying to solve the key maintenance
> and revocation capabilities. We'd like to have Vault generate and store an
> RSA key and provide an API that we could have publishers request against to
> get a public key. Then we could have another API call that consumers of the
> data could call to get the private key to decrypt the data. Both APIs could
> have different ACLs so that data publishers would only have access to public
> keys and data consumers would only have access to the private keys.

It'd be helpful if you described what maintenance you are referring to
and how you expect this revocation strategy to work (stock RSA doesn't
really have any revocation capability).

Is there a reason you can't use transit and just store the decrypted
key in memory? I guess I don't really see the difference between using
an API on this hypothetical backend to fetch a key and storing it in
memory vs. running an encrypted key through transit and storing that
value in memory. Either way it's a single API call.

Best,
Jeff

[Andrew Schneider]

My understanding of transit is that it encrypts/decrypts. Will it give me an actual key, or are you saying once I have a key, use it to encrypt?  If I already have a key, I can just store it in the generic back end for encrypted storage.

We would have multiple keys, so my desire would be able to programmatically request a portion of the key pair. Vault could detect if the key pair does or does not exist, if not, create it and return it.

This isn't just for convenience, it is primarily about security. We most likely will need a man in the middle generating these keys to store in Vault. That would most likely mean at some point or another the key will end up on a hard drive somewhere.  If Vault generates it, it will only ever be in memory or encrypted on disk.

Ultimately, we won't be able to control publishers who cache and publish messages after a "revocation", but our consumers will no longer be able to decrypt the messages if the key is removed or replaced.

I kind of view this as similar to the on demand password or SSH generation Vault already has, except it is creating and delivering an actual portion of a key pair.

--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.

GitHub Issues: https://github.com/hashicorp/vault/issues
IRC: #vault-tool on Freenode
---
You received this message because you are subscribed to a topic in the Google Groups "Vault" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/vault-tool/tf7E7jEmvtM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to vault-tool+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/vault-tool/CAORe8GGKEQqU1_GQxi5Do0BmcKgnFQE3WG2eVchRQAx7%2Bp%3DRwg%40mail.gmail.com.

[Jeff Mitchell]

Hi Andrew,

On Mon, May 9, 2016 at 9:39 PM, Andrew Schneider <asch...@gmail.com> wrote:
> My understanding of transit is that it encrypts/decrypts. Will it give me an
> actual key, or are you saying once I have a key, use it to encrypt? If I
> already have a key, I can just store it in the generic back end for
> encrypted storage.

Right -- I'm saying that if you already have a key, you could encrypt
it via transit if you wanted to. Then users could decrypt it and store
the decrypted value in memory. You wouldn't have to keep passing an
encrypted key through transit because you'd just store the result. Of
course, you can also just store the values in a generic mount and use
ACLs to control access. Transit however can rewrap the encryption on
the key and lock out older versions of the key and if you want to
"revoke" a key you can simply delete the key within transit that is
wrapping it.

> This isn't just for convenience, it is primarily about security. We most
> likely will need a man in the middle generating these keys to store in
> Vault. That would most likely mean at some point or another the key will end
> up on a hard drive somewhere. If Vault generates it, it will only ever be
> in memory or encrypted on disk.
>
> Ultimately, we won't be able to control publishers who cache and publish
> messages after a "revocation", but our consumers will no longer be able to
> decrypt the messages if the key is removed or replaced.

You're talking about giving actual keys out to applications...I don't
think you can really assume that an outside process generating keys
and putting them into Vault will persist the keys to disk (there's
certainly no requirement since it could generate them and send them
into Vault all in memory)...and at the same time, I don't think you
can *not* assume that any given user will simply store the value. In
fact, given the number of users of the keys vs a generator, you're
probably much more likely to have it persisted by a user than the
generator, because the generator would presumably be the only
application allowed to write a key in (using the 'create' capability).
It's far easier to make sure one service is locked down than X
services.

One of the main security benefits of transit is that there is no way
(within the backend) to export a key. Unlike the backend you're
proposing, Vault never hands the key out, so Vault maintains positive
control.

The one exception is transit's data key support which basically works
the way that you describe -- it can create a data key which can then
be used offline for applications that cannot handle. It's wrapped via
a transit key so you can distribute the key with your applications and
have appropriate applications decrypt it and hold the value in memory.
This is basically just a convenience function for a standardized way
of getting a series of random bytes suitable for AES encryption and
having it encrypted by a transit key -- but the transit key is never
revealed. The same model could work for RSA keys, although you'd have
to do the generation -- but doing that fully in-memory is pretty
trivial in most languages.

I'd encourage you to look at the transit backend docs
(https://www.vaultproject.io/docs/secrets/transit/index.html) and
think about the capabilities it offers -- rotation, minimum key
version decryption, etc. I think your use case is very similar to
using datakeys and you can already get the things you want via transit
and potentially a generic mount.

Best,
Jeff

[Andrew Schneider]

You're talking about giving actual keys out to applications...I don't
think you can really assume that an outside process generating keys
and putting them into Vault will persist the keys to disk (there's
certainly no requirement since it could generate them and send them
into Vault all in memory)...and at the same time, I don't think you
can *not* assume that any given user will simply store the value. In
fact, given the number of users of the keys vs a generator, you're
probably much more likely to have it persisted by a user than the
generator, because the generator would presumably be the only
application allowed to write a key in (using the 'create' capability).
It's far easier to make sure one service is locked down than X
services.

Ultimately, we are controlling the library to retrieve keys for encryption and decryption, so we control what is done with those keys.  It's certainly easy to generate a symmetric key on the fly, but with an asymmetric key, it is a little more difficult.  I obviously do not want the message producer to generate the key since they will then have access to the private key portion of the entire key.  We would only want to allow producers access to the public key to strictly encrypt the data.  

So, whether we have a person generate or some sort of self-service functionality, keys would need to be generated prior to actual need.  We could potentially front Vault with a service to retrieve keys and it could generate them on the fly if they do not already exist.  That would basically be writing all of the code and implementing the ACLs that we were thinking would be a potential for Vault(or potentially writing the backend to handle).  We do already secure our services, so it would be possible.  Since we have several other use cases for Vault within our enterprise, we thought it would be easier to simply do the request against an existing secret management tool.

 

The one exception is transit's data key support which basically works
the way that you describe -- it can create a data key which can then
be used offline for applications that cannot handle. It's wrapped via
a transit key so you can distribute the key with your applications and
have appropriate applications decrypt it and hold the value in memory.
This is basically just a convenience function for a standardized way
of getting a series of random bytes suitable for AES encryption and
having it encrypted by a transit key -- but the transit key is never
revealed. The same model could work for RSA keys, although you'd have
to do the generation -- but doing that fully in-memory is pretty
trivial in most languages.

I'd encourage you to look at the transit backend docs
(https://www.vaultproject.io/docs/secrets/transit/index.html) and
think about the capabilities it offers -- rotation, minimum key
version decryption, etc. I think your use case is very similar to
using datakeys and you can already get the things you want via transit
and potentially a generic mount.

Thanks, I had missed the datakeys discussion in the transit documentation when I read it previously.  I do already have a PoC generating an AES key and storing it in Vault via the generic mount.   This is not ideal, because this is the whole symmetric versus asymmetric we have already discussed.  We would still need to have some way of dynamically generating an RSA key and appropriately returning a public or private key, depending on the request.  As you said, the key generation is fairly trivial and we wouldn't really need Vault to handle that for us.  At that point, we could simply store them in the generic mount.

I do like the fact that data encrypted with older versions of the transit keys can still be decrypted with the newer keys.  That's very beneficial functionality.

Looking at your SSH capabilities, you do return RSA private keys already and can obviously handle public keys since they can get added to the server.  This is kind of the capabilities we are looking for with some additional functionality.

This whole conversation may be academic at this point anyway,  as the team I am working on is discussing whether using an AES token for internal application use is appropriate.  We have a way of generating those on the fly at this point and can store in the generic mount with appropriate ACLs.  In the future, if we have external clients who want to publish messages into our message brokers, we will then revisit the RSA public/private key scenario (or whether we even want them publishing directly).

Thanks,

Andrew